Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add AwsSigV4 signing functionality #279

Merged
merged 26 commits into from
Sep 7, 2022
Merged

Add AwsSigV4 signing functionality #279

Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
d78ae1c
Add AwsSigV4 signing functionality
harshavamsi Aug 25, 2022
59bbbf4
Adlicense text to signer types
harshavamsi Aug 25, 2022
caf4d74
Pulling aws signer into separate namespace
harshavamsi Aug 25, 2022
7faaae1
Adding separate injection point for v4Signer
harshavamsi Aug 26, 2022
6a67179
Fix name spacing and bump version
harshavamsi Aug 26, 2022
90a443d
Typo in readme
harshavamsi Aug 26, 2022
e50e8fc
Adding 0BSD to allow license
harshavamsi Aug 29, 2022
f58e21e
Split code snippets into USER GUIDE
harshavamsi Aug 29, 2022
f1e5a16
Remove un-used package and update license
harshavamsi Aug 29, 2022
93f347b
Merge branch 'main' into awsv4signer
harshavamsi Aug 29, 2022
6d7b79c
Fix language in user guide
harshavamsi Aug 29, 2022
8505f85
Add types to dev dependencies
harshavamsi Aug 29, 2022
562d683
Update USER_GUIDE.md
harshavamsi Aug 30, 2022
341df4f
add credentials refresh options
rawpixel-vincent Aug 31, 2022
f629fa1
fix AwsSigv4Signer type with Promise
rawpixel-vincent Aug 31, 2022
c952ba0
remove JSDoc
rawpixel-vincent Aug 31, 2022
4abf7ea
update example usage
rawpixel-vincent Sep 1, 2022
48e5b3a
update credentials refresh strategy
rawpixel-vincent Sep 2, 2022
e376ef4
update credentials refresh and expiration
rawpixel-vincent Sep 3, 2022
1475bf0
fix types
rawpixel-vincent Sep 3, 2022
52ab9ad
add failure to refresh credentials test case
rawpixel-vincent Sep 3, 2022
d31eb6f
cleanup and comments
rawpixel-vincent Sep 3, 2022
f036b30
clarify code example in the docs
rawpixel-vincent Sep 6, 2022
ef2b3c7
remove explicit async from code example
rawpixel-vincent Sep 6, 2022
13d30de
remove unused credentialsState.acquiredAt
rawpixel-vincent Sep 6, 2022
b1308b9
Minor doc and misc fixes
harshavamsi Sep 7, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
154 changes: 4 additions & 150 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ OpenSearch Node.js client
- [Welcome!](#welcome)
- [Example use](#example-use)
- [Setup](#setup)
- [Sample code](#sample-code)
- [Sample code](#sample-code)
- [Project Resources](#project-resources)
- [Code of Conduct](#code-of-conduct)
- [License](#license)
Expand Down Expand Up @@ -44,156 +44,10 @@ Then require the client:
const { Client } = require('@opensearch-project/opensearch');
```

### Sample code
## Sample code

Please see the [USER_GUIDE](USER_GUIDE.md) for code snippets.

```javascript
'use strict';

var host = 'localhost';
var protocol = 'https';
var port = 9200;
var auth = 'admin:admin'; // For testing only. Don't store credentials in code.
var ca_certs_path = '/full/path/to/root-ca.pem';

// Optional client certificates if you don't want to use HTTP basic authentication.
// var client_cert_path = '/full/path/to/client.pem'
// var client_key_path = '/full/path/to/client-key.pem'

// Create a client with SSL/TLS enabled.
var { Client } = require('@opensearch-project/opensearch');
var fs = require('fs');
var client = new Client({
node: protocol + '://' + auth + '@' + host + ':' + port,
ssl: {
ca: fs.readFileSync(ca_certs_path),
// You can turn off certificate verification (rejectUnauthorized: false) if you're using self-signed certificates with a hostname mismatch.
// cert: fs.readFileSync(client_cert_path),
// key: fs.readFileSync(client_key_path)
},
});

async function search() {

// Create an index with non-default settings.
var index_name = 'books';
var settings = {
settings: {
index: {
number_of_shards: 4,
number_of_replicas: 3,
},
},
};

var response = await client.indices.create({
index: index_name,
body: settings,
});

console.log('Creating index:');
console.log(response.body);

// Add a document to the index.
var document = {
title: 'The Outsider',
author: 'Stephen King',
year: '2018',
genre: 'Crime fiction',
};

var id = '1';

var response = await client.index({
id: id,
index: index_name,
body: document,
refresh: true,
});

console.log('Adding document:');
console.log(response.body);

// Add documents in bulk
var bulk_documents = [
{
index: {
_index: 'books-king',
_id: '2'
}
},
{
title: 'IT',
author: 'Stephen Kings',
year: '1986',
},
{
create: {
_index: 'test',
_id: '3'
}
},
{
title: 'The Green Mile',
author: 'Stephen Kings',
year: '1996',
},
{
create: {
_index: 'test',
_id: '4'
}
},
{
title: 'Carrie',
author: 'Stephen Kings',
year: '1974',
}
];

var response = await client.bulk({ body: bulk_documents });

console.log('Adding documents using the bulk API')
console.log(response.body);

// Search for a document.
var query = {
query: {
match: {
title: {
query: 'The Outsider',
},
},
},
};

var response = await client.search({
index: index_name,
body: query,
});

console.log('Search results:');
console.log(response.body.hits);

// Delete a document.
var response = await client.delete({
index: index_name,
id: id,
});

console.log('Deleting document:');
console.log(response.body);

// Delete the index.
var response = await client.indices.delete({
index: index_name,
});

console.log('Deleting index:');
console.log(response.body);
}

search().catch(console.log);
```

## Project Resources

Expand Down
171 changes: 171 additions & 0 deletions USER_GUIDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
# User Guide

- [User Guide](#user-guide)
- [Initializing a Client](#initializing-a-client)
- [To authenticate with Amazon OpenSearch Service using AwsSigv4Signer](#to-authenticate-with-amazon-opensearch-service-using-awssigv4signer)
- [Create an Index](#create-an-index)
- [Add a Document to the Index](#add-a-document-to-the-index)
- [Search for the Document](#search-for-the-document)
- [Delete the document](#delete-the-document)
- [Delete the index](#delete-the-index)

## Initializing a Client
```javascript
'use strict';

var host = 'localhost';
var protocol = 'https';
var port = 9200;
var auth = 'admin:admin'; // For testing only. Don't store credentials in code.
var ca_certs_path = '/full/path/to/root-ca.pem';

// Optional client certificates if you don't want to use HTTP basic authentication.
// var client_cert_path = '/full/path/to/client.pem'
// var client_key_path = '/full/path/to/client-key.pem'

// Create a client with SSL/TLS enabled.
var { Client } = require('@opensearch-project/opensearch');
var fs = require('fs');
var client = new Client({
node: protocol + '://' + auth + '@' + host + ':' + port,
ssl: {
ca: fs.readFileSync(ca_certs_path),
// You can turn off certificate verification (rejectUnauthorized: false) if you're using self-signed certificates with a hostname mismatch.
// cert: fs.readFileSync(client_cert_path),
// key: fs.readFileSync(client_key_path)
},
});
```

### To authenticate with [Amazon OpenSearch Service](https://aws.amazon.com/opensearch-service/) using AwsSigv4Signer
harshavamsi marked this conversation as resolved.
Show resolved Hide resolved

```javascript
const AWS = require('aws-sdk'); // V2 SDK.
const { defaultProvider } = require("@aws-sdk/credential-provider-node"); // V3 SDK.
const { Client } = require('@opensearch-project/opensearch');
const { AwsSigv4Signer } = require('@opensearch-project/opensearch/aws');

const client = new Client({
...AwsSigv4Signer({
region: 'us-east-1',
// Must return a Promise that resolve to an AWS.Credentials object.
// This function is used to acquire the credentials when the client start and
// when the credentials are expired.
// The Client will refresh the Credentials only when they are expired.
// With AWS SDK V2, Credentials.refreshPromise is used when available to refresh the credentials.

// Example with AWS SDK V3:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the default? Do I need to always implement getCredentials()?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes there is no default, getCredentials is required.
if not given an error is thrown, it's one of the test case.
I think the wording above make that clear, but it seems not, how can we update it to make it clear that getCredentials() is required?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can provide a default version?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you mean, those two examples are not meant to be copy/pasted, although they would work in the majority of case if people do copy/paste them.
I've tried to make it clear in the comment above how getCredentials() should be implemented.
I think it is good enough, although as discussed we can make it more clear by splitting the code example for v2 and v3.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay I got you, in my opinion giving a default is not the best idea as this is the kind of thing that should be thought out, and it could lead to confusion for consumer using different version of the sdk or just expecting this library to handle the credentials the way they think it should.
a default getCredentials() method could look very different between implementations and there would be good arguments for every implementation as why it should be the default.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that would also mean adding aws-sdk as a peer-dependency / optional-dependency or something like that.. I think it's best to leave it to the users that implement it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @rawpixel-vincent . This allows the user to use the library and implementation of his choice. Looking at community driven implementations of sigv4 there are users using both types of implementations and aws sdk versions.

getCredentials: () => {
// Any other method to acquire a new Credentials object can be used.
const credentialsProvider = defaultProvider();
return credentialsProvider();
},

// Example with AWS SDK V2:
harshavamsi marked this conversation as resolved.
Show resolved Hide resolved
getCredentials: () =>
new Promise((resolve, reject) => {
// Any other method to acquire a new Credentials object can be used.
AWS.config.getCredentials((err, credentials) => {
if (err) {
reject(err);
} else {
resolve(credentials);
}
});
}),
}),
node: "https://search-xxx.region.es.amazonaws.com", // OpenSearch domain URL
});

```

## Create an Index

```javascript
var index_name = 'books';
var settings = {
settings: {
index: {
number_of_shards: 4,
number_of_replicas: 3,
},
},
};

var response = await client.indices.create({
index: index_name,
body: settings,
});

console.log('Creating index:');
harshavamsi marked this conversation as resolved.
Show resolved Hide resolved
console.log(response.body);
```

## Add a Document to the Index

```javascript
var document = {
title: 'The Outsider',
author: 'Stephen King',
year: '2018',
genre: 'Crime fiction',
};

var id = '1';

var response = await client.index({
id: id,
index: index_name,
body: document,
refresh: true,
});

console.log('Adding document:');
console.log(response.body);
```

## Search for the Document

```javascript
var query = {
query: {
match: {
title: {
query: 'The Outsider',
},
},
},
};

var response = await client.search({
index: index_name,
body: query,
});

console.log('Search results:');
console.log(response.body.hits);
```

## Delete the document

```javascript
var response = await client.delete({
index: index_name,
id: id,
});

console.log('Deleting document:');
console.log(response.body);
```

## Delete the index

```javascript
var response = await client.indices.delete({
index: index_name,
});

console.log('Deleting index:');
console.log(response.body);
}
```
Loading