Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC 510: DynamoDB Global Table L2 #511

Merged
merged 12 commits into from
Jul 3, 2023

Conversation

vinayak-kukreja
Copy link
Contributor

@vinayak-kukreja vinayak-kukreja commented Jun 14, 2023

This is a request for comments about DynamoDB GlobalTable L2 construct. See #510 for additional details.

APIs are signed off by @rix0rrr and @madeline-k .


By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache-2.0 license

Signed-off-by: Vinayak Kukreja <vinakuk@amazon.com>
Signed-off-by: Vinayak Kukreja <vinakuk@amazon.com>
Signed-off-by: Vinayak Kukreja <vinakuk@amazon.com>
Signed-off-by: Vinayak Kukreja <vinakuk@amazon.com>
Signed-off-by: Vinayak Kukreja <vinakuk@amazon.com>
@vinayak-kukreja vinayak-kukreja marked this pull request as ready for review June 15, 2023 07:17
@vinayak-kukreja vinayak-kukreja changed the title RFC 510: DynamoDb Global Table L2 RFC 510: DynamoDB Global Table L2 Jun 15, 2023
text/0510-dynamodb-global-table.md Outdated Show resolved Hide resolved
text/0510-dynamodb-global-table.md Outdated Show resolved Hide resolved
text/0510-dynamodb-global-table.md Outdated Show resolved Hide resolved
text/0510-dynamodb-global-table.md Outdated Show resolved Hide resolved
text/0510-dynamodb-global-table.md Outdated Show resolved Hide resolved
text/0510-dynamodb-global-table.md Show resolved Hide resolved
text/0510-dynamodb-global-table.md Outdated Show resolved Hide resolved
text/0510-dynamodb-global-table.md Outdated Show resolved Hide resolved
text/0510-dynamodb-global-table.md Outdated Show resolved Hide resolved
text/0510-dynamodb-global-table.md Outdated Show resolved Hide resolved
Signed-off-by: Vinayak Kukreja <vinakuk@amazon.com>
Signed-off-by: Vinayak Kukreja <vinakuk@amazon.com>
Copy link
Contributor

@rix0rrr rix0rrr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it! There are three big things left in my mind:

  • I think a GSI probably shouldn't have a default capacity, we should make the user specify it? At least the default for a replica's GSI capacity should be the base GSI, not the replica table.
  • I'd like one final thought to be spent on using enums vs enum-like classes for billing mode (and articulated why you prefer one over the other). I can also see why you might not want to.
  • I'd like an example for encryption, it's not clear to me now whether it's a central property or a per-replica property

text/0510-dynamodb-global-table.md Outdated Show resolved Hide resolved
text/0510-dynamodb-global-table.md Outdated Show resolved Hide resolved
text/0510-dynamodb-global-table.md Outdated Show resolved Hide resolved
text/0510-dynamodb-global-table.md Outdated Show resolved Hide resolved
replicas: [{
region: 'us-east-1',
read: Capacity.autoscaled({ max: 70 }),
globalSecondaryIndexOptions: [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary to "turn on" the GSI for the replica? Or are all GSIs automatically turned on, and this is only necessary to tweak settings?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was there a conscious decision between a list and a map?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not required to turn on the GSIs. They would be enabled for each replica.

This is just to specify some settings for GSIs that are configurable per replica.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was there a conscious decision between a list and a map?

Yes, I was just trying to keep it similar to how GSIs were defined. We can also have it like,

{
	'UniqueGsiName': {
		read: Capacity.fixed(55),
		contributorInsightsEnabled: false,
	}
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

read and contributorInsightsEnabled are the only two props that can be configured for the GSI per replica.

CFN resource: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-dynamodb-globaltable-replicaglobalsecondaryindexspecification.html

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{
	'UniqueGsiName': {
		read: Capacity.fixed(55),
		contributorInsightsEnabled: false,
	}
}

I kind of enjoy the above. Feels like less ceremony to me. No?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, I have just updated the Replica GSI options from list to map. I can see this can also be done for GSI, Replica and LSI. Let me know if that is what you would prefer. :)

For me, it makes sense to have them as list since it conveys intent like, list of global secondary indexes, or list of replicas, but I am fine either way.

* You need a sort key to define a local secondary index.
* You can create up to five local secondary indexes.

#### Encryption
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs an example.

Also above in the RFC you said that kmsKeyArn was a per-replica property, but if the encryption setting is global that might be confusing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will add an example.


The keyArns only need to be provided if the encryption mode is KEY_ARNS. The encryption mode will remain the same for all replicas but the keys would be specific to the replica.

I can instead add a table level property to add [ { region: keyArn} ] but I believe it make more sense to have this in replica props.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again this could be an enum-like class: `

encryption: TableEncryption.regionKeyArns({
  'us-east-1': 'arn:kms:...',
  'us-east-2': 'arn:kms:...',
});

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taking into account that the table itself also needs a key, and that should probably be an IKey for single-region consistency:

encryption: TableEncryption.customerManagedKey(tableKey, {
  replicaKeyArns: {
    'us-east-1': 'arn:kms:...',
    'us-east-2': 'arn:kms:...',
  }
});

Perhaps?

text/0510-dynamodb-global-table.md Outdated Show resolved Hide resolved
text/0510-dynamodb-global-table.md Outdated Show resolved Hide resolved
Signed-off-by: Vinayak Kukreja <vinakuk@amazon.com>
Signed-off-by: Vinayak Kukreja <vinakuk@amazon.com>
Signed-off-by: Vinayak Kukreja <vinakuk@amazon.com>
@rix0rrr rix0rrr added the pr/do-not-merge Let mergify know not to auto merge label Jun 30, 2023
rix0rrr
rix0rrr previously approved these changes Jun 30, 2023
Copy link
Contributor

@rix0rrr rix0rrr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there! A couple of final details to think about, but they're very minor. I like this, after you consider the last couple of points I think this is good to merge.

name: 'FooHashKey',
type: AttributeType.STRING,
},
encryption: GlobalTableEncryption.KEY_ARNS,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget the keyARN for the table itself. Which probably shouldn't be an ARN but an IKey.

* You need a sort key to define a local secondary index.
* You can create up to five local secondary indexes.

#### Encryption
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taking into account that the table itself also needs a key, and that should probably be an IKey for single-region consistency:

encryption: TableEncryption.customerManagedKey(tableKey, {
  replicaKeyArns: {
    'us-east-1': 'arn:kms:...',
    'us-east-2': 'arn:kms:...',
  }
});

Perhaps?

replicas: [
{
region: 'us-west-2',
encryptionKey: 'FooKmsKeyArn',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Encryption is not really relevant for this example, so if you remove it the example will be shorter and easier to read.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will update this.

Signed-off-by: Vinayak Kukreja <vinakuk@amazon.com>
@mergify mergify bot dismissed rix0rrr’s stale review June 30, 2023 22:06

Pull request has been modified.

Signed-off-by: Vinayak Kukreja <vinakuk@amazon.com>
@rix0rrr rix0rrr removed the pr/do-not-merge Let mergify know not to auto merge label Jul 3, 2023
@vinayak-kukreja vinayak-kukreja merged commit 60120d2 into master Jul 3, 2023
1 of 2 checks passed
@vinayak-kukreja vinayak-kukreja deleted the vkukreja/dyanmodb-global-tables branch July 3, 2023 18:07
@evgenyka evgenyka added the l2-request request for new L2 construct label Oct 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
l2-request request for new L2 construct
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants