Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(msk-alpha): update KafkaVersion #29440

Merged
merged 7 commits into from
Mar 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions packages/@aws-cdk/aws-msk-alpha/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,15 +217,16 @@ You can configure an MSK cluster storage mode using the `storageMode` property.
Tiered storage is a low-cost storage tier for Amazon MSK that scales to virtually unlimited storage,
making it cost-effective to build streaming data applications.

> Visit [Tiered storage](https://docs.aws.amazon.com/msk/latest/developerguide/msk-tiered-storage.html) for more details.
> Visit [Tiered storage](https://docs.aws.amazon.com/msk/latest/developerguide/msk-tiered-storage.html)
to see the list of compatible Kafka versions and for more details.

```ts
declare const vpc: ec2.Vpc;
declare const bucket: s3.IBucket;

const cluster = new msk.Cluster(this, 'cluster', {
clusterName: 'myCluster',
kafkaVersion: msk.KafkaVersion.V2_8_2_TIERED,
kafkaVersion: msk.KafkaVersion.V3_6_0,
vpc,
storageMode: msk.StorageMode.TIERED,
});
Expand Down
35 changes: 34 additions & 1 deletion packages/@aws-cdk/aws-msk-alpha/lib/cluster-version.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,15 @@ export class KafkaVersion {
*/
public static readonly V1_1_1 = KafkaVersion.of('1.1.1');

/**
* **Deprecated by Amazon MSK. You can't create a Kafka cluster with a deprecated version.**
*
* Kafka version 2.1.0
*
* @deprecated use the latest runtime instead
*/
public static readonly V2_1_0 = KafkaVersion.of('2.1.0');

/**
* Kafka version 2.2.1
*/
Expand All @@ -21,6 +30,15 @@ export class KafkaVersion {
*/
public static readonly V2_3_1 = KafkaVersion.of('2.3.1');

/**
* **Deprecated by Amazon MSK. You can't create a Kafka cluster with a deprecated version.**
*
* Kafka version 2.4.1
*
* @deprecated use the latest runtime instead
*/
public static readonly V2_4_1 = KafkaVersion.of('2.4.1');

/**
* Kafka version 2.4.1
*/
Expand Down Expand Up @@ -111,6 +129,11 @@ export class KafkaVersion {
*/
public static readonly V3_5_1 = KafkaVersion.of('3.5.1');

/**
* Kafka version 3.6.0
*/
public static readonly V3_6_0 = KafkaVersion.of('3.6.0');

/**
* Custom cluster version
* @param version custom version number
Expand All @@ -119,6 +142,16 @@ export class KafkaVersion {
return new KafkaVersion(version);
}

/**
* List of Kafka versions that support tiered storage
*
* @see https://docs.aws.amazon.com/msk/latest/developerguide/msk-tiered-storage.html#msk-tiered-storage-requirements
*/
private static readonly TIERED_STORAGE_COMPATIBLE_VERSIONS = [
GavinZZ marked this conversation as resolved.
Show resolved Hide resolved
KafkaVersion.V2_8_2_TIERED,
KafkaVersion.V3_6_0,
].map(({ version }) => version);

/**
*
* @param version cluster version number
Expand All @@ -129,6 +162,6 @@ export class KafkaVersion {
* Checks if the cluster version supports tiered storage mode.
*/
public isTieredStorageCompatible() {
return this.version.endsWith('.tiered');
return KafkaVersion.TIERED_STORAGE_COMPATIBLE_VERSIONS.includes(this.version);
};
}
8 changes: 7 additions & 1 deletion packages/@aws-cdk/aws-msk-alpha/test/cluster.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -790,14 +790,20 @@ describe('MSK Cluster', () => {

describe('created with storage mode', () => {
describe('with tiered storage mode', () => {
test('version.isTieredStorageCompatible', () => {
expect(msk.KafkaVersion.V2_8_2_TIERED.isTieredStorageCompatible()).toBeTruthy();
expect(msk.KafkaVersion.V3_5_1.isTieredStorageCompatible()).toBeFalsy();
expect(msk.KafkaVersion.V3_6_0.isTieredStorageCompatible()).toBeTruthy();
});

test('create a cluster with tiered storage mode', () => {
new msk.Cluster(stack, 'Cluster', {
clusterName: 'cluster',
instanceType: ec2.InstanceType.of(ec2.InstanceClass.M5, ec2.InstanceSize.LARGE),
kafkaVersion: msk.KafkaVersion.V2_8_2_TIERED,
vpc,
storageMode: msk.StorageMode.TIERED,
}),
});
Template.fromStack(stack).hasResourceProperties('AWS::MSK::Cluster', {
StorageMode: 'TIERED',
});
Expand Down