Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(new construct): aws-fargate-kinesisstreams #877

Merged
merged 13 commits into from
Dec 23, 2022
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
lib/*.js
test/*.js
*.d.ts
coverage
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
lib/*.js
test/*.js
!test/lambda/*
*.js.map
*.ts.map
*.d.ts
node_modules
*.generated.ts
dist
.jsii

.LAST_BUILD
.nyc_output
coverage
.nycrc
.LAST_PACKAGE
*.snk
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Exclude typescript source and config
*.ts
tsconfig.json
coverage
.nyc_output
*.tgz
*.snk
*.tsbuildinfo

# Include javascript files and typescript declarations
!*.js
!*.d.ts

# Exclude jsii outdir
dist

# Include .jsii
!.jsii

# Include .jsii
!.jsii
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

| **Language** | **Package** |
|:-------------|-----------------|
|![Python Logo](https://docs.aws.amazon.com/cdk/api/latest/img/python32.png) Python|`aws_solutions_constructs.aws_fargate_kinesis_stream`|
|![Python Logo](https://docs.aws.amazon.com/cdk/api/latest/img/python32.png) Python|`aws_solutions_constructs.aws_fargate_kinesisstreams`|
|![Typescript Logo](https://docs.aws.amazon.com/cdk/api/latest/img/typescript32.png) Typescript|`@aws-solutions-constructs/aws-fargate-kinesisstreams`|
|![Java Logo](https://docs.aws.amazon.com/cdk/api/latest/img/java32.png) Java|`software.amazon.awsconstructs.services.fargatekinesisstreams`|

Expand All @@ -31,23 +31,22 @@ import { FargateToKinesisStreamsProps } from '@aws-solutions-constructs/aws-farg
import * as fargate from 'aws-cdk-lib/aws-fargate';

new FargateToKinesisStreams(this, 'FargateToKinesisStreams', {
defaultVpcIsIsolated: true,
publicApi: true,
ecrRepositoryArn: "arn:aws:ecr:us-east-1:123456789012:repository/your-ecr-repo",
});
```

Python
``` python
from aws_solutions_constructs.aws_fargate_kinesis_stream import FargateToKinesisStreams
from aws_solutions_constructs.aws_fargate_kinesisstreams import FargateToKinesisStreams
from aws_cdk import (
aws_fargate as _fargate,
aws_kinesis as kinesis,
Stack
)
from constructs import Construct

FargateToKinesisStreams(self, 'FargateToKinesisStreams',
default_vpc_is_isolated=True,
public_api=True,
ecr_repository_arn="arn:aws:ecr:us-east-1:123456789012:repository/your-ecr-repo"
)
```
Expand All @@ -64,7 +63,7 @@ import software.amazon.awscdk.services.fargate.Runtime;
import software.amazon.awsconstructs.services.fargatekinesisstreams.*;

new FargateToKinesisStreams(this, "FargateToKinesisStreams", new FargateToKinesisStreamsProps.Builder()
.defaultVpcIsIsolated(true)
.publicApi(true)
.ecrRepositoryArn("arn:aws:ecr:us-east-1:123456789012:repository/your-ecr-repo")
.build());
```
Expand All @@ -73,7 +72,7 @@ new FargateToKinesisStreams(this, "FargateToKinesisStreams", new FargateToKinesi

| **Name** | **Type** | **Description** |
|:-------------|:----------------|-----------------|
| defaultVpcIsIsolated | `boolean` | True if the VPC provisioned by this construct should contain only Isolated Subnets, otherwise False for the provisioned VPC to contain Public/Private Subnets. Note this property is ignored if an existing VPC is specified in the `existingVpc` property. |
| publicApi | `boolean` | True if the VPC provisioned by this construct should contain only Isolated Subnets, otherwise False for the provisioned VPC to contain Public/Private Subnets. Note this property is ignored if an existing VPC is specified in the `existingVpc` property. |
| vpcProps? | [`ec2.VpcProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ec2.VpcProps.html) | Optional custom properties for a new VPC the construct will create. Providing both this and `existingVpc` is an error. An Amazon Kinesis Streams Interface Endpoint will be added to this VPC. |
| existingVpc? | [`ec2.IVpc`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ec2.IVpc.html) | An existing VPC in which to deploy the Fargate Service. Providing both this and `vpcProps` is an error. If the client provides an existing Fargate Service in the `existingFargateServiceObject` property, this value must be the VPC where the service is running. An Amazon Kinesis Streams Interface Endpoint will be added to this VPC. |
| clusterProps? | [`ecs.ClusterProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs.ClusterProps.html) | Optional properties to create a new ECS cluster. To provide an existing cluster, use the cluster attribute of fargateServiceProps. |
Expand All @@ -94,7 +93,7 @@ new FargateToKinesisStreams(this, "FargateToKinesisStreams", new FargateToKinesi
| **Name** | **Type** | **Description** |
|:-------------|:----------------|-----------------|
| vpc | [`ec2.IVpc`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ec2.IVpc.html) | The new or existing VPC used by the construct. |
| fargateService | [`ecs.FargateService`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs.FargateService.html) | The new or existing AWS Fargate service used by this construct. |
| service | [`ecs.FargateService`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs.FargateService.html) | The new or existing AWS Fargate service used by this construct. |
| container | [`ecs.ContainerDefinition`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs.ContainerDefinition.html) | The container associated with the AWS Fargate service in the service property. |
| kinesisStream | [`kinesis.Stream`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_kinesis.Stream.html) | The new or existing Kinesis Data Stream used by this construct. |
| cloudwatchAlarms? | [`cloudwatch.Alarm[]`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cloudwatch.Alarm.html) | Returns the CloudWatch Alarms created to monitor the Kinesis Data Stream.|
Expand All @@ -109,12 +108,12 @@ Out of the box implementation of the Construct without any overrides will set th
* Sets an Environment Variable named KINESIS_DATASTREAM_NAME that holds the Kinesis Data Stream Name, which is a required property of the Kinesis Data Streams SDK when making calls to it

### Amazon Kinesis Stream
* Enable server-side encryption for the Kinesis Data Stream using AWS Managed CMK
* Enable server-side encryption for the Kinesis Data Stream using an AWS Managed CMK
* Deploy best practices CloudWatch Alarms for the Kinesis Data Stream
* An Interface Endpoint on the VPC for private communication between the Fargate Service and the Kinesis Data Stream

## Architecture
![Architecture Diagram](architecture.png)

***
© Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
© Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
Original file line number Diff line number Diff line change
@@ -0,0 +1,206 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
* with the License. A copy of the License is located at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* or in the 'license' file accompanying this file. This file is distributed on an 'AS IS' BASIS, WITHOUT WARRANTIES
* OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions
* and limitations under the License.
*/

// Imports
import * as cloudwatch from 'aws-cdk-lib/aws-cloudwatch';
import * as ec2 from "aws-cdk-lib/aws-ec2";
import * as ecs from "aws-cdk-lib/aws-ecs";
import * as kinesis from 'aws-cdk-lib/aws-kinesis';
import * as defaults from '@aws-solutions-constructs/core';
// Note: To ensure CDKv2 compatibility, keep the import statement for Construct separate
import { Construct } from 'constructs';

/**
* The properties for the FargateToKinesisStreams class.
*/
export interface FargateToKinesisStreamsProps {
/**
* True if the VPC provisioned by this construct should contain only Isolated Subnets,
georgebearden marked this conversation as resolved.
Show resolved Hide resolved
* otherwise False for the provisioned VPC to contain Public/Private Subnets.
*
* Note this property is ignored if an existing VPC is specified in the `existingVpc` property.
*
* @default - False
*/
readonly publicApi: boolean;
/**
* Optional custom properties for a new VPC the construct will create. Providing both this and `existingVpc` is an error.
*
* An Amazon Kinesis Streams Interface Endpoint will be added to this VPC.
*
* @default - None
*/
readonly vpcProps?: ec2.VpcProps;
/**
* An existing VPC in which to deploy the Fargate Service. Providing both this and `vpcProps` is an error.
* If the client provides an existing Fargate Service in the `existingFargateServiceObject` property,
* this value must be the VPC where the service is running.
*
* An Amazon Kinesis Streams Interface Endpoint will be added to this VPC.
*
* @default - None
*/
readonly existingVpc?: ec2.IVpc;
/**
* Optional properties to create a new ECS cluster. To provide an existing cluster, use the cluster attribute of the `fargateServiceProps` property.
*/
readonly clusterProps?: ecs.ClusterProps;
/**
* The arn of an ECR Repository containing the image to use to generate the containers.
*
* Either this or the image property of the `containerDefinitionProps` property must be provided.
*
* format:
* arn:aws:ecr:[region]:[account number]:repository/[Repository Name]
*/
readonly ecrRepositoryArn?: string;
/**
* The version of the image to use from the repository.
*
* @default - 'latest'
*/
readonly ecrImageVersion?: string;
/*
* Optional props to define the container created for the Fargate Service
*
* defaults - fargate-defaults.ts
*/
readonly containerDefinitionProps?: ecs.ContainerDefinitionProps | any;
/*
* Optional props to define the Fargate Task Definition for this construct
*
* defaults - fargate-defaults.ts
*/
readonly fargateTaskDefinitionProps?: ecs.FargateTaskDefinitionProps | any;
/**
* Optional values to override default Fargate Task definition properties (fargate-defaults.ts).
* The construct will default to launching the service is the most isolated subnets available (precedence: Isolated, Private and Public).
* Override those and other defaults here.
*
* defaults - fargate-defaults.ts
*/
readonly fargateServiceProps?: ecs.FargateServiceProps | any;
/**
* A Fargate Service already instantiated. If this is specified, then no props defining a new service can be provided, including:
* ecrImageVersion, containerDefinitionProps, fargateTaskDefinitionProps, ecrRepositoryArn, fargateServiceProps, clusterProps.
*
* Note - If this property is set, then the `existingContainerDefinitionObject` property must be set as well.
*
* @default - None
*/
readonly existingFargateServiceObject?: ecs.FargateService;
/*
* A container definition already instantiated as part of a Fargate service.
* This must be the container used in the `existingFargateServiceObject` property.
*
* @default - None
*/
readonly existingContainerDefinitionObject?: ecs.ContainerDefinition;
/**
* Existing instance of Kinesis Stream, providing both this and `kinesisStreamProps` will cause an error.
*
* @default - None
*/
readonly existingStreamObj?: kinesis.Stream;
/**
* Optional user-provided props to override the default props for the Kinesis stream.
*
* @default - Default props are used.
*/
readonly kinesisStreamProps?: kinesis.StreamProps;
/**
* Whether to create recommended CloudWatch alarms for the Kinesis Stream.
*
* @default - Alarms are created
*/
readonly createCloudWatchAlarms?: boolean;
/**
* Optional Name to override the Fargate Service default environment variable name that holds the Kinesis Data Stream name value.
*
* @default - KINESIS_DATASTREAM_NAME
*/
readonly streamEnvironmentVariableName?: string;
}

/**
* @summary The FargateToKinesisStream class.
*/
export class FargateToKinesisStreams extends Construct {
public readonly vpc?: ec2.IVpc;
public readonly service: ecs.FargateService;
public readonly container: ecs.ContainerDefinition;
public readonly kinesisStream: kinesis.Stream;
public readonly cloudwatchAlarms?: cloudwatch.Alarm[];

/**
* @summary Constructs a new instance of the KinesisStreamsToFargate class.
* @param {cdk.App} scope - represents the scope for all the resources.
* @param {string} id - this is a a scope-unique id.
* @param {FargateToKinesisStreamsProps} props - user provided props for the construct
* @since 0.8.0
* @access public
*/
constructor(scope: Construct, id: string, props: FargateToKinesisStreamsProps) {
super(scope, id);
defaults.CheckProps(props);
defaults.CheckFargateProps(props);

// Setup the VPC
this.vpc = defaults.buildVpc(scope, {
existingVpc: props.existingVpc,
defaultVpcProps: props.publicApi ? defaults.DefaultPublicPrivateVpcProps() : defaults.DefaultIsolatedVpcProps(),
userVpcProps: props.vpcProps,
constructVpcProps: { enableDnsHostnames: true, enableDnsSupport: true }
});

// Add the interface endpoint to the VPC for Kinesis Streams
defaults.AddAwsServiceEndpoint(scope, this.vpc, defaults.ServiceEndpointTypes.KINESIS_STREAMS);

// Setup the Fargate Service
if (props.existingFargateServiceObject) {
this.service = props.existingFargateServiceObject;
// CheckFargateProps confirms that the container is provided
this.container = props.existingContainerDefinitionObject!;
} else {
[this.service, this.container] = defaults.CreateFargateService(
scope,
id,
this.vpc,
props.clusterProps,
props.ecrRepositoryArn,
props.ecrImageVersion,
props.fargateTaskDefinitionProps,
props.containerDefinitionProps,
props.fargateServiceProps
);
}

// Setup the Kinesis Stream
this.kinesisStream = defaults.buildKinesisStream(this, {
existingStreamObj: props.existingStreamObj,
kinesisStreamProps: props.kinesisStreamProps
});

// Configure container environment variables
const streamNameEnvironmentVariableName = props.streamEnvironmentVariableName || 'KINESIS_DATASTREAM_NAME';
this.container.addEnvironment(streamNameEnvironmentVariableName, this.kinesisStream.streamName);

// Grant the Fargate Service permission to write to the Kinesis Stream
this.kinesisStream.grantWrite(this.service.taskDefinition.taskRole);

// By default, deploy CloudWatch Alarms to monitor the Kinesis Stream
if (props.createCloudWatchAlarms === undefined || props.createCloudWatchAlarms) {
this.cloudwatchAlarms = defaults.buildKinesisStreamCWAlarms(this);
}
}
}
Loading