fix(ecs-patterns): update constructs for ECS/Fargate consistency (#2795)

As per offline discussions within the team, updating L3 constructs to be consistent across ECS/Fargate launch types as well as within the module itself.

LoadBalancedServices for both ECS and Fargate support the same properties.
QueueWorkerService have been renamed to QueueProcessingService.
Updated unit tests and added missing tests.
Updated README and design docs

BREAKING CHANGE: Renamed QueueWorkerService for base, ec2 and fargate to QueueProcessingService, QueueProcessingEc2Service, and QueueProcessingFargateService.

design/aws-ecs/aws-ecs-autoscaling-queue-worker.md

@@ -1,38 +1,38 @@
 # AWS ECS - L3 Construct for Autoscaling ECS/Fargate Service that Processes Items in a SQS Queue
 
-To address issue [#2396](https://github.com/awslabs/aws-cdk/issues/2396), the AWS ECS CDK construct library should provide a way for customers to create a queue worker service (an AWS ECS/Fargate service that processes items from an sqs queue). This would mean adding new ECS CDK constructs `Ec2QueueWorkerService` and `FargateQueryWorkerService`, that would take in the necessary properties required to create a task definition, an SQS queue as well as an ECS/Fargate service and enable autoscaling for the service based on cpu usage and the SQS queue's approximateNumberOfMessagesVisible metric. 
+To address issue [#2396](https://github.com/awslabs/aws-cdk/issues/2396), the AWS ECS CDK construct library should provide a way for customers to create a queue processing service (an AWS ECS/Fargate service that processes items from an sqs queue). This would mean adding new ECS CDK constructs `QueueProcessingEc2Service` and `QueueProcessingFargateService`, that would take in the necessary properties required to create a task definition, an SQS queue as well as an ECS/Fargate service and enable autoscaling for the service based on cpu usage and the SQS queue's approximateNumberOfMessagesVisible metric. 
 
 ## General approach
 
-The new `ecs.QueueWorkerServiceBase`, `ecs.Ec2QueueWorkerService` and `ecs.FargateQueueWorkerService` classes will create L3 constructs for:
+The new `ecs.QueueProcessingServiceBase`, `ecs.QueueProcessingEc2Service` and `ecs.QueueProcessingFargateService` classes will create L3 constructs for:
 
-* Ec2QueueWorkerService
-* FargateQueueWorkerService
+* QueueProcessingEc2Service
+* QueueProcessingFargateService
 
-A `QueueWorkerService` will create a task definition with the specified container (on both EC2 and Fargate). An AWS SQS `Queue` will be created and autoscaling of the ECS Service will be dependent on both CPU as well as the SQS queue's `ApproximateNumberOfMessagesVisible` metric. 
+A `QueueProcessingService` will create a task definition with the specified container (on both EC2 and Fargate). An AWS SQS `Queue` will be created and autoscaling of the ECS Service will be dependent on both CPU as well as the SQS queue's `ApproximateNumberOfMessagesVisible` metric. 
 
-The `QueueWorkerService` constructs (for EC2 and Fargate) will use the following existing constructs:
+The `QueueProcessingService` constructs (for EC2 and Fargate) will use the following existing constructs:
 
 * Ec2TaskDefinition/FargateTaskDefinition - To create a Task Definition for the container to start
-* SQSQueue - The queue that the worker is processing from
+* SQSQueue - The queue that the service is processing from
 * Ec2Service/FargateService - The Service running the container
 
 ## Code changes
 
-Given the above, we should make the following changes to support queue workers on ECS (for both EC2 and Fargate):
-1. Create `QueueWorkerServiceBaseProps` interface and `QueueWorkerServiceBase` construct
-2. Create `Ec2QueueWorkerServiceProps` interface  and `Ec2QueueWorkerService` construct
-3. Create `FargateQueueWorkerServiceProps` interface  and `FargateQueueWorkerService` construct
+Given the above, we should make the following changes to support queue processing on ECS (for both EC2 and Fargate):
+1. Create `QueueProcessingServiceBaseProps` interface and `QueueProcessingServiceBase` construct
+2. Create `QueueProcessingEc2ServiceProps` interface  and `QueueProcessingEc2Service` construct
+3. Create `QueueProcessingFargateServiceProps` interface  and `QueueProcessingFargateService` construct
 
-### Part 1: Create `QueueWorkerServiceBaseProps` interface  and `QueueWorkerServiceBase` construct  
+### Part 1: Create `QueueProcessingServiceBaseProps` interface  and `QueueProcessingServiceBase` construct  
 
-The `QueueWorkerServiceBaseProps` interface will contain common properties used to construct both the Ec2QueueWorkerService and the FargateQueueWorkerService:
+The `QueueProcessingServiceBaseProps` interface will contain common properties used to construct both the QueueProcessingEc2Service and the QueueProcessingFargateService:
 
 ```ts
 /**
- * Properties to define a Query Worker service
+ * Properties to define a queue processing service
  */
-export interface QueueWorkerServiceBaseProps {
+export interface QueueProcessingServiceBaseProps {
   /**
    * Cluster where service will be deployed
    */
@@ -100,15 +100,15 @@ export interface QueueWorkerServiceBaseProps {
 }
 ```
 
-### Part 2: Create `Ec2QueueWorkerServiceProps` interface  and `Ec2QueueWorkerService` construct  
+### Part 2: Create `QueueProcessingEc2ServiceProps` interface  and `QueueProcessingEc2Service` construct  
 
-The `Ec2QueueWorkerServiceProps` interface will contain properties to construct the Ec2TaskDefinition, SQSQueue and Ec2Service:
+The `QueueProcessingEc2ServiceProps` interface will contain properties to construct the Ec2TaskDefinition, SQSQueue and Ec2Service:
 
 ```ts
 /**
  * Properties to define an ECS service
  */
-export interface Ec2QueueWorkerServiceProps {
+export interface QueueProcessingEc2ServiceProps {
   /**
    * The minimum number of CPU units to reserve for the container.
    *
@@ -146,18 +146,18 @@ export interface Ec2QueueWorkerServiceProps {
 
 An example use case:
 ```ts
-// Create the vpc and cluster used by the Queue Worker task
+// Create the vpc and cluster used by the queue processing service
 const vpc = new ec2.VpcNetwork(stack, 'Vpc', { maxAZs: 1 });
 const cluster = new ecs.Cluster(stack, 'EcsCluster', { vpc });
 cluster.addCapacity('DefaultAutoScalingGroup', {
   instanceType: new ec2.InstanceType('t2.micro')
 });
-const queue = new sqs.Queue(stack, 'WorkerQueue', { 
-  QueueName: 'EcsWorkerQueue'
+const queue = new sqs.Queue(stack, 'ProcessingQueue', { 
+  QueueName: 'EcsEventQueue'
 });
 
-// Create the Queue Worker  task
-new Ec2QueueWorkerService(stack, 'EcsQueueWorkerService', {
+// Create the queue processing service
+new QueueProcessingEc2Service(stack, 'QueueProcessingEc2Service', {
   cluster,
   image: ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'),
   desiredTaskCount: 2,
@@ -168,15 +168,15 @@ new Ec2QueueWorkerService(stack, 'EcsQueueWorkerService', {
 });
 ```
 
-### Part 3: Create `FargateQueueWorkerServiceProps` interface  and `FargateQueueWorkerService` construct  
+### Part 3: Create `QueueProcessingFargateServiceProps` interface  and `QueueProcessingFargateService` construct  
 
-The `FargateQueueWorkerServiceProps` interface will contain properties to construct the FargateTaskDefinition, SQSQueue and FargateService:
+The `QueueProcessingFargateServiceProps` interface will contain properties to construct the FargateTaskDefinition, SQSQueue and FargateService:
 
 ```ts
 /**
- * Properties to define an Fargate service
+ * Properties to define a Fargate service
  */
-export interface FargateQueueWorkerServiceProps {
+export interface QueueProcessingFargateServiceProps {
   /**
    * The number of cpu units used by the task.
    * Valid values, which determines your range of valid values for the memory parameter:
@@ -218,15 +218,15 @@ export interface FargateQueueWorkerServiceProps {
 
 An example use case:
 ```ts
-// Create the vpc and cluster used by the Queue Worker task
+// Create the vpc and cluster used by the queue processing service
 const vpc = new ec2.VpcNetwork(stack, 'Vpc', { maxAZs: 2 });
 const cluster = new ecs.Cluster(stack, 'FargateCluster', { vpc });
-const queue = new sqs.Queue(stack, 'WorkerQueue', { 
-  QueueName: 'FargateWorkerQueue'
+const queue = new sqs.Queue(stack, 'ProcessingQueue', { 
+  QueueName: 'FargateEventQueue'
 });
 
-// Create the Queue Worker task
-new FargateQueueWorkerService(stack, 'FargateQueueWorkerService', {
+// Create the queue processing service
+new QueueProcessingFargateService(stack, 'QueueProcessingFargateService', {
   cluster,
   image: ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'),
   desiredTaskCount: 2,

packages/@aws-cdk/aws-ecs-patterns/README.md

@@ -15,7 +15,7 @@
 This library provides higher-level ECS constructs which follow common architectural patterns. It contains:
 
 * Load Balanced Services
-* Queue Worker Services
+* Queue Processing Services
 * Scheduled Tasks (cron jobs)
 
 ## Load Balanced Services
@@ -48,14 +48,14 @@ const loadBalancedFargateService = new ecsPatterns.LoadBalancedFargateService(st
 });
 ```
 
-## Queue Worker Services
+## Queue Processing Services
 
 To define a service that creates a queue and reads from that queue, instantiate one of the following:
 
-* `Ec2QueueWorkerService`
+* `QueueProcessingEc2Service`
 
 ```ts
-const ecsQueueWorkerService = new Ec2QueueWorkerService(stack, 'Service', {
+const queueProcessingEc2Service = new QueueProcessingEc2Service(stack, 'Service', {
   cluster,
   memoryLimitMiB: 1024,
   image: ecs.ContainerImage.fromRegistry('test'),
@@ -71,10 +71,10 @@ const ecsQueueWorkerService = new Ec2QueueWorkerService(stack, 'Service', {
 });
 ```
 
-* `FargateQueueWorkerService`
+* `QueueProcessingFargateService`
 
 ```ts
-const fargateQueueWorkerService = new FargateQueueWorkerService(stack, 'Service', {
+const queueProcessingFargateService = new QueueProcessingFargateService(stack, 'Service', {
   cluster,
   memoryMiB: '512',
   image: ecs.ContainerImage.fromRegistry('test'),

packages/@aws-cdk/aws-ecs-patterns/lib/base/load-balanced-service-base.ts

@@ -1,13 +1,18 @@
 import { ICertificate } from '@aws-cdk/aws-certificatemanager';
 import ecs = require('@aws-cdk/aws-ecs');
 import elbv2 = require('@aws-cdk/aws-elasticloadbalancingv2');
+import { AddressRecordTarget, ARecord, IHostedZone } from '@aws-cdk/aws-route53';
+import route53targets = require('@aws-cdk/aws-route53-targets');
 import cdk = require('@aws-cdk/cdk');
 
 export enum LoadBalancerType {
   Application,
   Network
 }
 
+/**
+ * Base properties for load-balanced Fargate and ECS services
+ */
 export interface LoadBalancedServiceBaseProps {
   /**
    * The cluster where your service will be deployed
@@ -61,10 +66,38 @@ export interface LoadBalancedServiceBaseProps {
    * @default - No environment variables.
    */
   readonly environment?: { [key: string]: string };
+
+  /**
+   * Whether to create an AWS log driver
+   *
+   * @default true
+   */
+  readonly enableLogging?: boolean;
+
+  /**
+   * Determines whether your Fargate Service will be assigned a public IP address.
+   *
+   * @default false
+   */
+  readonly publicTasks?: boolean;
+
+  /**
+   * Domain name for the service, e.g. api.example.com
+   *
+   * @default - No domain name.
+   */
+  readonly domainName?: string;
+
+  /**
+   * Route53 hosted zone for the domain, e.g. "example.com."
+   *
+   * @default - No Route53 hosted domain zone.
+   */
+  readonly domainZone?: IHostedZone;
 }
 
 /**
- * Base class for load-balanced Fargate and ECS service
+ * Base class for load-balanced Fargate and ECS services
  */
 export abstract class LoadBalancedServiceBase extends cdk.Construct {
   public readonly loadBalancerType: LoadBalancerType;
@@ -75,9 +108,15 @@ export abstract class LoadBalancedServiceBase extends cdk.Construct {
 
   public readonly targetGroup: elbv2.ApplicationTargetGroup | elbv2.NetworkTargetGroup;
 
+  public readonly logDriver?: ecs.LogDriver;
+
   constructor(scope: cdk.Construct, id: string, props: LoadBalancedServiceBaseProps) {
     super(scope, id);
 
+    // Create log driver if logging is enabled
+    const enableLogging = props.enableLogging !== undefined ? props.enableLogging : true;
+    this.logDriver = enableLogging ? this.createAWSLogDriver(this.node.id) : undefined;
+
     // Load balancer
     this.loadBalancerType = props.loadBalancerType !== undefined ? props.loadBalancerType : LoadBalancerType.Application;
 
@@ -122,6 +161,18 @@ export abstract class LoadBalancedServiceBase extends cdk.Construct {
       this.targetGroup = this.listener.addTargets('ECS', targetProps);
     }
 
+    if (typeof props.domainName !== 'undefined') {
+      if (typeof props.domainZone === 'undefined') {
+        throw new Error('A Route53 hosted domain zone name is required to configure the specified domain name');
+      }
+
+      new ARecord(this, "DNS", {
+        zone: props.domainZone,
+        recordName: props.domainName,
+        target: AddressRecordTarget.fromAlias(new route53targets.LoadBalancerTarget(this.loadBalancer)),
+      });
+    }
+
     new cdk.CfnOutput(this, 'LoadBalancerDNS', { value: this.loadBalancer.loadBalancerDnsName });
   }
 
@@ -132,4 +183,8 @@ export abstract class LoadBalancedServiceBase extends cdk.Construct {
       (this.targetGroup as elbv2.NetworkTargetGroup).addTarget(service);
     }
   }
+
+  private createAWSLogDriver(prefix: string): ecs.AwsLogDriver {
+    return new ecs.AwsLogDriver(this, 'Logging', { streamPrefix: prefix });
+  }
 }

packages/@aws-cdk/aws-ecs-patterns/lib/base/queue-worker-service-base.ts → packages/@aws-cdk/aws-ecs-patterns/lib/base/queue-processing-service-base.ts

@@ -4,9 +4,9 @@ import sqs = require('@aws-cdk/aws-sqs');
 import cdk = require('@aws-cdk/cdk');
 
 /**
- * Properties to define a queue worker service
+ * Properties to define a queue processing service
  */
-export interface QueueWorkerServiceBaseProps {
+export interface QueueProcessingServiceBaseProps {
   /**
    * Cluster where service will be deployed
    */
@@ -74,15 +74,15 @@ export interface QueueWorkerServiceBaseProps {
 }
 
 /**
- * Base class for a Fargate and ECS queue worker service
+ * Base class for a Fargate and ECS queue processing service
  */
-export abstract class QueueWorkerServiceBase extends cdk.Construct {
+export abstract class QueueProcessingServiceBase extends cdk.Construct {
   /**
-   * The SQS queue that the worker service will process from
+   * The SQS queue that the service will process from
    */
   public readonly sqsQueue: sqs.IQueue;
 
-  // Properties that have defaults defined. The Queue Worker will handle assigning undefined properties with default
+  // Properties that have defaults defined. The Queue Processing Service will handle assigning undefined properties with default
   // values so that derived classes do not need to maintain the same logic.
 
   /**
@@ -106,19 +106,19 @@ export abstract class QueueWorkerServiceBase extends cdk.Construct {
    */
   public readonly logDriver?: ecs.LogDriver;
 
-  constructor(scope: cdk.Construct, id: string, props: QueueWorkerServiceBaseProps) {
+  constructor(scope: cdk.Construct, id: string, props: QueueProcessingServiceBaseProps) {
     super(scope, id);
 
-    // Create the worker SQS queue if one is not provided
-    this.sqsQueue = props.queue !== undefined ? props.queue : new sqs.Queue(this, 'EcsWorkerServiceQueue', {});
+    // Create the SQS queue if one is not provided
+    this.sqsQueue = props.queue !== undefined ? props.queue : new sqs.Queue(this, 'EcsProcessingQueue', {});
 
     // Setup autoscaling scaling intervals
     const defaultScalingSteps = [{ upper: 0, change: -1 }, { lower: 100, change: +1 }, { lower: 500, change: +5 }];
     this.scalingSteps = props.scalingSteps !== undefined ? props.scalingSteps : defaultScalingSteps;
 
     // Create log driver if logging is enabled
     const enableLogging = props.enableLogging !== undefined ? props.enableLogging : true;
-    this.logDriver = enableLogging ? this.createAwsLogDriver(this.node.id) : undefined;
+    this.logDriver = enableLogging ? this.createAWSLogDriver(this.node.id) : undefined;
 
     // Add the queue name to environment variables
     this.environment = { ...(props.environment || {}), QUEUE_NAME: this.sqsQueue.queueName };
@@ -152,7 +152,7 @@ export abstract class QueueWorkerServiceBase extends cdk.Construct {
    *
    * @param prefix the Cloudwatch logging prefix
    */
-  private createAwsLogDriver(prefix: string): ecs.AwsLogDriver {
-    return new ecs.AwsLogDriver(this, 'QueueWorkerLogging', { streamPrefix: prefix });
+  private createAWSLogDriver(prefix: string): ecs.AwsLogDriver {
+    return new ecs.AwsLogDriver(this, 'ProcessingContainerLogging', { streamPrefix: prefix });
   }
 }

packages/@aws-cdk/aws-ecs-patterns/lib/ecs/load-balanced-ecs-service.ts

@@ -52,17 +52,20 @@ export class LoadBalancedEc2Service extends LoadBalancedServiceBase {
       image: props.image,
       memoryLimitMiB: props.memoryLimitMiB,
       memoryReservationMiB: props.memoryReservationMiB,
-      environment: props.environment
+      environment: props.environment,
+      logging: this.logDriver,
     });
 
     container.addPortMappings({
       containerPort: props.containerPort || 80
     });
 
+    const assignPublicIp = props.publicTasks !== undefined ? props.publicTasks : false;
     const service = new ecs.Ec2Service(this, "Service", {
       cluster: props.cluster,
       desiredCount: props.desiredCount || 1,
-      taskDefinition
+      taskDefinition,
+      assignPublicIp
     });
 
     this.service = service;