awslabs · krokoko · Oct 12, 2023 · Oct 12, 2023 · Oct 12, 2023 · Oct 12, 2023
@@ -19,6 +19,7 @@
 - [CDK Versions](#cdk-versions)
 - [Contributing](#contributing)
 - [Adding new constructs](#design-guidelines-and-development-guide)
+- [Catalog](#catalog)
 - [Sample Use Cases](#sample-use-cases)
 - [Roadmap](#roadmap)
 
@@ -40,6 +41,17 @@ Contributions of all kinds are welcome! Check out our [contributor's guide](./CO
 
 If you want to add a new construct to the library, check out our first our [design guidelines](./DESIGN_GUIDELINES.md), then follow the [development guide](./DEVELOPER_GUIDE.md)
 
+# Catalog
+
+The following constructs are available in the library:
+
+| **Construct**| Emerging Tech Area |Description| AWS Services used |
+|:-------------|:-------------|:-------------|:-------------|
+| [Data ingestion pipeline](./src/patterns/gen-ai/aws-rag-appsync-stepfn-opensearch/) | Generative AI | Ingestion pipeline providing a RAG (Retrieval Augmented Generation) source for storing documents in a knowledge base. | Amazon Opensearch, AWS Step Functions, Amazon bedrock, AWS Appsync, AWS Lambda | 
+| [Question answering](./src/patterns/gen-ai/aws-qa-appsync-opensearch/) | Generative AI | Question answering against a Large Language Model (Anthropic Claude V2) using a RAG (Retrieval Augmented Generation) source and/or long context window. | Amazon Opensearch, AWS Lambda, Amazon bedrock, AWS Appsync | 
+| [Summarization](./src/patterns/gen-ai/aws-summarization-appsync-stepfn/) | Generative AI | Document summarization using a Large Language Model (Anthropic Claude V2). | AWS Lambda, Amazon bedrock, AWS Appsync, Amazon Elasticache for Redis |
+| [Lambda layer](./src/patterns/gen-ai/aws-langchain-common-layer/) | Generative AI | Python lambda layer providing dependencies and utilities to develop Generative AI applications on AWS. | AWS Lambda, Amazon bedrock, Amazon SageMaker |  
+
 # Sample Use Cases
 
 This library includes a collection of functional use case implementations to demonstrate the usage of AWS Emerging Tech Constructs architectural patterns. These can be used in the same way as architectural patterns, and can be conceptualized as an additional "higher-level" abstraction of those patterns. Those patterns (constructs) are composed together into [stacks](https://docs.aws.amazon.com/cdk/latest/guide/stacks.html), forming a "CDK app".

@@ -55,8 +55,8 @@ Parameters
 
 | **Name**     | **Type**        | **Required** |**Description** |
 |:-------------|:----------------|-----------------|-----------------|
-| openSearchDomainEndpoint | string | ![Required](https://img.shields.io/badge/required-ff0000) | Domain endpoint for the OpenSearch Service. |
-| openSearchDomainName | string| ![Required](https://img.shields.io/badge/required-ff0000) | Domain name for the OpenSearch Service. |
+| existingOpenSearchDomain | [aws_opensearchservice.IDomain](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_opensearchservice.IDomain.html)| ![Required](https://img.shields.io/badge/required-ff0000) | Existing domain for the OpenSearch Service. |
+| openSearchSecret | [aws_secrets_manager.ISecret](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_secretsmanager.ISecret.html) | ![Required](https://img.shields.io/badge/required-ff0000) | Secret containing credentials to authenticate to the existing opensearch domain. |
 | openSearchIndexName | string | ![Required](https://img.shields.io/badge/required-ff0000) | Domain endpoint for the OpenSearch Service. |
 | cognitoUserPool | [cognito.IUserPool](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cognito.IUserPool.html) | ![Required](https://img.shields.io/badge/required-ff0000) | Cognito user pool used for authentication. |
 | vpcProps | [ec2.VpcProps](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ec2.VpcProps.html) | ![Optional](https://img.shields.io/badge/optional-4169E1) | Custom properties for a VPC the construct will create. This VPC will be used by the Lambda functions the construct creates. Providing both this and existingVpc is an error. |
@@ -66,8 +66,7 @@ Parameters
 | existingInputAssetsBucketObj | [s3.IBucket](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_s3.IBucket.html) | ![Optional](https://img.shields.io/badge/optional-4169E1) | Existing instance of S3 Bucket object, providing both this and `bucketInputsAssetsProps` will cause an error. |
 | bucketInputsAssetsProps | [s3.BucketProps](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_s3.BucketProps.html) | ![Optional](https://img.shields.io/badge/optional-4169E1) | User provided props to override the default props for the S3 Bucket. Providing both this and `existingInputAssetsBucketObj` will cause an error. |
 | stage | string | ![Optional](https://img.shields.io/badge/optional-4169E1) | Value will be appended to resources name Service. |
-| mergedApiGraphQLEndpoint | string | ![Optional](https://img.shields.io/badge/optional-4169E1) | URL endpoint of the appsync merged api |
-| mergedApiGraphQLId | string | ![Optional](https://img.shields.io/badge/optional-4169E1) | Api Id of the appsync merged api |
+| existingMergedApi | [appsync.CfnGraphQLApi](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_appsync.CfnGraphQLApi.html) | ![Optional](https://img.shields.io/badge/optional-4169E1) | Existing merged api instance. The merge API provode a federated schema over source API schemas.|
 
 ## Pattern Properties
 

@@ -20,7 +20,9 @@ import * as targets from 'aws-cdk-lib/aws-events-targets';
 import * as iam from 'aws-cdk-lib/aws-iam';
 import * as lambda from 'aws-cdk-lib/aws-lambda';
 import * as logs from 'aws-cdk-lib/aws-logs';
+import * as opensearchservice from 'aws-cdk-lib/aws-opensearchservice';
 import * as s3 from 'aws-cdk-lib/aws-s3';
+import * as secret from 'aws-cdk-lib/aws-secretsmanager';
 import { Construct } from 'constructs';
 import * as s3_bucket_helper from '../../../common/helpers/s3-bucket-helper';
 import * as vpc_helper from '../../../common/helpers/vpc-helper';
@@ -71,41 +73,29 @@ export interface QaAppsyncOpensearchProps {
      */
   readonly bucketInputsAssetsProps?: s3.BucketProps;
   /**
-     * Domain name for the OpenSearch Service.
+     * Existing Amazon OpenSearch Service domain.
      *
      * @default - None
      */
-  readonly openSearchDomainName: string;
+  readonly existinOpensearchDomain: opensearchservice.IDomain;
   /**
-     * Domain endpoint for the OpenSearch Service.
-     *
-     * @default - None
-     */
-  readonly openSearchDomainEndpoint: string;
-  /**
-     * Index name for the OpenSearch Service.
+     * Data Index name for the OpenSearch Service.
      *
      * @default - None
      */
   readonly openSearchIndexName: string;
   /**
-     * SecretsManager secret id to access the OpenSearch Service.
+     * SecretsManager secret to authenticate against the OpenSearch Service domain.
      *
      * @default - None
      */
-  readonly openSearchSecretId: string;
+  readonly openSearchSecret: secret.ISecret;
   /**
-     * URL endpoint of the appsync merged api.
-     *
-     * @default - None
-     */
-  readonly mergedApiGraphQLEndpoint?: string;
-  /**
-     * ApiId of the appsync merged api.
-     *
-     * @default - None
-     */
-  readonly mergedApiGraphQLId?: string;
+   * Existing merged Appsync GraphQL api.
+   *
+   * @default - None
+   */
+  readonly existingMergedApi?: appsync.CfnGraphQLApi;
   /**
      * Cognito user pool used for authentication.
      *
@@ -256,8 +246,8 @@ export class QaAppsyncOpensearch extends Construct {
 
     // If the user provides a mergedApi endpoint, the lambda
     // functions will use this endpoint to send their status updates
-    const updateGraphQlApiEndpoint = !props.mergedApiGraphQLEndpoint ? question_answering_graphql_api.graphqlUrl : props.mergedApiGraphQLEndpoint;
-    const updateGraphQlApiId = !props.mergedApiGraphQLId ? question_answering_graphql_api.apiId : props.mergedApiGraphQLId;
+    const updateGraphQlApiEndpoint = !props.existingMergedApi ? question_answering_graphql_api.graphqlUrl : props.existingMergedApi.attrGraphQlUrl;
+    const updateGraphQlApiId = !props.existingMergedApi ? question_answering_graphql_api.apiId : props.existingMergedApi.attrApiId;
 
     const job_status_data_source = new appsync.NoneDataSource(
       this,
@@ -321,22 +311,25 @@ export class QaAppsyncOpensearch extends Construct {
         environment: {
           GRAPHQL_URL: updateGraphQlApiEndpoint,
           INPUT_BUCKET: this.s3InputAssetsBucketInterface.bucketName,
-          OPENSEARCH_DOMAIN_ENDPOINT: props.openSearchDomainEndpoint,
+          OPENSEARCH_DOMAIN_ENDPOINT: props.existinOpensearchDomain.domainEndpoint,
           OPENSEARCH_INDEX: props.openSearchIndexName,
-          OPENSEARCH_SECRET_ID: props.openSearchSecretId,
+          OPENSEARCH_SECRET_ID: props.openSearchSecret.secretName,
         },
       },
     );
 
+    // The lambda will access the opensearch credentials
+    props.openSearchSecret.grantRead(question_answering_function);
+
     // The lambda will pull processed files and create embeddings
     this.s3InputAssetsBucketInterface.grantRead(question_answering_function);
 
     question_answering_function.addToRolePolicy(new iam.PolicyStatement({
       effect: iam.Effect.ALLOW,
       actions: ['es:*'],
       resources: [
-        'arn:aws:es:'+Aws.REGION+':'+Aws.ACCOUNT_ID+':domain/'+props.openSearchDomainName+'/*',
-        'arn:aws:es:'+Aws.REGION+':'+Aws.ACCOUNT_ID+':domain/'+props.openSearchDomainName,
+        'arn:aws:es:'+Aws.REGION+':'+Aws.ACCOUNT_ID+':domain/'+props.existinOpensearchDomain.domainName+'/*',
+        'arn:aws:es:'+Aws.REGION+':'+Aws.ACCOUNT_ID+':domain/'+props.existinOpensearchDomain.domainName,
       ],
     }));
 

@@ -51,8 +51,8 @@ Parameters
 
 | **Name**     | **Type**        | **Required** |**Description** |
 |:-------------|:----------------|-----------------|-----------------|
-| openSearchDomainEndpoint | string | ![Required](https://img.shields.io/badge/required-ff0000) | Domain endpoint for the OpenSearch Service. |
-| openSearchDomainName | string| ![Required](https://img.shields.io/badge/required-ff0000) | Domain name for the OpenSearch Service. |
+| existingOpenSearchDomain | [aws_opensearchservice.IDomain](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_opensearchservice.IDomain.html)| ![Required](https://img.shields.io/badge/required-ff0000) | Existing domain for the OpenSearch Service. |
+| openSearchSecret | [aws_secrets_manager.ISecret](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_secretsmanager.ISecret.html) | ![Required](https://img.shields.io/badge/required-ff0000) | Secret containing credentials to authenticate to the existing opensearch domain. |
 | openSearchIndexName | string | ![Required](https://img.shields.io/badge/required-ff0000) | Domain endpoint for the OpenSearch Service. |
 | cognitoUserPool | [cognito.IUserPool](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cognito.IUserPool.html) | ![Required](https://img.shields.io/badge/required-ff0000) | Cognito user pool used for authentication. |
 | vpcProps | [ec2.VpcProps](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ec2.VpcProps.html) | ![Optional](https://img.shields.io/badge/optional-4169E1) | Custom properties for a VPC the construct will create. This VPC will be used by the Lambda functions the construct creates. Providing both this and existingVpc is an error. |
@@ -64,7 +64,7 @@ Parameters
 | existingProcessedAssetsBucketObj | [s3.IBucket](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_s3.IBucket.html) | ![Optional](https://img.shields.io/badge/optional-4169E1) | Existing instance of S3 Bucket object, providing both this and `bucketProcessedAssetsProps` will cause an error. |
 | bucketProcessedAssetsProps | [s3.BucketProps](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_s3.BucketProps.html) | ![Optional](https://img.shields.io/badge/optional-4169E1) | User provided props to override the default props for the S3 Bucket. Providing both this and `existingProcessedAssetsBucketObj` will cause an error. |
 | stage | string | ![Optional](https://img.shields.io/badge/optional-4169E1) | Value will be appended to resources name Service. |
-| mergedApiGraphQL | string | ![Optional](https://img.shields.io/badge/optional-4169E1) | URL endpoint of the appsync merged api |
+| existingMergedApi | [appsync.CfnGraphQLApi](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_appsync.CfnGraphQLApi.html) | ![Optional](https://img.shields.io/badge/optional-4169E1) | Existing merged api instance. The merge API provode a federated schema over source API schemas.|
 
 ## Pattern Properties
 

@@ -20,8 +20,9 @@ import * as targets from 'aws-cdk-lib/aws-events-targets';
 import * as iam from 'aws-cdk-lib/aws-iam';
 import * as lambda from 'aws-cdk-lib/aws-lambda';
 import * as logs from 'aws-cdk-lib/aws-logs';
+import * as opensearchservice from 'aws-cdk-lib/aws-opensearchservice';
 import * as s3 from 'aws-cdk-lib/aws-s3';
-import * as secrets from 'aws-cdk-lib/aws-secretsmanager';
+import * as secret from 'aws-cdk-lib/aws-secretsmanager';
 import * as stepfn from 'aws-cdk-lib/aws-stepfunctions';
 import * as stepfn_task from 'aws-cdk-lib/aws-stepfunctions-tasks';
 import { Construct } from 'constructs';
@@ -87,41 +88,29 @@ export interface RagAppsyncStepfnOpensearchProps {
    */
   readonly bucketProcessedAssetsProps?: s3.BucketProps;
   /**
-   * Domain name for the OpenSearch Service.
-   *
-   * @default - None
-   */
-  readonly openSearchDomainName: string;
-  /**
-   * Domain endpoint for the OpenSearch Service.
-   *
-   * @default - None
-   */
-  readonly openSearchDomainEndpoint: string;
+     * Existing Amazon OpenSearch Service domain.
+     *
+     * @default - None
+     */
+  readonly existinOpensearchDomain: opensearchservice.IDomain;
   /**
    * Index name for the OpenSearch Service.
    *
    * @default - None
    */
   readonly openSearchIndexName: string;
   /**
-   * SecretsManager secret id to access the OpenSearch Service.
-   *
-   * @default - None
-   */
-  readonly openSearchSecretId: string;
-  /**
-   * URL endpoint of the appsync merged api.
-   *
-   * @default - None
-   */
-  readonly mergedApiGraphQLEndpoint?: string;
+     * SecretsManager secret to authenticate against the OpenSearch Service domain.
+     *
+     * @default - None
+     */
+  readonly openSearchSecret: secret.ISecret;
   /**
-   * ApiId of the appsync merged api.
+   * Existing merged Appsync GraphQL api.
    *
    * @default - None
    */
-  readonly mergedApiGraphQLId?: string;
+  readonly existingMergedApi?: appsync.CfnGraphQLApi;
   /**
    * Cognito user pool used for authentication.
    *
@@ -317,8 +306,8 @@ export class RagAppsyncStepfnOpensearch extends Construct {
 
     // If the user provides a mergedApi endpoint, the lambda
     // functions will use this endpoint to send their status updates
-    const updateGraphQlApiEndpoint = !props.mergedApiGraphQLEndpoint ? ingestion_graphql_api.graphqlUrl : props.mergedApiGraphQLEndpoint;
-    const updateGraphQlApiId = !props.mergedApiGraphQLId ? ingestion_graphql_api.apiId : props.mergedApiGraphQLId;
+    const updateGraphQlApiEndpoint = !props.existingMergedApi ? ingestion_graphql_api.graphqlUrl : props.existingMergedApi.attrGraphQlUrl;
+    const updateGraphQlApiId = !props.existingMergedApi ? ingestion_graphql_api.apiId : props.existingMergedApi.attrApiId;
 
     const job_status_data_source = new appsync.NoneDataSource(
       this,
@@ -453,15 +442,14 @@ export class RagAppsyncStepfnOpensearch extends Construct {
           OUTPUT_BUCKET: this.s3ProcessedAssetsBucketInterface.bucketName,
           GRAPHQL_URL: updateGraphQlApiEndpoint,
           OPENSEARCH_INDEX: props.openSearchIndexName,
-          OPENSEARCH_DOMAIN_ENDPOINT: props.openSearchDomainEndpoint,
-          OPENSEARCH_SECRET_ID: props.openSearchSecretId,
+          OPENSEARCH_DOMAIN_ENDPOINT: props.existinOpensearchDomain.domainEndpoint,
+          OPENSEARCH_SECRET_ID: props.openSearchSecret.secretName,
         },
       },
     );
 
     // The lambda will access the opensearch credentials
-    const openSearchSecret = secrets.Secret.fromSecretNameV2(this, 'openSearchSecret', props.openSearchSecretId);
-    openSearchSecret.grantRead(embeddings_job_function);
+    props.openSearchSecret.grantRead(embeddings_job_function);
 
     // The lambda will pull processed files and create embeddings
     this.s3ProcessedAssetsBucket?.grantRead(embeddings_job_function);
@@ -470,8 +458,8 @@ export class RagAppsyncStepfnOpensearch extends Construct {
       effect: iam.Effect.ALLOW,
       actions: ['es:*'],
       resources: [
-        'arn:aws:es:'+Aws.REGION+':'+Aws.ACCOUNT_ID+':domain/'+props.openSearchDomainName+'/*',
-        'arn:aws:es:'+Aws.REGION+':'+Aws.ACCOUNT_ID+':domain/'+props.openSearchDomainName,
+        'arn:aws:es:'+Aws.REGION+':'+Aws.ACCOUNT_ID+':domain/'+props.existinOpensearchDomain.domainName+'/*',
+        'arn:aws:es:'+Aws.REGION+':'+Aws.ACCOUNT_ID+':domain/'+props.existinOpensearchDomain.domainName,
       ],
     }));