feat(s3-stepfunctions): removed CloudTrail dependency after new S3 feature #529

mickychetta · 2021-12-15T00:49:39Z

Issue #519 , if available:

Description of changes:
-Removed CloudTrail dependency, uses S3 Event Notifications
-Enable EventBridge in S3 Bucket creation
-Updated integ tests and README

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

fixes #519

…onstructs into s3tosf

…-constructs into s3tosf

aws-solutions-constructs-team · 2021-12-15T00:53:11Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v2
Commit ID: 77225ee
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-15T00:58:08Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v2
Commit ID: c3cc329
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-15T00:59:49Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v1
Commit ID: c3cc329
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-15T00:59:52Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v2
Commit ID: 53eb3c5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-15T01:02:32Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v1
Commit ID: 53eb3c5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-15T01:12:06Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v1
Commit ID: 77225ee
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-15T01:40:28Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v2
Commit ID: 00b5f2b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-15T01:42:46Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v1
Commit ID: 00b5f2b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

biffgaut

Changes to the README mean the code needs changing before I review it.

biffgaut · 2021-12-15T16:29:19Z

source/patterns/@aws-solutions-constructs/aws-s3-step-function/README.md

@@ -22,7 +22,7 @@

 This AWS Solutions Construct implements an Amazon S3 bucket connected to an AWS Step Function.

-*Note - This construct uses Amazon EventBridge (Amazon CloudWatch Events) to trigger AWS Step Functions. EventBridge is more flexible, but triggering Step Functions with S3 Event Notifications has less latency and is more cost effective. If cost and/or latency is an issue, you should consider deploy aws-s3-lambda and aws-lambda-stepfunctions in place of this construct.*
+*Note - This construct uses Amazon EventBridge (S3 Event Notifications) to trigger AWS Step Functions State Machine executions. EventBridge is more flexible, but triggering executions with S3 Event Notifications has less latency and is more cost effective. If cost is an issue, you should consider deploying aws-s3-lambda and aws-lambda-stepfunctions in place of this construct.*


This is confusing, because everything is now directly dependent upon S3 Event Notifications, and we use that term to discuss both solutions. Reading between the lines it sounds like the options are:

Send S3 Event Notifications to EventBridge (what this now does) - more flexible, more expensive, more latency

Trigger a Lambda function from S3 Event Notifications, have Lambda function invoke the Step Functions state machine. Less flexible, more work, more cost effective, lower latency.

Let's rewrite this accordingly (and can we confirm that option 2 is still cheaper and faster?)

source/patterns/@aws-solutions-constructs/aws-s3-step-function/README.md

aws-solutions-constructs-team · 2021-12-15T23:22:31Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v1
Commit ID: 6b1e1bf
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-15T23:22:54Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v2
Commit ID: 6b1e1bf
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-16T00:20:30Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v2
Commit ID: f958f5c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-16T00:22:54Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v1
Commit ID: f958f5c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-16T00:30:36Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v2
Commit ID: 73b2a40
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-16T00:32:55Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v1
Commit ID: 73b2a40
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

biffgaut

A couple wording changes in the doc, but the functionality looks right and we can go ahead and code.

source/patterns/@aws-solutions-constructs/aws-s3-step-function/README.md

source/patterns/@aws-solutions-constructs/aws-s3-step-function/lib/index.ts

…-constructs into s3tosf

aws-solutions-constructs-team · 2021-12-16T19:15:20Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v2
Commit ID: ddcd313
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-16T19:18:04Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v1
Commit ID: ddcd313
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-16T19:27:35Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v2
Commit ID: 492ee67
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-16T19:29:48Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v1
Commit ID: 492ee67
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

biffgaut

Couple more wording changes.

source/patterns/@aws-solutions-constructs/aws-s3-step-function/README.md

...ce/patterns/@aws-solutions-constructs/aws-s3-step-function/test/integ.pre-existing-bucket.ts

biffgaut · 2021-12-16T20:02:11Z

source/patterns/@aws-solutions-constructs/aws-s3-step-function/test/s3-step-function.test.ts

@@ -61,18 +44,10 @@ test('override eventRuleProps', () => {
    eventRuleProps: {
      eventPattern: {
        source: ['aws.s3'],
-        detailType: ['AWS API Call via CloudTrail'],
+        detailType: ['Object Created'],


This defines the single event we're currently worried about, correct?

source/patterns/@aws-solutions-constructs/aws-s3-stepfunctions/README.md

source/patterns/@aws-solutions-constructs/aws-s3-stepfunctions/lib/index.ts

…-constructs into s3tosf

aws-solutions-constructs-team · 2021-12-16T21:13:17Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v2
Commit ID: 135a7d4
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-16T21:14:05Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v1
Commit ID: 135a7d4
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-16T21:46:59Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v2
Commit ID: 2347dec
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-16T21:47:30Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v1
Commit ID: 2347dec
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-16T22:22:18Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v2
Commit ID: 4a590c0
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-16T22:24:30Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v1
Commit ID: 4a590c0
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-16T22:31:58Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v2
Commit ID: ddfdaf7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-solutions-constructs-team · 2021-12-16T22:33:45Z

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v1
Commit ID: ddfdaf7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mickychetta added 2 commits December 15, 2021 00:30

enabled eventbridge in bucket creation

77225ee

Merge branch 'main' of https://github.com/mickychetta/aws-solutions-c…

06836d9

…onstructs into s3tosf

mickychetta self-assigned this Dec 15, 2021

mickychetta requested review from biffgaut, hayesry and hnishar as code owners December 15, 2021 00:49

mickychetta and others added 3 commits December 14, 2021 16:49

Merge branch 'main' into s3tosf

c3cc329

updated README

978116b

Merge branch 's3tosf' of https://github.com/mickychetta/aws-solutions…

53eb3c5

…-constructs into s3tosf

empty commit

00b5f2b

biffgaut requested changes Dec 15, 2021

View reviewed changes

updated README and readded deploy CloudTrail

6b1e1bf

fixed eslint errors

f958f5c

Merge branch 'main' into s3tosf

73b2a40

biffgaut requested changes Dec 16, 2021

View reviewed changes

updated README doc by removing properties and improvinng notes

c93c42c

Merge branch 's3tosf' of https://github.com/mickychetta/aws-solutions…

ddcd313

…-constructs into s3tosf

Merge branch 'main' into s3tosf

492ee67

biffgaut requested changes Dec 16, 2021

View reviewed changes

mickychetta added 2 commits December 16, 2021 21:07

updated README and enabled eventbridge in bucket test

85796af

Merge branch 's3tosf' of https://github.com/mickychetta/aws-solutions…

135a7d4

…-constructs into s3tosf

enabled eventbridge in bucket for pre-existing tests

2347dec

updated pre-existing-bucket test

4a590c0

Merge branch 'main' into s3tosf

ddfdaf7

biffgaut approved these changes Dec 16, 2021

View reviewed changes

biffgaut merged commit 639f473 into awslabs:main Dec 16, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(s3-stepfunctions): removed CloudTrail dependency after new S3 feature #529

feat(s3-stepfunctions): removed CloudTrail dependency after new S3 feature #529

mickychetta commented Dec 15, 2021

aws-solutions-constructs-team commented Dec 15, 2021

aws-solutions-constructs-team commented Dec 15, 2021

aws-solutions-constructs-team commented Dec 15, 2021

aws-solutions-constructs-team commented Dec 15, 2021

aws-solutions-constructs-team commented Dec 15, 2021

aws-solutions-constructs-team commented Dec 15, 2021

aws-solutions-constructs-team commented Dec 15, 2021

aws-solutions-constructs-team commented Dec 15, 2021

biffgaut left a comment

biffgaut Dec 15, 2021

aws-solutions-constructs-team commented Dec 15, 2021

aws-solutions-constructs-team commented Dec 15, 2021

aws-solutions-constructs-team commented Dec 16, 2021

aws-solutions-constructs-team commented Dec 16, 2021

aws-solutions-constructs-team commented Dec 16, 2021

aws-solutions-constructs-team commented Dec 16, 2021

biffgaut left a comment

aws-solutions-constructs-team commented Dec 16, 2021

aws-solutions-constructs-team commented Dec 16, 2021

aws-solutions-constructs-team commented Dec 16, 2021

aws-solutions-constructs-team commented Dec 16, 2021

biffgaut left a comment

biffgaut Dec 16, 2021

mickychetta Dec 16, 2021

aws-solutions-constructs-team commented Dec 16, 2021

aws-solutions-constructs-team commented Dec 16, 2021

aws-solutions-constructs-team commented Dec 16, 2021

aws-solutions-constructs-team commented Dec 16, 2021

aws-solutions-constructs-team commented Dec 16, 2021

aws-solutions-constructs-team commented Dec 16, 2021

aws-solutions-constructs-team commented Dec 16, 2021

aws-solutions-constructs-team commented Dec 16, 2021

feat(s3-stepfunctions): removed CloudTrail dependency after new S3 feature #529

feat(s3-stepfunctions): removed CloudTrail dependency after new S3 feature #529

Conversation

mickychetta commented Dec 15, 2021

aws-solutions-constructs-team commented Dec 15, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 15, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 15, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 15, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 15, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 15, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 15, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 15, 2021

AWS CodeBuild CI Report

biffgaut left a comment

Choose a reason for hiding this comment

biffgaut Dec 15, 2021

Choose a reason for hiding this comment

aws-solutions-constructs-team commented Dec 15, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 15, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 16, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 16, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 16, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 16, 2021

AWS CodeBuild CI Report

biffgaut left a comment

Choose a reason for hiding this comment

aws-solutions-constructs-team commented Dec 16, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 16, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 16, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 16, 2021

AWS CodeBuild CI Report

biffgaut left a comment

Choose a reason for hiding this comment

biffgaut Dec 16, 2021

Choose a reason for hiding this comment

mickychetta Dec 16, 2021

Choose a reason for hiding this comment

aws-solutions-constructs-team commented Dec 16, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 16, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 16, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 16, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 16, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 16, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 16, 2021

AWS CodeBuild CI Report

aws-solutions-constructs-team commented Dec 16, 2021

AWS CodeBuild CI Report