Skip to content

Commit

Permalink
fix(custom-resource-handlers): don't recursively process s3 bucket ob…
Browse files Browse the repository at this point in the history
…jects (aws#30209)

Fixes aws#30573.

I recently had the mispleasure of trying to empty a bucket with ~600000 objects using CDK's `autoDeleteObjects` feature. What I observed was that each lambda invocation would get through a few tens of thousands of objects in relatively good time (a few minutes), then the lambda would grind to a halt doing very little until it reached its 15 minute timeout. This process then repeats with subsequent invocations of the lambda.  I had to empty the bucket in the web console to make real progress toward deleting the bucket.

I have proven that the low memory allocated to the lambda (the default 128mb) plus this recursion is to blame. There is no need to recurse, and doing so will put pressure on the stack, the heap, and (because this is an async function) the event loop.

Switch the recursion to iteration.

aws#30209 (comment)

- [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md)

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
  • Loading branch information
isker authored and Raviteja Saranga committed Jun 21, 2024
1 parent 5c65250 commit 9c6bf72
Show file tree
Hide file tree
Showing 164 changed files with 36,579 additions and 7,007 deletions.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@
"S3Bucket": {
"Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
},
"S3Key": "96ea260348625427bb7f28ab3b379f62da6f428bcd155cb13f9261aa6a404b0d.zip"
"S3Key": "44e9c4d7a5d3fd2d677e1a7e416b2b56f6b0104bd5eff9cac5557b4c65a9dc61.zip"
},
"Timeout": 900,
"MemorySize": 128,
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

This file was deleted.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@
"S3Bucket": {
"Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
},
"S3Key": "96ea260348625427bb7f28ab3b379f62da6f428bcd155cb13f9261aa6a404b0d.zip"
"S3Key": "44e9c4d7a5d3fd2d677e1a7e416b2b56f6b0104bd5eff9cac5557b4c65a9dc61.zip"
},
"Timeout": 900,
"MemorySize": 128,
Expand Down Expand Up @@ -449,7 +449,7 @@
"Value": "AWSServiceRoleForElasticBeanstalkManagedUpdates"
}
],
"SolutionStackName": "64bit Amazon Linux 2023 v6.1.2 running Node.js 20"
"SolutionStackName": "64bit Amazon Linux 2023 v6.1.6 running Node.js 20"
},
"DependsOn": [
"beastalkapp",
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ const beanstalkEnv = new elasticbeanstalk.CfnEnvironment(stack, 'beanstlk-env',
applicationName: beanstalkApp.applicationName!,
environmentName: 'codepipeline-test-env',
// see https://docs.aws.amazon.com/elasticbeanstalk/latest/platforms/platforms-supported.html#platforms-supported.nodejs
solutionStackName: '64bit Amazon Linux 2023 v6.1.2 running Node.js 20',
solutionStackName: '64bit Amazon Linux 2023 v6.1.6 running Node.js 20',
optionSettings: [
{
namespace: 'aws:autoscaling:launchconfiguration',
Expand Down
Loading

0 comments on commit 9c6bf72

Please sign in to comment.