Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(cli): --concurrency option (aws#20345)
Closes aws#1973 (reattempting aws#19378) Add `--concurrency` parameter to `cdk deploy` command to enable concurrent deployments while respecting stack dependencies. Concurrency mode will only work with `--progress events` due to the poor interaction of concurrent deployments and the progress bar rendering. ### Open Questions - [x] How best to write automated tests around this? Added unit and integration tests - [x] Should other commands (ex: `destroy`) support concurrency? Only supporting deploy command in this PR - [x] Any other concerns with this approach as it changes a key component of `cdk` - [x] How should this work with the `--exclusively` flag? Only check dependencies between requested stacks <details> <summary>Example Output (Success):</summary> <pre> $ yarn cdk deploy --all --require-approval "never" --concurrency 3 --progress bar yarn run v1.22.17 warning package.json: No license field $ cdk deploy --all --require-approval never --concurrency 3 --progress bar ✨ Synthesis time: 16.04s⚠️ The --concurrency flag only supports --progress "events". Switching to "events". relm-test-1 relm-test-1: deploying... relm-test-2: deploying... relm-test-2: creating CloudFormation changeset... relm-test-1: creating CloudFormation changeset... relm-test-1 | 0/3 | 4:57:19 PM | REVIEW_IN_PROGRESS | AWS::CloudFormation::Stack | relm-test-1 User Initiated relm-test-1 | 0/3 | 4:57:24 PM | CREATE_IN_PROGRESS | AWS::CloudFormation::Stack | relm-test-1 User Initiated relm-test-1 | 0/3 | 4:57:30 PM | CREATE_IN_PROGRESS | AWS::SNS::Topic | TempTopic (TempTopic9C0CBD7C) relm-test-2 | 0/3 | 4:57:19 PM | REVIEW_IN_PROGRESS | AWS::CloudFormation::Stack | relm-test-2 User Initiated relm-test-2 | 0/3 | 4:57:24 PM | CREATE_IN_PROGRESS | AWS::CloudFormation::Stack | relm-test-2 User Initiated relm-test-2 | 0/3 | 4:57:29 PM | CREATE_IN_PROGRESS | AWS::SNS::Topic | TempTopic (TempTopic9C0CBD7C) relm-test-2 | 0/3 | 4:57:29 PM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) relm-test-1 | 0/3 | 4:57:30 PM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) relm-test-1 | 0/3 | 4:57:30 PM | CREATE_IN_PROGRESS | AWS::SNS::Topic | TempTopic (TempTopic9C0CBD7C) Resource creation Initiated relm-test-1 | 0/3 | 4:57:32 PM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) Resource creation Initiated relm-test-1 | 1/3 | 4:57:33 PM | CREATE_COMPLETE | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) relm-test-2 | 0/3 | 4:57:30 PM | CREATE_IN_PROGRESS | AWS::SNS::Topic | TempTopic (TempTopic9C0CBD7C) Resource creation Initiated relm-test-2 | 0/3 | 4:57:31 PM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) Resource creation Initiated relm-test-2 | 1/3 | 4:57:32 PM | CREATE_COMPLETE | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) relm-test-2 | 2/3 | 4:57:41 PM | CREATE_COMPLETE | AWS::SNS::Topic | TempTopic (TempTopic9C0CBD7C) relm-test-2 | 3/3 | 4:57:43 PM | CREATE_COMPLETE | AWS::CloudFormation::Stack | relm-test-2 relm-test-1 | 2/3 | 4:57:41 PM | CREATE_COMPLETE | AWS::SNS::Topic | TempTopic (TempTopic9C0CBD7C) relm-test-1 | 3/3 | 4:57:43 PM | CREATE_COMPLETE | AWS::CloudFormation::Stack | relm-test-1 ✅ relm-test-1 ✨ Deployment time: 28.39s Stack ARN: arn:aws:cloudformation:us-east-1:XXXXXXXXXXXX:stack/relm-test-1/ffcdb210-a6fd-11ec-b05c-0a02909bf4fb ✨ Total time: 44.43s ✅ relm-test-2 ✨ Deployment time: 28.45s Outputs: relm-test-2.ExportsOutputRefTempTopic9C0CBD7CC97C6BE5 = arn:aws:sns:us-east-1:XXXXXXXXXXXX:relm-test-2-TempTopic9C0CBD7C-2ETJWELY3MFD Stack ARN: arn:aws:cloudformation:us-east-1:XXXXXXXXXXXX:stack/relm-test-2/ffbc7400-a6fd-11ec-ac95-0ea201c9d581 ✨ Total time: 44.49s relm-test-2B relm-test-2B: deploying... relm-test-2B: creating CloudFormation changeset... relm-test-2B | 0/4 | 4:57:47 PM | REVIEW_IN_PROGRESS | AWS::CloudFormation::Stack | relm-test-2B User Initiated relm-test-2B | 0/4 | 4:57:58 PM | CREATE_IN_PROGRESS | AWS::CloudFormation::Stack | relm-test-2B User Initiated relm-test-2B | 0/4 | 4:58:04 PM | CREATE_IN_PROGRESS | AWS::IAM::Role | TopicRole (TopicRole3526982D) relm-test-2B | 0/4 | 4:58:04 PM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) relm-test-2B | 0/4 | 4:58:05 PM | CREATE_IN_PROGRESS | AWS::IAM::Role | TopicRole (TopicRole3526982D) Resource creation Initiated relm-test-2B | 0/4 | 4:58:06 PM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) Resource creation Initiated relm-test-2B | 1/4 | 4:58:07 PM | CREATE_COMPLETE | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) relm-test-2B | 2/4 | 4:58:19 PM | CREATE_COMPLETE | AWS::IAM::Role | TopicRole (TopicRole3526982D) relm-test-2B | 2/4 | 4:58:22 PM | CREATE_IN_PROGRESS | AWS::IAM::Policy | TopicRole/DefaultPolicy (TopicRoleDefaultPolicy489E2B68) relm-test-2B | 2/4 | 4:58:23 PM | CREATE_IN_PROGRESS | AWS::IAM::Policy | TopicRole/DefaultPolicy (TopicRoleDefaultPolicy489E2B68) Resource creation Initiated relm-test-2B | 3/4 | 4:58:36 PM | CREATE_COMPLETE | AWS::IAM::Policy | TopicRole/DefaultPolicy (TopicRoleDefaultPolicy489E2B68) relm-test-2B | 4/4 | 4:58:37 PM | CREATE_COMPLETE | AWS::CloudFormation::Stack | relm-test-2B ✅ relm-test-2B ✨ Deployment time: 54.71s Stack ARN: arn:aws:cloudformation:us-east-1:XXXXXXXXXXXX:stack/relm-test-2B/10b341d0-a6fe-11ec-a535-12d49b16bf9b ✨ Total time: 70.75s ✨ Done in 99.79s. </pre> </details> <details> <summary>Example Output (Failure):</summary> <pre> yarn run v1.22.18 warning package.json: No license field $ cdk deploy --all --require-approval never --concurrency 3 --progress bar ✨ Synthesis time: 16.18s⚠️ The --concurrency flag only supports --progress "events". Switching to "events". relm-broken-1 relm-broken-1: deploying... relm-broken-2 relm-broken-2: deploying... relm-test-1: deploying... relm-test-1: creating CloudFormation changeset... relm-broken-1: creating CloudFormation changeset... relm-broken-2: creating CloudFormation changeset... relm-broken-2 | 0/3 | 9:12:48 PM | REVIEW_IN_PROGRESS | AWS::CloudFormation::Stack | relm-broken-2 User Initiated relm-broken-2 | 0/3 | 9:12:53 PM | CREATE_IN_PROGRESS | AWS::CloudFormation::Stack | relm-broken-2 User Initiated relm-broken-1 | 0/3 | 9:12:48 PM | REVIEW_IN_PROGRESS | AWS::CloudFormation::Stack | relm-broken-1 User Initiated relm-broken-1 | 0/3 | 9:12:53 PM | CREATE_IN_PROGRESS | AWS::CloudFormation::Stack | relm-broken-1 User Initiated relm-test-1 | 0/3 | 9:12:48 PM | REVIEW_IN_PROGRESS | AWS::CloudFormation::Stack | relm-test-1 User Initiated relm-test-1 | 0/3 | 9:12:54 PM | CREATE_IN_PROGRESS | AWS::CloudFormation::Stack | relm-test-1 User Initiated relm-test-1 | 0/3 | 9:12:59 PM | CREATE_IN_PROGRESS | AWS::SNS::Topic | TestTopic (TestTopic339EC197) relm-test-1 | 0/3 | 9:12:59 PM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) relm-test-1 | 0/3 | 9:13:00 PM | CREATE_IN_PROGRESS | AWS::SNS::Topic | TestTopic (TestTopic339EC197) Resource creation Initiated relm-test-1 | 0/3 | 9:13:01 PM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) Resource creation Initiated relm-test-1 | 1/3 | 9:13:02 PM | CREATE_COMPLETE | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) relm-broken-2 | 0/3 | 9:12:59 PM | CREATE_IN_PROGRESS | AWS::SNS::Topic | TestTopic (TestTopic339EC197) relm-broken-2 | 0/3 | 9:12:59 PM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) relm-broken-2 | 0/3 | 9:13:00 PM | CREATE_IN_PROGRESS | AWS::SNS::Topic | TestTopic (TestTopic339EC197) Resource creation Initiated relm-broken-2 | 0/3 | 9:13:02 PM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) Resource creation Initiated relm-broken-2 | 1/3 | 9:13:02 PM | CREATE_COMPLETE | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) relm-broken-1 | 0/3 | 9:13:04 PM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) relm-broken-1 | 0/3 | 9:13:04 PM | CREATE_IN_PROGRESS | AWS::SNS::Topic | TestTopic (TestTopic339EC197) relm-broken-1 | 0/3 | 9:13:04 PM | CREATE_FAILED | AWS::SNS::Topic | TestTopic (TestTopic339EC197) TestTopic already exists in stack arn:aws:cloudformation:us-east-1:__ACCOUNT_ID__:stack/relm-broken-2/79fc1770-cc10-11ec-8279-0a02b75d1237 new Topic (/Users/relm/Development/aws-cdk/packages/aws-cdk-lib/aws-sns/lib/topic.ts:102:22) \_ new BrokenStack (/Users/relm/Development/aws-cdk-test-stack/lib/broken-stack.ts:11:18) \_ Object.<anonymous> (/Users/relm/Development/aws-cdk-test-stack/bin/aws-cdk-test-stack.ts:12:1) \_ Module._compile (node:internal/modules/cjs/loader:1103:14) \_ Module.m._compile (/Users/relm/Development/aws-cdk-test-stack/node_modules/ts-node/src/index.ts:1455:23) \_ Module._extensions..js (node:internal/modules/cjs/loader:1157:10) \_ Object.require.extensions.<computed> [as .ts] (/Users/relm/Development/aws-cdk-test-stack/node_modules/ts-node/src/index.ts:1458:12) \_ Module.load (node:internal/modules/cjs/loader:981:32) \_ Function.Module._load (node:internal/modules/cjs/loader:822:12) \_ Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12) \_ phase4 (/Users/relm/Development/aws-cdk-test-stack/node_modules/ts-node/src/bin.ts:567:12) \_ bootstrap (/Users/relm/Development/aws-cdk-test-stack/node_modules/ts-node/src/bin.ts:85:10) \_ main (/Users/relm/Development/aws-cdk-test-stack/node_modules/ts-node/src/bin.ts:54:10) \_ Object.<anonymous> (/Users/relm/Development/aws-cdk-test-stack/node_modules/ts-node/src/bin.ts:717:3) \_ Module._compile (node:internal/modules/cjs/loader:1103:14) \_ Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10) \_ Module.load (node:internal/modules/cjs/loader:981:32) \_ Function.Module._load (node:internal/modules/cjs/loader:822:12) \_ Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12) \_ node:internal/main/run_main_module:17:47 relm-broken-1 | 0/3 | 9:13:05 PM | CREATE_FAILED | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) Resource creation cancelled new MetadataResource (/Users/relm/Development/aws-cdk/packages/aws-cdk-lib/core/lib/private/metadata-resource.ts:22:24) \_ /Users/relm/Development/aws-cdk/packages/aws-cdk-lib/core/lib/private/synthesis.ts:166:5 \_ visit (/Users/relm/Development/aws-cdk/packages/aws-cdk-lib/core/lib/private/synthesis.ts:231:5) \_ visit (/Users/relm/Development/aws-cdk/packages/aws-cdk-lib/core/lib/private/synthesis.ts:227:5) \_ injectMetadataResources (/Users/relm/Development/aws-cdk/packages/aws-cdk-lib/core/lib/private/synthesis.ts:157:3) \_ Object.synthesize (/Users/relm/Development/aws-cdk/packages/aws-cdk-lib/core/lib/private/synthesis.ts:18:3) \_ App.synth (/Users/relm/Development/aws-cdk/packages/aws-cdk-lib/core/lib/stage.ts:180:23) \_ process.<anonymous> (/Users/relm/Development/aws-cdk/packages/aws-cdk-lib/core/lib/app.ts:131:45) \_ Object.onceWrapper (node:events:646:26) \_ process.emit (node:events:526:28) \_ process.emit (node:domain:475:12) \_ process.emit.sharedData.processEmitHook.installedValue [as emit] (/Users/relm/Development/aws-cdk-test-stack/node_modules/@cspotcode/source-map-support/source-map-support.js:613:40) relm-broken-1 | 0/3 | 9:13:06 PM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | relm-broken-1 The following resource(s) failed to create: [TestTopic339EC197, CDKMetadata]. Rollback requested by user. relm-broken-1 | 1/3 | 9:13:10 PM | DELETE_COMPLETE | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) relm-broken-1 | 2/3 | 9:13:10 PM | DELETE_COMPLETE | AWS::SNS::Topic | TestTopic (TestTopic339EC197) relm-broken-1 | 3/3 | 9:13:11 PM | ROLLBACK_COMPLETE | AWS::CloudFormation::Stack | relm-broken-1 relm-broken-2 | 2/3 | 9:13:11 PM | CREATE_COMPLETE | AWS::SNS::Topic | TestTopic (TestTopic339EC197) relm-broken-2 | 3/3 | 9:13:12 PM | CREATE_COMPLETE | AWS::CloudFormation::Stack | relm-broken-2 relm-test-1 | 2/3 | 9:13:10 PM | CREATE_COMPLETE | AWS::SNS::Topic | TestTopic (TestTopic339EC197) Failed resources: relm-broken-1 | 9:13:04 PM | CREATE_FAILED | AWS::SNS::Topic | TestTopic (TestTopic339EC197) TestTopic already exists in stack arn:aws:cloudformation:us-east-1:__ACCOUNT_ID__:stack/relm-broken-2/79fc1770-cc10-11ec-8279-0a02b75d1237 new Topic (/Users/relm/Development/aws-cdk/packages/aws-cdk-lib/aws-sns/lib/topic.ts:102:22) \_ new BrokenStack (/Users/relm/Development/aws-cdk-test-stack/lib/broken-stack.ts:11:18) \_ Object.<anonymous> (/Users/relm/Development/aws-cdk-test-stack/bin/aws-cdk-test-stack.ts:12:1) \_ Module._compile (node:internal/modules/cjs/loader:1103:14) \_ Module.m._compile (/Users/relm/Development/aws-cdk-test-stack/node_modules/ts-node/src/index.ts:1455:23) \_ Module._extensions..js (node:internal/modules/cjs/loader:1157:10) \_ Object.require.extensions.<computed> [as .ts] (/Users/relm/Development/aws-cdk-test-stack/node_modules/ts-node/src/index.ts:1458:12) \_ Module.load (node:internal/modules/cjs/loader:981:32) \_ Function.Module._load (node:internal/modules/cjs/loader:822:12) \_ Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12) \_ phase4 (/Users/relm/Development/aws-cdk-test-stack/node_modules/ts-node/src/bin.ts:567:12) \_ bootstrap (/Users/relm/Development/aws-cdk-test-stack/node_modules/ts-node/src/bin.ts:85:10) \_ main (/Users/relm/Development/aws-cdk-test-stack/node_modules/ts-node/src/bin.ts:54:10) \_ Object.<anonymous> (/Users/relm/Development/aws-cdk-test-stack/node_modules/ts-node/src/bin.ts:717:3) \_ Module._compile (node:internal/modules/cjs/loader:1103:14) \_ Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10) \_ Module.load (node:internal/modules/cjs/loader:981:32) \_ Function.Module._load (node:internal/modules/cjs/loader:822:12) \_ Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12) \_ node:internal/main/run_main_module:17:47 ❌ relm-broken-1 failed: Error: The stack named relm-broken-1 failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE at Object.waitForStackDeploy (/Users/relm/Development/aws-cdk/packages/aws-cdk/lib/api/util/cloudformation.ts:307:11) at processTicksAndRejections (node:internal/process/task_queues:96:5) at prepareAndExecuteChangeSet (/Users/relm/Development/aws-cdk/packages/aws-cdk/lib/api/deploy-stack.ts:355:26) at deployStack (/Users/relm/Development/aws-cdk/packages/aws-cdk/lib/cdk-toolkit.ts:229:24) at /Users/relm/Development/aws-cdk/packages/aws-cdk/lib/cdk-toolkit.ts:306:13 at run (/Users/relm/Development/aws-cdk/node_modules/p-queue/dist/index.js:163:29) /Users/relm/Development/aws-cdk/packages/aws-cdk/lib/api/util/cloudformation.ts:307 throw new Error(`The stack named ${stackName} failed creation, it may need to be manually deleted from the AWS console: ${status}`); ^ Error: The stack named relm-broken-1 failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE at Object.waitForStackDeploy (/Users/relm/Development/aws-cdk/packages/aws-cdk/lib/api/util/cloudformation.ts:307:11) at processTicksAndRejections (node:internal/process/task_queues:96:5) at prepareAndExecuteChangeSet (/Users/relm/Development/aws-cdk/packages/aws-cdk/lib/api/deploy-stack.ts:355:26) at deployStack (/Users/relm/Development/aws-cdk/packages/aws-cdk/lib/cdk-toolkit.ts:229:24) at /Users/relm/Development/aws-cdk/packages/aws-cdk/lib/cdk-toolkit.ts:306:13 at run (/Users/relm/Development/aws-cdk/node_modules/p-queue/dist/index.js:163:29) error Command failed with exit code 1. info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command. </pre> </details> ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
- Loading branch information