feat(cli): implementation of the new bootstrap logic #5856

skinny85 · 2020-01-18T02:21:59Z

This is the logic of the new bootstrap command, deploying the template discussed in #4461 and then further in #5757 . You activate the new bootstrap behavior by passing the --new-bootstrap=true option to the CLI.

I've manually verified that bootstrapping succeeds, and that a CDK app using assets can successfully be deployed using this new bootstrap stack.

I've posting this as a draft, to get early feedback, as there are a few ways we can go about incorporating this new functionality, so I wanted to get the team's opinion on the direction taken.

I'm in the process of writing an automated integ test that checks whether this works without the need for manual testing, but it's proving to be a little tricky, so I figured I won't hold up the main logic PR on it.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

aws-cdk-automation · 2020-01-18T03:09:25Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

packages/aws-cdk/bin/cdk.ts

packages/aws-cdk/lib/api/bootstrap/bootstrap-template.json

eladb · 2020-01-20T14:47:23Z

packages/aws-cdk/lib/api/bootstrap/bootstrap-environment2.ts

+  const builder = new cxapi.CloudAssemblyBuilder(outdir);
+  const templateFile = `${toolkitStackName}.template.json`;
+
+  await fs.copy(
+    path.join(__dirname, 'bootstrap-template.json'),
+    path.join(builder.outdir, templateFile));
+
+  builder.addArtifact(toolkitStackName, {
+    type: cxapi.ArtifactType.AWS_CLOUDFORMATION_STACK,
+    environment: cxapi.EnvironmentUtils.format(environment.account, environment.region),
+    properties: {
+      templateFile,
+    },
+  });
+
+  const assembly = builder.buildAssembly();
+  return await deployStack({
+    stack: assembly.getStackByName(toolkitStackName),
+    sdk,
+    roleArn,
+    tags: props.tags,
+    execute: props.execute
+  });


Perhaps extract this to deployStackFromTemplate as a more low-level building block and then just call it from here with the bootstrap template.

Also, I expected to see the bootstrap options passed in to CFN parameters...

I tried extracting a deployStackFromTemplate() function, but it was super awkward (the file are created differently (fs.writeJson() versus fs.copy(), the temporary directories have different hints, we need to reference the output property of the CloudAssemblyBuilder to know where to put the files, etc.), so I abandoned it for now. If you have any suggestions how to make that part better, I'm all ears.

I've added passing the CLI options through the parameters.

packages/aws-cdk/lib/api/bootstrap/bootstrap-template.json

skinny85 · 2020-02-04T02:19:57Z

@eladb @rix0rrr this is ready for a re-review.

aws-cdk-automation · 2020-02-04T02:22:38Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-02-04T02:32:01Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-02-04T18:37:27Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-02-04T18:41:43Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-02-04T20:14:23Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-02-06T03:56:47Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

packages/aws-cdk/bin/cdk.ts

packages/aws-cdk/lib/api/bootstrap/bootstrap-environment2.ts

skinny85 · 2020-02-06T19:20:13Z

Apologies for leaving the old comments in the PR @eladb , they are removed now.

aws-cdk-automation · 2020-02-06T20:09:54Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

rix0rrr · 2020-02-07T12:23:39Z

packages/aws-cdk/lib/api/bootstrap/bootstrap-environment2.ts

+                                            toolkitStackName: string, roleArn: string | undefined,
+                                            props: BootstrapEnvironmentProps = {}): Promise<DeployStackResult> {
+  if (props.trustedAccounts?.length && !props.cloudFormationExecutionPolicies?.length) {
+    throw new Error('--cloudformation-execution-policies are required if --trust has been passed!');


I'm not super happy with this user experience. What if I'm trusting a new account? Not only do I have to list all previous accounts, but I also have to re-list the policy?

And the policy even needs to be referenced by ARN, which is super annoying.

I know we're going bare-bones functionality for now and worry about the experience later, but this is going to need some attention I feel...

I agree this is medium.

For the "trusting new accounts" problem: would your comment here be applicable? We read the current value of the parameter, and add to it (with set semantics) the array of accounts that were passed in --trust, and that's the final value of the parameter? This way, --trust would always be additive.

For the "policy needs to be referenced by ARN" problem: originally, we wanted to prompt the customer if they didn't pass anything in the --cloudformation-execution-policies with a list like this:

This will create a new IAM Role, trusting any principal from the ACCOUNT_ID AWS account, that will be used for deploying your CloudFormation stack(s) in that environment. Please choose the Managed Policy or Policies to attach to the Role that contain the permissions required to perform those deployments: [1] AdministratorAccess [2] AmazonDynamoDBFullAccess [3] AmazonEC2FullAccess [4] AmazonECS_FullAccess [5] AmazonS3FullAccess [6] AmazonSageMakerFullAccess [7] AmazonSESFullAccess [8] AmazonSNSFullAccess [9] AmazonSQSFullAccess [10] AWSLambdaFullAccess [11] SimpleWorkflowFullAccess ... Please input the number of the chosen Policy from the above list, or provide a fully qualified ARN of a different Managed Policy. You can input multiple numbers and/or ARNs (up to 10), separated by commas. >

But it was scrapped for time purposes. Do you want to revisit this topic?

For the "trusting new accounts" problem: would your comment here be applicable? We read the current value of the parameter, and add to it (with set semantics) the array of accounts that were passed in --trust, and that's the final value of the parameter? This way, --trust would always be additive.

Yes, I would definitely like that. We would only use UsePreviousValue if there was NO change to the parameter, but if we used --trust or --untrust (?) we would do set logic on the parameter.

I think I might want to revisit, or get people to the "right" solutions in an easier way. Maybe a UI, maybe we recognize some aliases for "standard" policies ("admin" => "arn:aws:iam:::policy/AdministratorAccess"). Doesn't have to be here and now (let's get this merged), but I want to put it into your head already.

Yes, I would definitely like that. We would only use UsePreviousValue if there was NO change to the parameter, but if we used --trust or --untrust (?) we would do set logic on the parameter.

Blah. This is kind of hard ☹️.

The problem is that you can't use UsePreviousValue if you're creating the Stack. Which means it's not good enough for our case. We would have to call DescribeStack before the deployment, and based on what that returns:

If the bootstrap Stack doesn't exist yet (= we're creating it), then pass whatever accounts were given in --trust as the value of the TrustedAccounts parameter.

If the bootstrap Stack does exist, add the current value of the TrustedAccounts parameter (split on commas of course, as its Type is CommaSeparatedList) to the set of accounts passed through --trust, and use that as the value.

That also means we need some way to remove accounts from the trust relationship (like the --untrust CLI option you mentioned).

Do you want me to add this logic as part of this PR, or are you OK coming back to this later @rix0rrr ?

Totally later!

rix0rrr · 2020-02-07T12:26:26Z

packages/aws-cdk/lib/api/deploy-stack.ts

@@ -54,6 +66,16 @@ export async function deployStack(options: DeployStackOptions): Promise<DeploySt

  const params = await prepareAssets(options.stack, options.toolkitInfo, options.reuseAssets);

+  // add passed CloudFormation parameters
+  for (const [paramName, paramValue] of Object.entries((options.parameters || {}))) {
+    if (paramValue) {


We can consider looking at the existing stack here, and if there are any Parameters on the current stack that aren't passed here, we can be passing UsePreviousValue: true to make ergonomics around parameters a lot better.

Maybe not for this PR, but keep that in mind.

https://docs.aws.amazon.com/AWSCloudFormation/latest/APIReference/API_Parameter.html

Great idea. I think we can use it to make --trust better - see #5856 (comment) .

packages/aws-cdk/package.json

packages/aws-cdk/test/api/bootstrap2.test.ts

packages/aws-cdk/test/integ/bootstrap/bootstrap.integ-test.ts

packages/aws-cdk/test/integ/cli/test.sh

aws-cdk-automation · 2020-02-11T01:26:58Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-02-11T02:20:45Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

This adds a new `cdk bootstrap` mode, hidden behind the environment variable CDK_NEW_BOOTSTRAP, that adds the necessary resources (ECR repository, IAM roles, etc.) for our "CI/CD for CDK apps" epic, as well as 2 new (hidden for now) options: --trust (for specifying the accounts that are trusted to deploy into this environment) and --cloudformation-execution-policies (for specifying the ARNs of the Managed Policis that will be attached to the CFN deploying role).

skinny85 · 2020-02-11T22:36:20Z

@rix0rrr should be ready for another round of reviews (included your last comments).

rix0rrr · 2020-02-17T17:01:09Z

packages/aws-cdk/lib/api/bootstrap/bootstrap-environment2.ts

+                                            toolkitStackName: string, roleArn: string | undefined,
+                                            props: BootstrapEnvironmentProps = {}): Promise<DeployStackResult> {
+  if (props.trustedAccounts?.length && !props.cloudFormationExecutionPolicies?.length) {
+    throw new Error('--cloudformation-execution-policies are required if --trust has been passed!');


Totally later!

aws-cdk-automation · 2020-02-17T18:13:11Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: 1f72085
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

There was a release since the PR was published.

aws-cdk-automation · 2020-02-17T19:20:50Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: e5b5fa7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-02-17T19:32:34Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: f131e07
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-02-17T20:17:44Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: f131e07
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-02-17T21:12:40Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: cba700b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

skinny85 requested review from rix0rrr, eladb and NetaNir January 18, 2020 02:21

skinny85 self-assigned this Jan 18, 2020

mergify bot added the contribution/core This is a PR that came from AWS. label Jan 18, 2020

rix0rrr reviewed Jan 20, 2020

View reviewed changes