-
Notifications
You must be signed in to change notification settings - Fork 3.9k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix(ecr-assets): fix repeated deploys of stacks with tar assets
The current implementation fails because it assumes the output of
`docker load` can only be
```
Loaded Image: {image}
```
However, if docker has to take any actions (such as untagging a previous
image) the output will look like
```
{error message}
{error message}
Loaded Image: {image}
```
To fix this, we can simply take the last line of the output via `tail`
before we attempt to extract the image. If the last line isn't of the
correct form, this will fail in an effectively equivalent way to how it
failed previously.
A previous attempt at fixing this (#18823) used a more complicated `sed`
command which used a cli flag that is not consistently available. This
approach won't work on all platforms (ex. macos) and is also much less
clear that it's correct.
This change also adds an integration test that tests the tarbell stack, though
it doesn't test the repeated deploy automatically. I have tested that by
running the integration test multiple times with an image modification in
between. We also vendor the tarball of the hello-world docker image, which
results in ~10kb of additional data in the repo. This is about as small as we
can get a vendored image with as straightforward a vendoring process.
fixes #18822- Loading branch information
Showing
14 changed files
with
631 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file added
BIN
+19.5 KB
packages/@aws-cdk/aws-ecr-assets/test/demo-tarball-hello-world/hello-world.tar
Binary file not shown.
19 changes: 19 additions & 0 deletions
19
...teg.assets-tarball.js.snapshot/LoadFromTarballDefaultTestDeployAssert46673615.assets.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| { | ||
| "version": "22.0.0", | ||
| "files": { | ||
| "21fbb51d7b23f6a6c262b46a9caee79d744a3ac019fd45422d988b96d44b2a22": { | ||
| "source": { | ||
| "path": "LoadFromTarballDefaultTestDeployAssert46673615.template.json", | ||
| "packaging": "file" | ||
| }, | ||
| "destinations": { | ||
| "current_account-current_region": { | ||
| "bucketName": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}", | ||
| "objectKey": "21fbb51d7b23f6a6c262b46a9caee79d744a3ac019fd45422d988b96d44b2a22.json", | ||
| "assumeRoleArn": "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cdk-hnb659fds-file-publishing-role-${AWS::AccountId}-${AWS::Region}" | ||
| } | ||
| } | ||
| } | ||
| }, | ||
| "dockerImages": {} | ||
| } |
36 changes: 36 additions & 0 deletions
36
...g.assets-tarball.js.snapshot/LoadFromTarballDefaultTestDeployAssert46673615.template.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| { | ||
| "Parameters": { | ||
| "BootstrapVersion": { | ||
| "Type": "AWS::SSM::Parameter::Value<String>", | ||
| "Default": "/cdk-bootstrap/hnb659fds/version", | ||
| "Description": "Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]" | ||
| } | ||
| }, | ||
| "Rules": { | ||
| "CheckBootstrapVersion": { | ||
| "Assertions": [ | ||
| { | ||
| "Assert": { | ||
| "Fn::Not": [ | ||
| { | ||
| "Fn::Contains": [ | ||
| [ | ||
| "1", | ||
| "2", | ||
| "3", | ||
| "4", | ||
| "5" | ||
| ], | ||
| { | ||
| "Ref": "BootstrapVersion" | ||
| } | ||
| ] | ||
| } | ||
| ] | ||
| }, | ||
| "AssertDescription": "CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI." | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| } |
Binary file added
BIN
+19.5 KB
...ll.js.snapshot/asset.a5fbe3aa4a0e49fac7d3fbe7b913cf022d307bfad6ef1e88e06cc2a229523474.tar
Binary file not shown.
1 change: 1 addition & 0 deletions
1
packages/@aws-cdk/aws-ecr-assets/test/integ.assets-tarball.js.snapshot/cdk.out
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| {"version":"22.0.0"} |
36 changes: 36 additions & 0 deletions
36
...cdk/aws-ecr-assets/test/integ.assets-tarball.js.snapshot/integ-assets-tarball.assets.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| { | ||
| "version": "22.0.0", | ||
| "files": { | ||
| "7e85363095650cdc9e77b849e3afda82da3123dd43f5d237b9601f816ddf583e": { | ||
| "source": { | ||
| "path": "integ-assets-tarball.template.json", | ||
| "packaging": "file" | ||
| }, | ||
| "destinations": { | ||
| "current_account-current_region": { | ||
| "bucketName": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}", | ||
| "objectKey": "7e85363095650cdc9e77b849e3afda82da3123dd43f5d237b9601f816ddf583e.json", | ||
| "assumeRoleArn": "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cdk-hnb659fds-file-publishing-role-${AWS::AccountId}-${AWS::Region}" | ||
| } | ||
| } | ||
| } | ||
| }, | ||
| "dockerImages": { | ||
| "a5fbe3aa4a0e49fac7d3fbe7b913cf022d307bfad6ef1e88e06cc2a229523474": { | ||
| "source": { | ||
| "executable": [ | ||
| "sh", | ||
| "-c", | ||
| "docker load -i asset.a5fbe3aa4a0e49fac7d3fbe7b913cf022d307bfad6ef1e88e06cc2a229523474.tar | tail -n 1 | sed \"s/Loaded image: //g\"" | ||
| ] | ||
| }, | ||
| "destinations": { | ||
| "current_account-current_region": { | ||
| "repositoryName": "cdk-hnb659fds-container-assets-${AWS::AccountId}-${AWS::Region}", | ||
| "imageTag": "a5fbe3aa4a0e49fac7d3fbe7b913cf022d307bfad6ef1e88e06cc2a229523474", | ||
| "assumeRoleArn": "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cdk-hnb659fds-image-publishing-role-${AWS::AccountId}-${AWS::Region}" | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } |
105 changes: 105 additions & 0 deletions
105
...k/aws-ecr-assets/test/integ.assets-tarball.js.snapshot/integ-assets-tarball.template.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| { | ||
| "Resources": { | ||
| "MyUserDC45028B": { | ||
| "Type": "AWS::IAM::User" | ||
| }, | ||
| "MyUserDefaultPolicy7B897426": { | ||
| "Type": "AWS::IAM::Policy", | ||
| "Properties": { | ||
| "PolicyDocument": { | ||
| "Statement": [ | ||
| { | ||
| "Action": [ | ||
| "ecr:BatchCheckLayerAvailability", | ||
| "ecr:BatchGetImage", | ||
| "ecr:GetDownloadUrlForLayer" | ||
| ], | ||
| "Effect": "Allow", | ||
| "Resource": { | ||
| "Fn::Join": [ | ||
| "", | ||
| [ | ||
| "arn:", | ||
| { | ||
| "Ref": "AWS::Partition" | ||
| }, | ||
| ":ecr:", | ||
| { | ||
| "Ref": "AWS::Region" | ||
| }, | ||
| ":", | ||
| { | ||
| "Ref": "AWS::AccountId" | ||
| }, | ||
| ":repository/", | ||
| { | ||
| "Fn::Sub": "cdk-hnb659fds-container-assets-${AWS::AccountId}-${AWS::Region}" | ||
| } | ||
| ] | ||
| ] | ||
| } | ||
| }, | ||
| { | ||
| "Action": "ecr:GetAuthorizationToken", | ||
| "Effect": "Allow", | ||
| "Resource": "*" | ||
| } | ||
| ], | ||
| "Version": "2012-10-17" | ||
| }, | ||
| "PolicyName": "MyUserDefaultPolicy7B897426", | ||
| "Users": [ | ||
| { | ||
| "Ref": "MyUserDC45028B" | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| }, | ||
| "Outputs": { | ||
| "ImageUri": { | ||
| "Value": { | ||
| "Fn::Sub": "${AWS::AccountId}.dkr.ecr.${AWS::Region}.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-${AWS::AccountId}-${AWS::Region}:a5fbe3aa4a0e49fac7d3fbe7b913cf022d307bfad6ef1e88e06cc2a229523474" | ||
| } | ||
| }, | ||
| "ImageUri2": { | ||
| "Value": { | ||
| "Fn::Sub": "${AWS::AccountId}.dkr.ecr.${AWS::Region}.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-${AWS::AccountId}-${AWS::Region}:a5fbe3aa4a0e49fac7d3fbe7b913cf022d307bfad6ef1e88e06cc2a229523474" | ||
| } | ||
| } | ||
| }, | ||
| "Parameters": { | ||
| "BootstrapVersion": { | ||
| "Type": "AWS::SSM::Parameter::Value<String>", | ||
| "Default": "/cdk-bootstrap/hnb659fds/version", | ||
| "Description": "Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]" | ||
| } | ||
| }, | ||
| "Rules": { | ||
| "CheckBootstrapVersion": { | ||
| "Assertions": [ | ||
| { | ||
| "Assert": { | ||
| "Fn::Not": [ | ||
| { | ||
| "Fn::Contains": [ | ||
| [ | ||
| "1", | ||
| "2", | ||
| "3", | ||
| "4", | ||
| "5" | ||
| ], | ||
| { | ||
| "Ref": "BootstrapVersion" | ||
| } | ||
| ] | ||
| } | ||
| ] | ||
| }, | ||
| "AssertDescription": "CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI." | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| } |
12 changes: 12 additions & 0 deletions
12
packages/@aws-cdk/aws-ecr-assets/test/integ.assets-tarball.js.snapshot/integ.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
| "version": "22.0.0", | ||
| "testCases": { | ||
| "LoadFromTarball/DefaultTest": { | ||
| "stacks": [ | ||
| "integ-assets-tarball" | ||
| ], | ||
| "assertionStack": "LoadFromTarball/DefaultTest/DeployAssert", | ||
| "assertionStackName": "LoadFromTarballDefaultTestDeployAssert46673615" | ||
| } | ||
| } | ||
| } |
129 changes: 129 additions & 0 deletions
129
packages/@aws-cdk/aws-ecr-assets/test/integ.assets-tarball.js.snapshot/manifest.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| { | ||
| "version": "22.0.0", | ||
| "artifacts": { | ||
| "integ-assets-tarball.assets": { | ||
| "type": "cdk:asset-manifest", | ||
| "properties": { | ||
| "file": "integ-assets-tarball.assets.json", | ||
| "requiresBootstrapStackVersion": 6, | ||
| "bootstrapStackVersionSsmParameter": "/cdk-bootstrap/hnb659fds/version" | ||
| } | ||
| }, | ||
| "integ-assets-tarball": { | ||
| "type": "aws:cloudformation:stack", | ||
| "environment": "aws://unknown-account/unknown-region", | ||
| "properties": { | ||
| "templateFile": "integ-assets-tarball.template.json", | ||
| "validateOnSynth": false, | ||
| "assumeRoleArn": "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cdk-hnb659fds-deploy-role-${AWS::AccountId}-${AWS::Region}", | ||
| "cloudFormationExecutionRoleArn": "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cdk-hnb659fds-cfn-exec-role-${AWS::AccountId}-${AWS::Region}", | ||
| "stackTemplateAssetObjectUrl": "s3://cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}/7e85363095650cdc9e77b849e3afda82da3123dd43f5d237b9601f816ddf583e.json", | ||
| "requiresBootstrapStackVersion": 6, | ||
| "bootstrapStackVersionSsmParameter": "/cdk-bootstrap/hnb659fds/version", | ||
| "additionalDependencies": [ | ||
| "integ-assets-tarball.assets" | ||
| ], | ||
| "lookupRole": { | ||
| "arn": "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cdk-hnb659fds-lookup-role-${AWS::AccountId}-${AWS::Region}", | ||
| "requiresBootstrapStackVersion": 8, | ||
| "bootstrapStackVersionSsmParameter": "/cdk-bootstrap/hnb659fds/version" | ||
| } | ||
| }, | ||
| "dependencies": [ | ||
| "integ-assets-tarball.assets" | ||
| ], | ||
| "metadata": { | ||
| "/integ-assets-tarball/MyUser/Resource": [ | ||
| { | ||
| "type": "aws:cdk:logicalId", | ||
| "data": "MyUserDC45028B" | ||
| } | ||
| ], | ||
| "/integ-assets-tarball/MyUser/DefaultPolicy/Resource": [ | ||
| { | ||
| "type": "aws:cdk:logicalId", | ||
| "data": "MyUserDefaultPolicy7B897426" | ||
| } | ||
| ], | ||
| "/integ-assets-tarball/ImageUri": [ | ||
| { | ||
| "type": "aws:cdk:logicalId", | ||
| "data": "ImageUri" | ||
| } | ||
| ], | ||
| "/integ-assets-tarball/ImageUri2": [ | ||
| { | ||
| "type": "aws:cdk:logicalId", | ||
| "data": "ImageUri2" | ||
| } | ||
| ], | ||
| "/integ-assets-tarball/BootstrapVersion": [ | ||
| { | ||
| "type": "aws:cdk:logicalId", | ||
| "data": "BootstrapVersion" | ||
| } | ||
| ], | ||
| "/integ-assets-tarball/CheckBootstrapVersion": [ | ||
| { | ||
| "type": "aws:cdk:logicalId", | ||
| "data": "CheckBootstrapVersion" | ||
| } | ||
| ] | ||
| }, | ||
| "displayName": "integ-assets-tarball" | ||
| }, | ||
| "LoadFromTarballDefaultTestDeployAssert46673615.assets": { | ||
| "type": "cdk:asset-manifest", | ||
| "properties": { | ||
| "file": "LoadFromTarballDefaultTestDeployAssert46673615.assets.json", | ||
| "requiresBootstrapStackVersion": 6, | ||
| "bootstrapStackVersionSsmParameter": "/cdk-bootstrap/hnb659fds/version" | ||
| } | ||
| }, | ||
| "LoadFromTarballDefaultTestDeployAssert46673615": { | ||
| "type": "aws:cloudformation:stack", | ||
| "environment": "aws://unknown-account/unknown-region", | ||
| "properties": { | ||
| "templateFile": "LoadFromTarballDefaultTestDeployAssert46673615.template.json", | ||
| "validateOnSynth": false, | ||
| "assumeRoleArn": "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cdk-hnb659fds-deploy-role-${AWS::AccountId}-${AWS::Region}", | ||
| "cloudFormationExecutionRoleArn": "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cdk-hnb659fds-cfn-exec-role-${AWS::AccountId}-${AWS::Region}", | ||
| "stackTemplateAssetObjectUrl": "s3://cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}/21fbb51d7b23f6a6c262b46a9caee79d744a3ac019fd45422d988b96d44b2a22.json", | ||
| "requiresBootstrapStackVersion": 6, | ||
| "bootstrapStackVersionSsmParameter": "/cdk-bootstrap/hnb659fds/version", | ||
| "additionalDependencies": [ | ||
| "LoadFromTarballDefaultTestDeployAssert46673615.assets" | ||
| ], | ||
| "lookupRole": { | ||
| "arn": "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cdk-hnb659fds-lookup-role-${AWS::AccountId}-${AWS::Region}", | ||
| "requiresBootstrapStackVersion": 8, | ||
| "bootstrapStackVersionSsmParameter": "/cdk-bootstrap/hnb659fds/version" | ||
| } | ||
| }, | ||
| "dependencies": [ | ||
| "LoadFromTarballDefaultTestDeployAssert46673615.assets" | ||
| ], | ||
| "metadata": { | ||
| "/LoadFromTarball/DefaultTest/DeployAssert/BootstrapVersion": [ | ||
| { | ||
| "type": "aws:cdk:logicalId", | ||
| "data": "BootstrapVersion" | ||
| } | ||
| ], | ||
| "/LoadFromTarball/DefaultTest/DeployAssert/CheckBootstrapVersion": [ | ||
| { | ||
| "type": "aws:cdk:logicalId", | ||
| "data": "CheckBootstrapVersion" | ||
| } | ||
| ] | ||
| }, | ||
| "displayName": "LoadFromTarball/DefaultTest/DeployAssert" | ||
| }, | ||
| "Tree": { | ||
| "type": "cdk:tree", | ||
| "properties": { | ||
| "file": "tree.json" | ||
| } | ||
| } | ||
| } | ||
| } |
Oops, something went wrong.