Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(cdk-assets): concurrent asset builds can leave a corrupted archive #23677

Merged
merged 4 commits into from
Jan 18, 2023

Conversation

rscharer
Copy link
Contributor

@rscharer rscharer commented Jan 13, 2023

Resolves #23290

A very simple fix for the issue where builds with --concurrency specified can lead to corrupt archives. Rather than use the outputFile as the basis for the temp file name we simply use a random UUID.

Please note that I was unable to run the integration tests in this instance, which are likely necessary given that this change impacts the behavior of the archiver.


All Submissions:

Adding new Construct Runtime Dependencies:

  • This PR adds new construct runtime dependencies following the process described here

New Features

  • Have you added the new feature to an integration test?
    • Did you use yarn integ to deploy the infrastructure and generate the snapshot (i.e. yarn integ without --dry-run)?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@gitpod-io
Copy link

gitpod-io bot commented Jan 13, 2023

@github-actions github-actions bot added the p2 label Jan 13, 2023
@aws-cdk-automation aws-cdk-automation requested a review from a team January 13, 2023 15:03
@github-actions github-actions bot added the beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK label Jan 13, 2023
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

@rix0rrr rix0rrr changed the title fix(cdk-assets): Use a UUID for the temp file prefix to prevent concurrent writes. Resolves #23290. fix(cdk-assets): concurrent artifact builds can leave a corrupted archive Jan 18, 2023
@rix0rrr rix0rrr changed the title fix(cdk-assets): concurrent artifact builds can leave a corrupted archive fix(cdk-assets): concurrent asset builds can leave a corrupted archive Jan 18, 2023
@rix0rrr rix0rrr added pr-linter/exempt-test The PR linter will not require test changes pr-linter/cli-integ-tested Assert that any CLI changes have been integ tested labels Jan 18, 2023
@rix0rrr rix0rrr added the pr-linter/exempt-integ-test The PR linter will not require integ test changes label Jan 18, 2023
@aws-cdk-automation aws-cdk-automation dismissed their stale review January 18, 2023 09:23

✅ Updated pull request passes all PRLinter validations. Dissmissing previous PRLinter review.

@mergify
Copy link
Contributor

mergify bot commented Jan 18, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 0c81b8d
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit 18e0481 into aws:main Jan 18, 2023
@mergify
Copy link
Contributor

mergify bot commented Jan 18, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

rix0rrr pushed a commit that referenced this pull request Feb 6, 2023
#23677)

Resolves #23290

A very simple fix for the issue where builds with `--concurrency` specified can lead to corrupt archives. Rather than use the outputFile as the basis for the temp file name we simply use a random UUID.

Please note that I was unable to run the integration tests in this instance, which are likely necessary given that this change impacts the behavior of the archiver.
rix0rrr pushed a commit that referenced this pull request Feb 6, 2023
#23677)

Resolves #23290

A very simple fix for the issue where builds with `--concurrency` specified can lead to corrupt archives. Rather than use the outputFile as the basis for the temp file name we simply use a random UUID.

Please note that I was unable to run the integration tests in this instance, which are likely necessary given that this change impacts the behavior of the archiver.
mergify bot pushed a commit that referenced this pull request Feb 6, 2023
This is a re-roll of #23677 which was reverted in #23994 because the `randomUUID()` function from the original solution was not available in Node versions below 14.17 (and we advertise compatibility with Node 14.*).

We didn't actually need a UUID, just any random string, so replace it with a function that generates a random string in a different way.

----------

Resolves #23290

A very simple fix for the issue where builds with `--concurrency` specified can lead to corrupt archives. Rather than use the outputFile as the basis for the temp file name we simply use a random string.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK bug This issue is a bug. effort/small Small work item – less than a day of effort p1 pr-linter/cli-integ-tested Assert that any CLI changes have been integ tested pr-linter/exempt-integ-test The PR linter will not require integ test changes pr-linter/exempt-test The PR linter will not require test changes
Projects
None yet
3 participants