Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(core): bundle assets in .cdk.staging #8601

Merged
merged 10 commits into from
Jun 18, 2020
Merged

chore(core): bundle assets in .cdk.staging #8601

merged 10 commits into from
Jun 18, 2020

Conversation

jogold
Copy link
Contributor

@jogold jogold commented Jun 17, 2020 •
edited
Loading

Using /tmp can be problematic with some Docker setups or in CI.

The directory .cdk.staging is already gitignored in the init
templates.

Closes #8589

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

eladb
eladb reviewed Jun 17, 2020
View reviewed changes
Copy link
Contributor

@eladb eladb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am okay with adding this, but I don't like the fact that it requires some users to tweak their environments.

I am wondering if it will make more sense to mount it by default under the project directory in ./.tmp or ./.cdk.tmp instead of the system temp directory.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: d6c1bc3
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@jogold
Copy link
Contributor Author

jogold commented Jun 17, 2020

I am wondering if it will make more sense to mount it by default under the project directory in ./.tmp or ./.cdk.tmp instead of the system temp directory.

Better, and renaming here should always work:

aws-cdk/packages/@aws-cdk/core/lib/asset-staging.ts

Lines 122 to 128 in b0286e9

try {
fs.renameSync(this.bundleDir, targetPath);
return;
} catch (err) {
// /tmp and cdk.out could be mounted across different mount points
// in this case we will fallback to copying. This can happen in Windows
// Subsystem for Linux (WSL).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 4f62c15
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@jogold
Copy link
Contributor Author

jogold commented Jun 17, 2020

There was previously a .cdk.staging apparently:

aws-cdk/packages/aws-cdk/lib/init-templates/app/typescript/.template.gitignore

Line 7 in b0286e9

.cdk.staging

aws-cdk/.gitignore

Line 32 in b0286e9

.cdk.staging/

Should I use this instead?

@eladb
Copy link
Contributor

eladb commented Jun 17, 2020

There was previously a .cdk.staging apparently:

aws-cdk/packages/aws-cdk/lib/init-templates/app/typescript/.template.gitignore

Line 7 in b0286e9

.cdk.staging

aws-cdk/.gitignore

Line 32 in b0286e9

.cdk.staging/

Should I use this instead?

Sure, why not

@jogold
chore(core): bundle assets in .cdk.staging
4ccf6ec 
Using `/tmp` can be problematic with some Docker setups or in CI.

The directory `.cdk.staging` is already gitignored in the init
templates.

Closes aws#8589
@jogold jogold changed the title feat(core): custom temp directory chore(core): bundle assets in .cdk.staging Jun 17, 2020
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 4ccf6ec
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: c18c0a3
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 1a3187c
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@jogold
Copy link
Contributor Author

jogold commented Jun 17, 2020 •
edited
Loading

Funny...
Error: EXDEV: cross-device link not permitted, rename '.cdk.staging/asset-bundle-nKh6F4' -> '/tmp/cdk.outSqgZ2f/asset.2f37f937c51e2c191af66acf9b09f548926008ec68c575bd2ee54b6e997c0e00'

We need a FileSystem.moveDirectory() that tries to rename and otherwise copies and unlinks (since we are not in /tmp anymore).

@jogold
Copy link
Contributor Author

jogold commented Jun 17, 2020

We need a FileSystem.moveDirectory() that tries to rename and otherwise copies and unlinks.

Wouldn't it be better to bundle fs-extra in core? I know you don't like adding dependencies but it's small well maintained and covers all the edge cases (EACCESS, EPERM, windows, etc.)

@eladb
Copy link
Contributor

eladb commented Jun 18, 2020

Funny...

Error: EXDEV: cross-device link not permitted, rename '.cdk.staging/asset-bundle-nKh6F4' -> '/tmp/cdk.outSqgZ2f/asset.2f37f937c51e2c191af66acf9b09f548926008ec68c575bd2ee54b6e997c0e00'

We need a FileSystem.moveDirectory() that tries to rename and otherwise copies and unlinks (since we are not in /tmp anymore).

Haha. This is for when cdk.out is under /tmp ha?

I am okay bundling fs-extra in core.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 05025bc
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 90ea756
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: ba382b5
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: e0314db
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: d71bf2d
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: ddf4ab8
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@eladb
Copy link
Contributor

eladb commented Jun 18, 2020

For the record here’s ‘fs-extra’s dep closure:

7C46DE7F-9141-40AA-86F9-D826980048C0

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 572f6f0
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

eladb
eladb approved these changes Jun 18, 2020
View reviewed changes
Copy link
Contributor

@eladb eladb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotta love it that all our init templates (and therefore all user projects) already contain .cdk.staging in their .gitignore file! Great call.

@mergify
Copy link
Contributor

mergify bot commented Jun 18, 2020

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit 4b68655 into aws:master Jun 18, 2020
@jogold jogold deleted the cdk-tmp branch June 18, 2020 11:38
@jogold jogold mentioned this pull request Jul 1, 2020
Closed
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eladb eladb eladb approved these changes

Assignees

@eladb eladb

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Change the bundler's /asset-output local volume mount location
3 participants
@jogold @aws-cdk-automation @eladb