-
Notifications
You must be signed in to change notification settings - Fork 4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(cli): synth fails if there was an error when synthesizing the stack #14613
Conversation
* feat: cloudformation spec v35.1.0 * chore: attribute exception Co-authored-by: AWS CDK Team <aws-cdk@amazon.com> Co-authored-by: Nick Lynch <nlynch@amazon.com>
The combination of CodePipeline+CodeBuild will not properly extract symlinks from a git repository. Instead, the symlink will be replaced with a textual representation of its target. To properly test the symlink behavior, we need to remove the symlink from version control and re-create it at test time, after CodePipeline+CodeBuild have extracted the archive. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
…alized JSON (#14512) Our logic for parsing `${}` expressions inside `Fn::Sub` incorrectly looked for closing braces everywhere in the string, instead of doing it to the right of the opening `${`. This would fail if the string passed to `Fn::Sub` was serialized JSON (as those would contain closing braces completely unrelated to the `${`). Fixes #14095 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
* deprecate MariaDB `10.0`, `10.1` * add MariaDB `10.2.32` * deprecate MySQL `5.5`, `5.6` * add PostgreSQL `9.6.20`, `9.6.21`, `10.15`, `10.16`, `11.10`, `11.11`, `12.6`, `13.2` fixes #14495. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Docker is only required when not bundling locally. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Ran npm-check-updates and yarn upgrade to keep the `yarn.lock` file up-to-date.
As a possible solution for #12597, refer to the 3rd-party module `cdk-ecr-deployment` which supports deploying image assets to a specific ECR repository. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Closes #12555 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
A previous [PR](#13823) added a dependency from the stable module `aws-events-targets` to the experimental module `aws-batch`. Depending on experimental modules is no longer allowed. In fact, this PR build should have failed when attempting to build `aws-cdk-lib`, preventing it from being merged. However, since the dependency was added but not used, the build succeeded, and the PR was merged. ### Why did we expect the PR build of #13823 to fail? In the build step of `aws-cdk-lib` we strip all the L2 of experimental modules. This means that when compiling `aws-cdk-lib` the L2 types of experimental modules does not exists, if a stable module reference such a type, the build of `aws-cdk-lib` will fail. ### Why did the PR build of #13823 succeeded? It added a dependency on the experimental `aws-batch` but didn't use it. ### What are we doing to prevent this from happening in the future? #14249 adds a lint rule that disallow depending on an experimental module. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
> blocked on #14291 This PR adds a pkglint rule to disallow `@aws-cdk` modules to depend on experimental `@aws-cdk` modules. In v2 experimental modules will be released separately from `aws-cdk-lib`. This means that stable modules can not depend on experimental modules as it will create a cyclic dependency. The lint rule **also** ban exp->exp dependencies. The reasoning for the latter is that if an experimental module A, depends on experimental module B, in order to graduate module A, module B will also have to be graduated (to satisfy no stable->exp). Currently there are a few exp->exp dependencies, these might be removed as a part of the work to release the experimental modules separately from `aws-cdk-lib`. In the meantime we explicitly allow them by having an exclusion map. The current implementation executes the check for each modules as a part of its `pkglint` phase, which is O(n^2) (for each module check the stability of all of its dependencies). This can be improved by executing the check only once, in the build of `aws-cdk-lib` (going over all the modules in a topological order and storing the stability). The disadvantage of executing it only on the build of `aws-cdk-lib` is that it will not be executed as a part of a typical developer flow, which is `buildup` from a specific package. I have testes it several times locally in the current implementation adds ~2 seconds to the root level `yarn pkglint`. Since it does not add significant time to our build, and gives better/earlier visibility to contributors I have decided to keep it simple and preform the check on each module build. Happy to discuss more if anyone have concerns or optimizations ideas I missed. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
introducing a higher level construct to define golang lambda functions. This was inspired by the other high level constructs for Nodejs and Python I'm setting a couple of things by default. *Environment Variables* * `GOOS=linux` * `GOARCH=amd64` * `GO111MODULE=on` *AssetHashType* * `AssetHashType.OUTPUT` I've added some details in the README on when you could change this to `INPUT`. I'm not really sure what the majority use case would be though. *Go Modules* I am requiring the use of Go modules which were added in Go 1.11 (current version is Go 1.15). I think at this point most Go developers will probably be using Go modules (for a comparison the [aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) requires 1.15) *Runtime* * `Runtime.PROVIDED_AL2` The `GO_1_X` runtime doesn't support newer Lambda features like Lambda extensions, and the [aws-lambda-go](https://github.com/aws/aws-lambda-go) library supports the provided runtimes out of the box. Also, I copied over a lot of the boilerplate (package.json, LICENSE, etc) from the lambda-nodejs package and just did a search and replace, let me know if I got any of it wrong. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
) Resolves #13294 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@@ -12,7 +12,10 @@ Permission is hereby granted, free of charge, to any person obtaining a copy of | |||
this software and associated documentation files (the "Software"), to deal in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had to make these changes to the NOTICE files because I was getting errors like this:
In package package.json
- [license/3p-attributions] Unnecessary attribution found for dependency 'lru-cache' in NOTICE file. Attribution is determined from package.json (all "bundledDependencies" or the list in "pkglint.attribution")
- [license/3p-attributions] Unnecessary attribution found for dependency 'yallist' in NOTICE file. Attribution is determined from package.json (all "bundledDependencies" or the list in "pkglint.attribution")
- [license/3p-attributions] Unnecessary attribution found for dependency 'semver' in NOTICE file. Attribution is determined from package.json (all "bundledDependencies" or the list in "pkglint.attribution")
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't know why you were getting these errors, but the attribution is/was correct.
The problem here is that the tool we're using to determine which NPM dependencies are "bundled dependencies" is a bit shit, and it's not always detecting all dependencies.
I have a sense we need to replace this tool with something fixed.
See:
aws-cdk/tools/pkglint/lib/rules.ts
Line 172 in 464b827
export class ThirdPartyAttributions extends ValidationRule { |
aws-cdk/tools/pkglint/lib/packagejson.ts
Line 218 in 464b827
return bundled.sync({ path: this.packageRoot }); |
const bundled = require('npm-bundled'); |
The package npm-bundled
is just bad, and very dependent on how NPM chooses to lay out the package tree on whether it gets the right answer or not. (Note that the author of npm-bundled
is in no way affiliated with NPM-the-package-manager, they just chose to name their package that way)
@@ -12,7 +12,10 @@ Permission is hereby granted, free of charge, to any person obtaining a copy of | |||
this software and associated documentation files (the "Software"), to deal in | |||
the Software without restriction, including without limitation the rights to | |||
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies | |||
of the Software, and to permit persons to whom the Software is furnished to do | |||
of the Software, and to permit persons to whom t |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is probably unintended.
@@ -12,7 +12,10 @@ Permission is hereby granted, free of charge, to any person obtaining a copy of | |||
this software and associated documentation files (the "Software"), to deal in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't know why you were getting these errors, but the attribution is/was correct.
The problem here is that the tool we're using to determine which NPM dependencies are "bundled dependencies" is a bit shit, and it's not always detecting all dependencies.
I have a sense we need to replace this tool with something fixed.
See:
aws-cdk/tools/pkglint/lib/rules.ts
Line 172 in 464b827
export class ThirdPartyAttributions extends ValidationRule { |
aws-cdk/tools/pkglint/lib/packagejson.ts
Line 218 in 464b827
return bundled.sync({ path: this.packageRoot }); |
const bundled = require('npm-bundled'); |
The package npm-bundled
is just bad, and very dependent on how NPM chooses to lay out the package tree on whether it gets the right answer or not. (Note that the author of npm-bundled
is in no way affiliated with NPM-the-package-manager, they just chose to name their package that way)
packages/@aws-cdk/core/NOTICE
Outdated
@@ -3,25 +3,6 @@ Copyright 2018-2021 Amazon.com, Inc. or its affiliates. All Rights Reserved. | |||
|
|||
------------------------------------------------------------------------------- | |||
|
|||
The AWS CDK includes the following third-party software/licensing: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Again, these attributions should not be removed.
packages/@aws-cdk/cx-api/NOTICE
Outdated
@@ -1,55 +1,2 @@ | |||
AWS Cloud Development Kit (AWS CDK) | |||
Copyright 2018-2021 Amazon.com, Inc. or its affiliates. All Rights Reserved. | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Again
packages/aws-cdk/lib/cdk-toolkit.ts
Outdated
const stacksToValidate = await this.selectStacksForList(idsToValidate); | ||
await this.validateStacks(stacksToValidate); | ||
const allStacks = await this.selectStacksForList([]); | ||
await this.validateStacks(allStacks.filter(art => art.validateOnSynth ?? false)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Preferably we'd validate all stacks in one validateStacks()
call. The difference is a more complete view of all errors, as synthesis will stop after this call if it encounters errors (and so wouldn't show any errors that would be encountered in the validateStacks()
call below)
Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
* chore(cloudfront): remove the use of calculateFunctionHash (#14583) `calculateFunctionHash()` was used to compute the 'refresh token' of the custom resource for the EdgeFunction construct. This method is private to the lambda module and is deemed to be changed. Instead, use the lambda function version's logical id. The logical id of the version includes computing the function hash (among others) and is a more reliable determinant of whether the underlying function version changed. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* * feat(cloudwatch): validate parameters for a metric dimensions (closes #3116) (#14365) As per #3116, the changes in this PR validate metric dimension values (length, and checking if the value is null or undefined) and throw errors if the values are not valid. I've also corrected a comment in the metric-types.ts to use the correct method name * feat(appmesh): change HealthChecks to use protocol-specific union-like classes (#14432) BREAKING CHANGE: HealthChecks require use of static factory methods fixes #11640 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* * feat(msk): Cluster L2 Construct (#9908) L2 Construct for a MSK Cluster. I wrote this for internal use and thought I'd share it. I tried to follow the [example resource](https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/example-construct-library/lib/example-resource.ts) and [design guidelines](https://github.com/aws/aws-cdk/blob/master/DESIGN_GUIDELINES.md) as much as I could. Default properties were chosen either based on defaults when creating a cluster in the console or defaults set from CloudFormation. Closes #9603 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* * feat(kms): allow specifying key spec and key usage (#14478) This allows specifying key spec and key usage, so you can create asymmetric keys. closes #5639 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* * chore: add `@types/jest` to a package that was missing it (#14609) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* * chore: issue template for bugs to have an SSCCE example (#14615) Ask customers to provide reproducible code snippets to reduce the amount of triage time required. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* * fix(cli): synth fails if there was an error when synthesizing the stack (#14613) All stacks created inside a pipeline stage will be flagged for validation. After synth is done, the CLI will validate all flagged stacks plus the stacks that were explicitly specified. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* * chore: annotate `aws-lambda-go` with `docker` requirement (#14618) The `nozem` build tool needs to know that `docker` is required to build/test this particular package. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* * feat(elbv2): preserveClientIp for NetworkTargetGroup (#14589) Allows users to configure client IP preservation for network target groups. See: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticloadbalancingv2-targetgroup-targetgroupattribute.html ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* * chore: npm-check-updates && yarn upgrade (#14620) Ran npm-check-updates and yarn upgrade to keep the `yarn.lock` file up-to-date. * chore(mergify): add @BenChaimberg to team roster * chore(release): 1.103.0 * chore: mark "otaviomacedo" as core contributor (#14619) Co-authored-by: Otavio Macedo <otaviomacedo@protonmail.com> * chore(cli): add npm command to upgrade notice (#14621) A colleague had to go look up the command to update the CDK CLI. It occurred to me that is probably common with developers who don't work with NPM on a daily basis, such as anyone who isn't developing in TypeScript or JavaScript. Put the necessary command right in the upgrade notice. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* * fix(lambda): custom resource fails to connect to efs filesystem (#14431) Fixes: #14430 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* * feat(cfnspec): cloudformation spec v35.2.0 (#14610) * feat: cloudformation spec v35.2.0 * add spec patches Co-authored-by: AWS CDK Team <aws-cdk@amazon.com> Co-authored-by: Elad Ben-Israel <benisrae@amazon.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * fix(lambda-nodejs): handler filename missing from error message (#14564) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* * chore(build): clarify prerequisites in CONTRIBUTING and verify before build (#14642) The build script at the top level of the repository performs a prerequisites check before beginning the build. This verification is not complete as it does not include the required checks for .NET and Python executables. Further, the prerequisites documentation in the contributing guide does not note the Docker requirement. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* Co-authored-by: Niranjan Jayakar <nija@amazon.com> Co-authored-by: OksanaH <34384274+OksanaH@users.noreply.github.com> Co-authored-by: Dominic Fezzie <fezzid@amazon.com> Co-authored-by: Curtis <curtis.eppel@cultureamp.com> Co-authored-by: Shinya Tsuda <shinya@dacci.org> Co-authored-by: Rico Huijbers <rix0rrr@gmail.com> Co-authored-by: Otavio Macedo <otaviomacedo@protonmail.com> Co-authored-by: Griffin Byatt <byatt.griffin@gmail.com> Co-authored-by: Ben Chaimberg <chaimber@amazon.com> Co-authored-by: AWS CDK Team <aws-cdk@amazon.com> Co-authored-by: Mitchell Valine <valinm@amazon.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Jerry Kindall <52084730+Jerry-AWS@users.noreply.github.com> Co-authored-by: Daniel Neilson <53624638+ddneilson@users.noreply.github.com> Co-authored-by: Elad Ben-Israel <benisrae@amazon.com>
In #14613, we introduced validation so that `cdk synth` would guard against incomplete context lookups. In the past, stacks with missing context would have successfully completed synthesis, propagated down the pipeline and caused confusing error messages during deployment. The new behavior was to error out early in case there was missing context. This broke people who had resorted to resolving context in the pipeline using multiple `synth` commands in `for`-loop: this used to work because the `synths` would be incomplete but silently succeed, but with the new validation the very first `cdk synth` would start failing and the `for` loop would never complete. This PR adds a `--no-validation` flag to `cdk synth` to stop the additional validation, so the `for` loop can complete successfully. The same behavior can be controlled with an environment variable, by setting `CDK_VALIDATION=false`. Fixes #15130.
In #14613, we introduced validation so that `cdk synth` would guard against incomplete context lookups. In the past, stacks with missing context would have successfully completed synthesis, propagated down the pipeline and caused confusing error messages during deployment. The new behavior was to error out early in case there was missing context. This broke people who had resorted to resolving context in the pipeline using multiple `synth` commands in `for`-loop: this used to work because the `synths` would be incomplete but silently succeed, but with the new validation the very first `cdk synth` would start failing and the `for` loop would never complete. This PR adds a `--no-validation` flag to `cdk synth` to stop the additional validation, so the `for` loop can complete successfully. The same behavior can be controlled with an environment variable, by setting `CDK_VALIDATION=false`. Fixes #15130.
In aws#14613, we introduced validation so that `cdk synth` would guard against incomplete context lookups. In the past, stacks with missing context would have successfully completed synthesis, propagated down the pipeline and caused confusing error messages during deployment. The new behavior was to error out early in case there was missing context. This broke people who had resorted to resolving context in the pipeline using multiple `synth` commands in `for`-loop: this used to work because the `synths` would be incomplete but silently succeed, but with the new validation the very first `cdk synth` would start failing and the `for` loop would never complete. This PR adds a `--no-validation` flag to `cdk synth` to stop the additional validation, so the `for` loop can complete successfully. The same behavior can be controlled with an environment variable, by setting `CDK_VALIDATION=false`. Fixes aws#15130.
…ck (aws#14613) All stacks created inside a pipeline stage will be flagged for validation. After synth is done, the CLI will validate all flagged stacks plus the stacks that were explicitly specified. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
In aws#14613, we introduced validation so that `cdk synth` would guard against incomplete context lookups. In the past, stacks with missing context would have successfully completed synthesis, propagated down the pipeline and caused confusing error messages during deployment. The new behavior was to error out early in case there was missing context. This broke people who had resorted to resolving context in the pipeline using multiple `synth` commands in `for`-loop: this used to work because the `synths` would be incomplete but silently succeed, but with the new validation the very first `cdk synth` would start failing and the `for` loop would never complete. This PR adds a `--no-validation` flag to `cdk synth` to stop the additional validation, so the `for` loop can complete successfully. The same behavior can be controlled with an environment variable, by setting `CDK_VALIDATION=false`. Fixes aws#15130.
All stacks created inside a pipeline stage will be flagged for validation. After synth is done, the CLI will validate all flagged stacks plus the stacks that were explicitly specified.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license