Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy error reporting is not showing the reason of failure when using nested stacks. #5974

Open
xoadrian opened this issue Jan 27, 2020 · 16 comments
Labels
bug This issue is a bug. effort/medium Medium work item – several days of effort p1 package/tools Related to AWS CDK Tools or CLI

Comments

@xoadrian
Copy link

xoadrian commented Jan 27, 2020

main/root stack is not properly reporting errors from nested stacks, there is no way to know why it actually failed because the error is generalized that some resources could not be created, but why, there is no reason. The only chance to see the error is to constantly monitor the deployment on AWS CloudFormation Console, and once it starts to roll back and delete nested stacks you can quickly see why the nested stack failed in events tab.

E.g.:

creating CloudFormation changeset...
 0/3 | 2:07:56 PM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata         | CDKMetadata
 0/3 | 2:07:56 PM | CREATE_IN_PROGRESS   | AWS::CloudFormation::Stack | appsync-bizon.NestedStack/appsync-bizon.NestedStackResource (appsyncbizonNestedStackappsyncbizonNestedStackResource814A53CF)
 0/3 | 2:07:57 PM | CREATE_IN_PROGRESS   | AWS::CloudFormation::Stack | appsync-bizon.NestedStack/appsync-bizon.NestedStackResource (appsyncbizonNestedStackappsyncbizonNestedStackResource814A53CF) Resource creation Initiated
 0/3 | 2:07:58 PM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata         | CDKMetadata Resource creation Initiated
 1/3 | 2:07:58 PM | CREATE_COMPLETE      | AWS::CDK::Metadata         | CDKMetadata
 2/3 | 2:08:19 PM | CREATE_FAILED        | AWS::CloudFormation::Stack | appsync-bizon.NestedStack/appsync-bizon.NestedStackResource (appsyncbizonNestedStackappsyncbizonNestedStackResource814A53CF) Embedded stack arn:aws:cloudformation:eu-west-2:452364846299:stack/ProductName-bizon-appsyncbizonNestedStackappsyncbizonNestedStackResource814A53CF-4NAE12ZLOQGT/a92a4b40-3c46-11ea-9fec-06452e71541c was not successfully created: The following resource(s) failed to create: [dynamodbstackbizonNestedStackdynamodbstackbizonNestedStackResource0AACEAF0, productnameresolversNestedStackproductnameresolversNestedStackResource42DF9B6A].

as you can see there is no reason why it failed, just saying that some resources failed to create. While actually the error on the nested stack events tab is:

E.g.:

Service role arn not specified (Service: AWSAppSync; Status Code: 400; Error Code: BadRequestException; Request ID: 5a9b00be-f21c-4cea-97ad-3d0e8552c5e5)

or

Only one resolver is allowed per field. (Service: AWSAppSync; Status Code: 400; Error Code: BadRequestException; Request ID: 39813c06-7d28-4d72-be32-f05785f4338e)
  QueryListEvents, WikiArticleLastRevision

or

Property validation failure: [Value for property {/Name} does not match pattern {[_A-Za-z][_0-9A-Za-z]*}]

So there is no way to know about those errors because on roll back those stacks are removed, and developer will never know why it failed.

Reproduction Steps

Just add a nested stack and try to simulate the above error examples:

  1. Create AppSync nested stack but don't create service roll for it, and try to create another resource which should rely on that service role.
  2. Name a resource in a nested stack improperly with - or .
  3. Create 2 same resolver in one nested stack.
    etc ... basically any error that will be thrown inside nested stack

Error Log

already specified above

Environment

  • CLI Version : 1.21.0
  • Framework Version:
  • OS : Windows
  • Language : Typescript

Other

I found this issue opened as a feature #4489 but I don't think it is a feature, rather it is a bug, since the feature of having Nested Stacks exists, this means that error reporting for nested stack to be chained in the parent wasn't.


This is 🐛 Bug Report

@xoadrian xoadrian added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jan 27, 2020
@SomayaB SomayaB added the package/tools Related to AWS CDK Tools or CLI label Jan 27, 2020
@shivlaks shivlaks added the p1 label Jan 28, 2020
@SomayaB SomayaB removed the needs-triage This issue or PR still needs to be triaged. label Mar 5, 2020
@SomayaB SomayaB added the in-progress This issue is being actively worked on. label May 5, 2020
@SomayaB SomayaB assigned eladb and unassigned shivlaks May 5, 2020
@SomayaB SomayaB removed the package/tools Related to AWS CDK Tools or CLI label May 5, 2020
@eladb eladb added package/tools Related to AWS CDK Tools or CLI and removed in-progress This issue is being actively worked on. p1 labels Aug 4, 2020
@eladb eladb assigned shivlaks and unassigned eladb Aug 4, 2020
@shivlaks shivlaks added p1 effort/medium Medium work item – several days of effort labels Aug 20, 2020
@NGL321 NGL321 assigned rix0rrr and unassigned shivlaks Jan 25, 2021
@VictorMorenoJimenez
Copy link

Hi, any updates on this? I'm working with nested stacks and it is pretty annoying to debug errors. Thanks!

@nlaffey-icario
Copy link

I'm also trying to learn CDK right now and wondering how I'm supposed to debug anything when all I'm getting is something like "The following resource(s) failed to create: [Serverapp, EbsStackawselasticbeanstalkec2roleD9735CA4]."

@ghost
Copy link

ghost commented Nov 8, 2021

News on this? Thanks!

@cosmin5mins
Copy link

Still an issue for us

@karlskellenPFG
Copy link

wack

@Zackhardtoname
Copy link

Just to highlight the OP's solution

The only chance to see the error is to constantly monitor the deployment on AWS CloudFormation Console

It actually worked for and unblocked me

@RcSepp
Copy link

RcSepp commented Jun 11, 2022

I faced the same issue, but found the failed nested stack in the CloudFormation CLI under stacks and when toggling the dropdown from "Active" to "Deleted". The stack events for that one contained the failure reason.

@songzh-AZ
Copy link

same issue still, any dev willing to look for this?

@fredtcaroli
Copy link

Can we get an update on this? Super annoying to debug

@DavidDryja
Copy link

Update please? It would help to see an error when debugging.

@Aeolun
Copy link

Aeolun commented Aug 4, 2023

You can actually get the error on the CloudFormation console by going to the parent stack, finding the event for the child stack that failed, copying the name of the stack. Going to the sidebar and selecting 'Deleted Stacks', then searching for the child stack name.

Then look through the events of the deleted child stack to find the actual issue.

This is very far from user friendly.

@cscetbon
Copy link

@TheRealAmazonKendra It seems the closed PR didn't fix the issue. Any update on this ? it's been 2 years ...

@LeoStage
Copy link

similar issues here, I'm receiving an error when try to deploy and getting this message
❌ Deployment failed: Error: The stack named xxx failed to deploy: UPDATE_ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: See the details in CloudWatch Log Stream: 2023/09/25/[$LATEST]1865cd90f7eb4da6aac8239dad6573d4 (RequestId: b550d905-e34b-41f9-a9b6-0c87986425f0)

When I went to the logs there is no information about the error

@Amplifiyer
Copy link
Contributor

Change #27318 is released with aws-cdk 2.100.0. Can you test with this version and see if this issue is resolved?

@StevenGBrown
Copy link

Change #27318 is released with aws-cdk 2.100.0. Can you test with this version and see if this issue is resolved?

Worked for me with aws-cdk 2.123.0.

@jpbndl
Copy link

jpbndl commented Mar 1, 2024

If it is CREATE_FAILED and it happened to rollback, please filter the CloudFormation list and select "Deleted". The actual reason will be in this stack which has been rolled back or been deleted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. effort/medium Medium work item – several days of effort p1 package/tools Related to AWS CDK Tools or CLI
Projects
None yet
Development

Successfully merging a pull request may close this issue.