Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MAKE CLOUDFORMATION ROLLBACKS FASTER AND DONT BLOCK SUBSEQUENTIAL DEPLOYMENTS #30029

Closed
2 tasks
AllanOricil opened this issue May 1, 2024 · 4 comments
Closed
2 tasks
Labels
@aws-cdk/aws-cloudformation Related to AWS CloudFormation feature-request A feature should be added or improved. p2

Comments

@AllanOricil
Copy link

Describe the feature

3 minutes to deploy and end up with an error, then a rollback initialized. This rollback took longer than the deploy? why? Is it done manually by an engineer?

image

Use Case

NOT GET PISSED OFF WITH THE AMOUNT OF TIME IT TAKES TO RETRY A CDK DEPLOYMENT

Proposed Solution

ROLLBACKS could be done instantly if all resources were put on a "trash bin" and later handled by AWS internal services. Why make us wait for you to purge them? Put them on a separate place and handle the deleting afterwards. Consider user initiated deleteion events as the a proof the User agrees to have those resources deleted. I'm extremelly irritaded right now because a simple stack that takes 3 minutes to be deployed took more than 20 minutes to be rolledback. This is a horrible DX.

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

CDK version used

2.139

Environment details (OS name and version, etc.)

macos

@AllanOricil AllanOricil added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels May 1, 2024
@github-actions github-actions bot added the @aws-cdk/aws-cloudformation Related to AWS CloudFormation label May 1, 2024
@pahud
Copy link
Contributor

pahud commented May 1, 2024

Thank you for your feedback and we appreciate it. This is actually not something CDK can do but cloudformation and we suggest submit your feedback to cloudformation. I still have some suggestions for you though to help you accelerate your development cycle from CDK's perspective.

  1. Incremental deployments for large stacks - if you have a lot of resources in a single stack, instead of deploying them all at once, try to deploy part of them every single time. This avoids rolling back all deployed resources that could take a long time.
  2. Use cdk deploy -R to stop rolling back and continue your deployment from where it failed. This allows you to inspect the failure description and resume at where it failed.
  3. If the failed stack has to rollback and it takes a lot of time, instead of waiting for the rollback complete, you can simply change a new stack name in your app.ts and redeploy it as a new stack. For example:
const app = new cdk.App();
new MySampleStack(app, 'my-stack'); // <== change `my-stack` to `my-stack2` and deploy

This allows you to have a separate stack deploying for a very quick inspection while the old one would eventually complete its rollback and stop.

  1. Last but not least, we are implementing the cloudformation optimistic stabilization strategy support in CDK CLI and hopefully you will see a huge improvement from the perspective of CDK. See this PR for more details and we welcome your feedback.

I hope you find the tips above useful and enjoy your CDK journey.

@pahud pahud added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. p2 and removed needs-triage This issue or PR still needs to be triaged. labels May 1, 2024
@AllanOricil
Copy link
Author

AllanOricil commented May 1, 2024

@pahud thanks for the insights!

I just got really frustraded yesterday due to the amount of time I had to wait and because I was facing again an error I thought I had already fixed.

@github-actions github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label May 1, 2024
Copy link

github-actions bot commented May 1, 2024

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

@aws-cdk-automation
Copy link
Collaborator

Comments on closed issues and PRs are hard for our team to see. If you need help, please open a new issue that references this one.

@aws aws locked as resolved and limited conversation to collaborators Jul 25, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
@aws-cdk/aws-cloudformation Related to AWS CloudFormation feature-request A feature should be added or improved. p2
Projects
None yet
Development

No branches or pull requests

3 participants