Skip to content

Commit

Permalink
Closes #112 (#115)
Browse files Browse the repository at this point in the history 
* Fixes #113

* Add change graph classes

* Refactor changeset formatter to XDocument

* Graphviz suppot
Includes abstraction for calling an external API for SVG rendering

* Small refactor of logic for dependency compare

* Graph impovements
Formatting
Debuuger display
Remove dep on MS json package
Bump Firefly.CloudFormtion

* Add SVG pan/zoom

* Code tidy

* Change graph tests

* Disable warning 649
Field populated by embedded resource loader

* Update documentation

* Update doc

* Update doc

* Bump Firefly.CloudFormation
  • Loading branch information
@fireflycons
fireflycons committed Aug 3, 2021
1 parent cc9277f commit c3e5b02
Show file tree
Hide file tree
Showing 28 changed files with 2,010 additions and 182 deletions.
46 changes: 13 additions & 33 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,48 +1,26 @@
# PSCloudFormation
[![Build status](https://ci.appveyor.com/api/projects/status/fgt7d0icj7emc6hl/branch/master?svg=true)](https://ci.appveyor.com/project/fireflycons/pscloudformation/branch/master)

This module depends on [AWS.Tools](https://docs.aws.amazon.com/powershell/latest/userguide/pstools-welcome.html) version `4.1.6.0` or higher which you should install/upgrade to first
A set PowerShell cmdlets for manipulating AWS CloudFormation stacks.

Required AWS.Tools modules:
## Features

* AWS.Tools.Common
* AWS.Tools.CloudFormation
* AWS.Tools.S3
* Live display of stack events as a template is being applied when running synchronously (without `-PassThru` switch). Where nested stacks are involved, the events from these are also shown interleaved with those of the parent stack in chronlogical order.
* When using a workstation with a GUI, [detailed changeset information](https://fireflycons.github.io/PSCloudFormation/articles/changesets.html) can be brought up in a browser, including an SVG graph depicting the relationships between resources that are being modifed - like `terraform graph`
* Automatic packaging and upload to S3 of dependencies such as nested stack templates, lambdas, and other resources that require S3 references as described in [aws cloudformation package](https://docs.aws.amazon.com/cli/latest/reference/cloudformation/package.html)
* Close argument parity with similar cmdlets in AWS.Tools.CloudFormation
## Dependencies

## Version 4 Notes
This module depends on [AWS.Tools](https://docs.aws.amazon.com/powershell/latest/userguide/pstools-welcome.html) version `4.1.6.0` or higher which you should install/upgrade to first

This version is a complete re-write in C#. I found that it was becoming a cumbersome beast keeping it in pure PowerShell, taking longer to load the module, and certain parts of it were running quite slowly.
Required AWS.Tools modules:

Turning it into a binary module addresses the above problems and reduces complexity given the cmdlets share many common arguments meaning that inheritance can be used to reduce code duplication. It also gives me the chance to showcase three of my other projects: [Firefly.CloudFormation](https://github.com/fireflycons/Firefly.CloudFormation) which underpins this module, [PSDynamicParameters](https://github.com/fireflycons/PSDynamicParameters) which is a library for managing PowerShell Dynamic Parameters for C# cmdlets and [CrossPlatformZip](https://github.com/fireflycons/CrossPlatformZip) which creates zip files targeting Windows or Linux/Unix/MacOS from any source platform - needed for the packaging component of this module. Lambda does not like zip files that don't contain Unix permission attributes!
* AWS.Tools.Common

### New Documentation Site
## New Documentation Site

Head over [here](https://fireflycons.github.io/PSCloudFormation/index.html) for further reading and more in-depth discussion on the featues of this module.

### Breaking Changes

* Minimum requirement Windows PowerShell 5.1. All PowerShell Core versions are supported.
* Requires modular [AWS.Tools](https://github.com/aws/aws-tools-for-powershell/issues/67) - currently version `4.1.6.0` or higher. Monolithic AWSPowerShell is no longer supported (since PSCloudFormation v3.x). Future releases of this module will be version number aligned with the required version of `AWS.Tools` as and when enhancements are added in the space occupied by these cmdlets.
* Meaning of `-Wait` parameter has changed. This only applies to `Update-PSCFNStack` and means that update should not begin if at the time the cmdlet is called, the target stack is found to be being updated by another process. In this case the update will wait for the other update to complete. All PSCloudFormation cmdlets will wait for their own operation to run to completion unless `-PassThru` is present.
* Return type from the cmdlets has changed. Instead of being just a stack status or an ARN, it is a structure containing both, defined [here](https://fireflycons.github.io/Firefly-CloudFormation/api/Firefly.CloudFormation.Model.CloudFormationResult.html).

### Enhancements

* More use of colour in changeset and stack event display.
* All properties of create, update and delete stack are now supported.
* More complete support for determining AWS credentials from all sources.
* [Resource Import](https://fireflycons.github.io/PSCloudFormation/articles/resource-import.html) supported (since v3.x).
* [Dependency Packaging](https://fireflycons.github.io/PSCloudFormation/articles/lambda-packager.html) - For script based lambdas, it is possible to package dependent modules directly.
* Support for Python lambda dependency resultion via `requirements.txt`
* [Nested Changeset support](https://fireflycons.github.io/PSCloudFormation/articles/changesets.html) - With caveats! See documentation.
* [Changeset Detail view](https://fireflycons.github.io/PSCloudFormation/articles/changesets.html) - View changeset detail in browser.

### Gotchas

Due to the fact that the entire PowerShell process is a single .NET AppDomain, it is possible to fall into DLL hell. This module has various dependencies such as [YamlDotNet](https://github.com/aaubry/YamlDotNet). If something else in the current PowerShell session has loaded a different version of a dependent library like YamlDotNet, then you will get an assembly version clash when importing this module and the import will fail. Start a new PowerShell session and import there.

There is a way round this for pure .NET Core applications, but then I would have to target PowerShell Core only. The time isn't right for that yet, but if there's sufficient interest, then that could be the v5 release.

## How to Install

The module is published on the PowerShell Gallery and can be installed by following the instructions there.
Expand Down Expand Up @@ -93,3 +71,5 @@ Thanks to
* [ramblingcookiemonster](http://ramblingcookiemonster.github.io/) for `PSDepend` and `PSDeploy` used in parts of the build of this project.
* [Antoine Aubry](https://github.com/aaubry/YamlDotNet) for `YamlDotNet`
* [Olivier Duhart](https://github.com/b3b00/csly) for `csly` used to parse python `METADATA` files.
* [Alexandre Rabérin](https://github.com/KeRNeLith/QuikGraph) for QuikGraph
* [Ian Webster](https://github.com/typpo/quickchart) and [QuickChart.io](https://quickchart.io/) for SVG rendering API.
30 changes: 30 additions & 0 deletions RELEASENOTES.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,36 @@
* Better authentication handling
* Some breaking changes - see README

## Version 4 Notes

This version is a complete re-write in C#. I found that it was becoming a cumbersome beast keeping it in pure PowerShell, taking longer to load the module, and certain parts of it were running quite slowly.

Turning it into a binary module addresses the above problems and reduces complexity given the cmdlets share many common arguments meaning that inheritance can be used to reduce code duplication. It also gives me the chance to showcase three of my other projects: [Firefly.CloudFormation](https://github.com/fireflycons/Firefly.CloudFormation) which underpins this module, [PSDynamicParameters](https://github.com/fireflycons/PSDynamicParameters) which is a library for managing PowerShell Dynamic Parameters for C# cmdlets and [CrossPlatformZip](https://github.com/fireflycons/CrossPlatformZip) which creates zip files targeting Windows or Linux/Unix/MacOS from any source platform - needed for the packaging component of this module. Lambda does not like zip files that don't contain Unix permission attributes!

### Breaking Changes

* Minimum requirement Windows PowerShell 5.1. All PowerShell Core versions are supported.
* Requires modular [AWS.Tools](https://github.com/aws/aws-tools-for-powershell/issues/67) - currently version `4.1.6.0` or higher. Monolithic AWSPowerShell is no longer supported (since PSCloudFormation v3.x). Future releases of this module will be version number aligned with the required version of `AWS.Tools` as and when enhancements are added in the space occupied by these cmdlets.
* Meaning of `-Wait` parameter has changed. This only applies to `Update-PSCFNStack` and means that update should not begin if at the time the cmdlet is called, the target stack is found to be being updated by another process. In this case the update will wait for the other update to complete. All PSCloudFormation cmdlets will wait for their own operation to run to completion unless `-PassThru` is present.
* Return type from the cmdlets has changed. Instead of being just a stack status or an ARN, it is a structure containing both, defined [here](https://fireflycons.github.io/Firefly-CloudFormation/api/Firefly.CloudFormation.Model.CloudFormationResult.html).

### Enhancements

* More use of colour in changeset and stack event display.
* All properties of create, update and delete stack are now supported.
* More complete support for determining AWS credentials from all sources.
* [Resource Import](https://fireflycons.github.io/PSCloudFormation/articles/resource-import.html) supported (since v3.x).
* [Dependency Packaging](https://fireflycons.github.io/PSCloudFormation/articles/lambda-packager.html) - For script based lambdas, it is possible to package dependent modules directly.
* Support for Python lambda dependency resultion via `requirements.txt`
* [Nested Changeset support](https://fireflycons.github.io/PSCloudFormation/articles/changesets.html) - With caveats! See documentation.
* [Changeset Detail view](https://fireflycons.github.io/PSCloudFormation/articles/changesets.html) - View changeset detail in browser.

### Gotchas

Due to the fact that the entire PowerShell process is a single .NET AppDomain, it is possible to fall into DLL hell. This module has various dependencies such as [YamlDotNet](https://github.com/aaubry/YamlDotNet). If something else in the current PowerShell session has loaded a different version of a dependent library like YamlDotNet, then you will get an assembly version clash when importing this module and the import will fail. Start a new PowerShell session and import there.

There is a way round this for pure .NET Core applications, but then I would have to target PowerShell Core only. The time isn't right for that yet, but if there's sufficient interest, then that could be the v5 release.

# 3.0.0

* Support for Resource Import
Expand Down
18 changes: 15 additions & 3 deletions docfx/articles/changesets.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,19 @@ When a GUI is detected, [Update-PSCFNStack](xref:Update-PSCFNStack) provides an

[New-PSCFNChangeset](xref:Update-PSCFNChangeset) will perform the above by addition of `-ShowInBrowser` switch argument.

The HTML document contains a formatted view of each change along with the detail provided in the `JSON Changes` view in the CloudFormation console.
The HTML document contains a formatted view of each change along with the detail provided in the `JSON Changes` view in the CloudFormation console. An SVG graph of relationships between modified resouces is also provided, which can be manipulated using the mouse. This has the following key

| Icon | Meaning |
|-----------------------|-----------------------------------------------------------------------------|
| Solid box, green font | New resource. |
| Solid box, blue font | Imported resource. |
| Dashed box, red font | Resource is being deleted. |
| Box, green fill | Resource is being modified, without replacement. |
| Box, amber fill | Resource is being modified, conditional replacement. |
| Box, red fill | Resource is being modified, and will be REPLACED. |
| Diamond | A parameter. Text is the parameter's name. |
| Ellipse | Direct modification, e.g. user changed a property directly in the template. |
| Connectors | Label shows property being changed on target resource. |

Additionally, only relevant information from the JSON changes are diplayed. Properties that are `null` in the JSON change are hidden. Change detail is only shown for `Modify` changes, as there isn't any relevant detail for `Add` or `Remove`. Full detail is intially hidden, however a button will show to un-hide this detail where it exists.

Expand All @@ -72,9 +84,9 @@ The HTML view relies on [jQuery](https://jquery.com) and [Bootstrap](https://get

### Nested Changeset Creation

There is a [server-side bug in this which I reported](https://github.com/fireflycons/aws-nested-changeset-bug) and has been acknowledged by AWS. If a nested stack has no changes, then any outputs of this nested stack are incorrectly deemed to have *all* been changed. Any other stacks in the nest which use these outputs as parameters then incorrectly show resource changes caused by these paameters.
There is a [server-side bug in this which I reported](https://github.com/fireflycons/aws-nested-changeset-bug) back in Dec 2020 and has been acknowledged by AWS. If a nested stack has no changes, then any outputs of this nested stack are incorrectly deemed to have *all* been changed. Any other stacks in the nest which use these outputs as parameters then incorrectly show resource changes caused by these paameters.

This issue renders the nested changeset feature fairly useless at present. Since the bug is server-side (i.e. within AWS itself), as soon as AWS roll out the fix, then PSCloudFormation will work with this feature without the need for a new release.

AWS estimate for the fix is March 2021.
AWS have still not fixed this as of July 2021!

Loading

0 comments on commit c3e5b02

Please sign in to comment.