Feature: Wave Config for Targets

[StewartW]

Issue #, if available: #290

Description of changes:
Introduction of an optional wave configuration in the target schema.

This contains a configurable int that defines the wave size. This wave size is then used to create batches inside the target structure class.

These batches are then passed into the cdk stacks and accessed in a similar fashion to how targets were already accessed.
This PR doesn't have tests yet, as I noticed that there's no test for pipeline generation beyond Source / Build.

The default batch size is set to 50 (Max actions per stage) and in the default adf_pipeline_type, it will create one stage per wave, allowing for OUs with more than 50 accounts to be seperated over multiple stages.

Also added in an exclude property to the target for removing accounts by Id. (issue: #145 )
Also fixed an issue where if an account shared a tag with an OU and that tag was used as a Target it would break. (Issue: #296 )

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[sbkok]

Thank you for adding this!
Only a few minor comments to fix. I'm looking forward to merge this one.

[sbkok]

Thank you! This looks great.

[deltagarrett]

Very minor fix to address.

[deltagarrett]

Minor typos to fix.

[deltagarrett]

LGTM

[sbkok]

LGTM!

[AngeloGelmini]

When I launch the aws-deployment-framework-pipelines I get this error:

Traceback (most recent call last):
  File "/codebuild/output/src433857646/src/adf-build/cdk/generate_pipeline_inputs.py", line 194, in <module>
    main()
  File "/codebuild/output/src433857646/src/adf-build/cdk/generate_pipeline_inputs.py", line 190, in main
    thread.join()
  File "/codebuild/output/src433857646/src/adf-build/python/thread.py", line 30, in join
    raise self.exc
  File "/codebuild/output/src433857646/src/adf-build/python/thread.py", line 20, in run
    self.ret = self._target(
  File "/codebuild/output/src433857646/src/adf-build/cdk/generate_pipeline_inputs.py", line 117, in worker_thread
    pipeline.template_dictionary["targets"].append(target.target_structure.generate_waves())
AttributeError: 'dict' object has no attribute 'target_structure'

My deployment map is:

pipelines:
  - name: parameters
    default_providers:
      source:
        provider: codecommit
        properties:
          account_id: 111111111111111 # id source account
      build:
        provider: codebuild
      deploy:
        provider: cloudformation
        properties:
          action: replace_on_failure
    params:
      notification_endpoint: ...
    targets: # accounts where to deploy resources
      - target: 99999999999 # deployment account
        name: name-
        regions: eu-central-1

It seems that in the line 117 target is used as class but target instead is a dict: for target in p.get('targets', []).
This cause the error in the line 117: pipeline.template_dictionary["targets"].append(target.target_structure.generate_waves())

I think that the rollback of this change to:
pipeline.template_dictionary["targets"].append(target_structure.generate_waves())
can fix the problem

[sbkok]

Hi @AngeloGelmini, thanks for reporting this.
I have fixed this issue together with Stewart, I will open the PR for that fix shortly.

I'm sure you are aware that Wave Config is not released as part of ADF yet. Therefore we don't recommend relying on that yet, as it might be (or actually is as you noticed) unstable.

We are actively debugging the code to make sure that the different features and changes introduced for the next release work together before we release ADF v3.2.0.