Accelerator Pipelines for Azure Devops

[picccard]

Overview/Summary

Added necessary pipelines for the ALZ Bicep Accelerator feature to work with Azure Devops Pipelines.
See #479, #483, #487 and #497 for previous accelerator PRs

This PR fixes/adds/changes/removes

1. Adds Azure Devops Pipelines in yaml

Breaking Changes

No breaking changes as this optional features of other aspects of ALZ-Bicep.

Testing Evidence

I've tested all but one pipeline, alz-bicep-4b.yml has not been tested as I don't use vWAN.

alz-bicep-1.yml

alz-bicep-2.yml

alz-bicep-3.yml

alz-bicep-4a.yml

alz-bicep-pr-1.yml

alz-bicep-pr-2.yml

As part of this Pull Request I have

• Read the Contribution Guide and ensured this PR is compliant with the guide
• Ensured the resource API versions in .bicep file/s I am adding/editing are using the latest API version possible
• Checked for duplicate Pull Requests
• Associated it with relevant GitHub Issues
• (ALZ Bicep Core Team Only) Associated it with relevant ADO Items
• Ensured my code/branch is up-to-date with the latest changes in the main branch
• Performed testing and provided evidence.
• Updated one or more of the following tests (if required)
  • Unit
  • Linting
  • E2E (End-To-End)
  • ValidateAzCloud (Base validation in Azure Cloud)
  • ValidateMcCloud (Base validation in Azure China Cloud)
• Updated relevant and associated documentation (e.g. Contribution Guide, Module READMEs, Wiki Docs etc.)
• If relevant, created or updated Code Tours here

[picccard]

@oZakari I've not played with your github workflows, but could you have a look at this?
Also see the last image, the 2nd PR pipeline runs for 15+ minutes as it lints the entire library of .json policyDefinitions, 400+ files. Would every PR have to trigger such a task or should more folders be excluded in the regex?

Lastly these AzDevops pipelines uses variables and does not follow the idea of a .env file yet.

[oZakari]

Hi @picccard, thank you very much for your contribution! We actually haven't even launched the Accelerator officially yet (likely will go into beta this week actually), but we were planning on incorporating an Azure Pipelines option in the near future so this is awesome!

As part of the Accelerator, we also built out a PowerShell module to essentially bootstrap the pipelines and create a development framework for ALZ-Bicep. Please see my most recent PR for more details. Now that you have provided the Azure pipelines, we'll be able to start adding some user input to the module to request what platform the user would prefer. We'll probably wait at least a few weeks to make sure there are no major issues during the beta before we support additional platforms though as a heads up.

As for your question about the super-linter, I see that you have VALIDATE_ALL_CODEBASE environment variable set to false. Therefore, it should only check the files during the pr if they are new or modified. Have you created a second PR after this one to verify?

Also, for the environment variables, GitHub Actions doesn't support loading in variables from a environment variables file either. I had to take the .env file and inject the values as shown here:

- name: Import Environment Variables from File
  run: cat ${{ env.ENV_FILE }} >> $GITHUB_ENV

The PowerShell module we built creates the .env file which is why we went with this approach. Also, we should be able to mimic a similar approach with Azure Pipelines using something like this.

Overall, this looks great though and thanks again! I will be on vacation for a couple of weeks, but my team can assist in the meantime as well.

[picccard]

👏 The powershell module for ALZ does some handy stuff with the creation of the entire folder structure and .env file!

✨ Support for loading values from the .env into the AzDo Pipelines is now present in this PR.

I've not found any solution for the super-linter... Running it locally in an AzurePipeline means

You need to pass the RUN_LOCAL flag to bypass some of the GitHub Actions checks

The flag:RUN_LOCAL will set: VALIDATE_ALL_CODEBASE to true. This means it will scan all the files in the directory you have mapped. If you want to only validate a subset of your codebase, map a folder with only the files you wish to have linted

As a result I've added some regex to exclude the policy assignment/definition lib folder.

[oZakari]

👏 The powershell module for ALZ does some handy stuff with the creation of the entire folder structure and .env file!

✨ Support for loading values from the .env into the AzDo Pipelines is now present in this PR.

I've not found any solution for the super-linter... Running it locally in an AzurePipeline means

You need to pass the RUN_LOCAL flag to bypass some of the GitHub Actions checks

The flag:RUN_LOCAL will set: VALIDATE_ALL_CODEBASE to true. This means it will scan all the files in the directory you have mapped. If you want to only validate a subset of your codebase, map a folder with only the files you wish to have linted

As a result I've added some regex to exclude the policy assignment/definition lib folder.

Awesome, thanks for adding the .env values to the pipelines!

As I'm back from vacation, I'll play around with the super-linter in Azure DevOps a bit over the next couple of days, but I think your approach makes sense. Will get back to you shortly.

[oZakari]

@picccard I've created a PR against your branch

Decided the best approach would be to just remove the upstream-releases folder entireley as changes shouldn't be made in that directory anyways. Also, everything within that directory would have already been linted by the pipelines within ALZ-Bicep.

[picccard]

@oZakari Good spot 😎

[oZakari]

LGTM and thanks again @picccard

[jtracey93]

@oZakari do we want to do the renaming here as per #537 to save ourselves work later?

[oZakari]

@jtracey93 We can just push this, and I'll make the changes for #537 for both GitHub and ADO in a separate PR for tracking.