Skip to content

Commit

Permalink
Add e2e tests (#1)
Browse files Browse the repository at this point in the history
* Add e2e tests

* Output version  of ALZ module

* Add additional logging

* Try checking out to sub folder

* Add debugging

* Add some debugging

* Add debugging

* Install moldule from source

* Rename folder to acvoid confusion

* Include hidden files

* Add missing field

* Fix duplicate variable

* Fix default management group id

* Make paths absolute

* Fix tests

* Fix typo

* Faster PS module install and bicep tests

* Format

* Add bicep inputs

* Add missing bicep input

* Fix bicep matrix

* Format

* Fix linting

* Fix foratting

* Minro tidy
  • Loading branch information
jaredfholgate authored Mar 15, 2024
1 parent 5d089f2 commit 65f3d95
Show file tree
Hide file tree
Showing 37 changed files with 1,176 additions and 69 deletions.
111 changes: 111 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
name: "Bug report \U0001F41B"
description: Report errors or unexpected behaviour
title: "bug:"
labels: [bug]
body:
- type: markdown
attributes:
value: |
Thank you for taking the time to fill out a bug report.
If you are not running the latest version of this module, please try to reproduce your bug with the latest version before opening an issue.
- type: checkboxes
attributes:
label: Is there an existing issue for this?
description: Please search to see if an issue already exists for the bug you encountered.
options:
- label: I have searched the existing issues
required: true
- type: dropdown
attributes:
label: Bootstrap or Starter Module Deployment? (Required)
description: Do you have a problem with deploying the Bootstrap Resources or the Starter Module through CI/CD?
multiple: false
options:
- bootstrap
- starter_module
validations:
required: true
- type: input
id: ps_module
attributes:
label: ALZ-PowerShell-Module Version (Required)
description: Which ALZ-PowerShell-Module version are you using?
placeholder: Example value, 1.1.6
validations:
required: true
- type: input
id: alz_tf_acc
attributes:
label: alz-terraform-accelerator Version (Required)
description: Which alz-terraform-accelerator version are you using?
placeholder: Example value, 1.1.0
validations:
required: true
- type: input
id: starter_module
attributes:
label: Starter Module (Required)
description: Which Starter Module are you using?
placeholder: Example value, complete
validations:
required: true
- type: textarea
id: inputs
attributes:
label: Input arguments of the ALZ-PowerShell-Module (Required)
description: |
Please provide the input arguments of the ALZ-PowerShell-Module that can reproduce the issue. If not stated explicitly please find these details in the `cache-bootstrap-<VCS>.json` and `cache-starter-<STARTER MODULE NAME>.json` files in the copy of the alz-terraform-accelerator that the ALZ-PowerShell-Module downloaded on execution.
render: YAML
validations:
required: true
- type: textarea
id: config
attributes:
label: Contents of the configuration file used for the `complete` starter module. (Optional)
description: |
Please provide contents of the configuration file used for the `complete` starter module.
render: YAML
- type: textarea
id: debug
attributes:
label: Debug Output/Panic Output (Optional)
description: |
For long debug logs please provide a link to a GitHub Gist containing the complete debug output. Please do NOT paste the debug output in the issue; just paste a link to the Gist.
To obtain the debug output, see the [Terraform documentation on debugging](https://www.terraform.io/docs/internals/debugging.html).
render: shell
- type: textarea
id: expected
attributes:
label: Expected Behaviour (Required)
description: What should have happened?
validations:
required: true
- type: textarea
id: actual
attributes:
label: Actual Behaviour (Required)
description: What actually happened?
validations:
required: true
- type: textarea
id: reproduce
attributes:
label: Steps to Reproduce (Optional)
description: |
Please list the steps required to reproduce the issue, e.g.
- type: input
id: facts
attributes:
label: Important Factoids (Optional)
description: |
Are there anything atypical about your accounts that we should know? For example: Running in a Azure China/Germany/Government?
- type: textarea
id: references
attributes:
label: References (Optional)
description: |
Information about referencing Github Issues: https://help.github.com/articles/basic-writing-and-formatting-syntax/#referencing-issues-and-pull-requests
Are there any other GitHub issues (open or closed) or pull requests that should be linked here? Such as vendor documentation?
2 changes: 2 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
---
blank_issues_enabled: false
31 changes: 31 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
name: "Feature request \U0001F680"
about: Suggest an idea for this project
title: 'Feature Request'
labels: 'needs triage :warning:'
assignees: ''
---

### Community Note

<!--- Please keep this note for the community --->

- Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment

<!--- Thank you for keeping this note for the community --->

### Description

**Is your feature request related to a problem?**

<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->

**Describe the solution you'd like**

<!-- A clear and concise description of what you want to happen. -->

<!-- A clear and concise description of any alternative solutions or features you've considered. -->

**Additional context**
27 changes: 27 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
<!-- Thank you for submitting a Pull Request. Please fill out the template below.-->
## Overview/Summary

Replace this with a brief description of what this Pull Request fixes, changes, etc.

## This PR fixes/adds/changes/removes

1. *Replace me*
2. *Replace me*
3. *Replace me*

### Breaking Changes

1. *Replace me*
2. *Replace me*

## Testing Evidence

Please provide any testing evidence to show that your Pull Request works/fixes as described and planned (include screenshots, if appropriate).

## As part of this Pull Request I have

- [ ] Checked for duplicate [Pull Requests](https://github.com/Azure/alz-terraform-accelerator/pulls)
- [ ] Associated it with relevant [issues](https://github.com/Azure/alz-terraform-accelerator/issues), for tracking and closure.
- [ ] Ensured my code/branch is up-to-date with the latest changes in the `main` [branch](https://github.com/Azure/alz-terraform-accelerator/tree/main)
- [ ] Performed testing and provided evidence.
- [ ] Updated relevant and associated documentation.
12 changes: 12 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates

version: 2
updates:
- package-ecosystem: "terraform" # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "weekly"
36 changes: 36 additions & 0 deletions .github/linters/.markdown-lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
###########################
###########################
## Markdown Linter rules ##
###########################
###########################

# Linter rules doc:
# - https://github.com/DavidAnson/markdownlint
#
# Note:
# To comment out a single error:
# <!-- markdownlint-disable -->
# any violations you want
# <!-- markdownlint-restore -->
#

###############
# Rules by id #
###############
MD004: false # ul-style - Unordered list style
MD007:
indent: 2 # ul-indent - Unordered list indentation
MD013:
line_length: 400 # line-length - Line length
MD026:
punctuation: ".,;:!。,;:" # no-trailing-punctuation - Trailing punctuation in heading
MD029: false # ol-prefix - Ordered list item prefix
MD033: false # no-inline-html - Inline HTML
MD036: false # no-emphasis-as-heading/no-emphasis-as-header - Emphasis used instead of a heading
MD041: false # first-line-heading/first-line-h1 - First line in a file should be a top-level heading

#################
# Rules by tags #
#################
blank_lines: false # MD012, MD022, MD031, MD032, MD047
11 changes: 11 additions & 0 deletions .github/linters/.yaml-lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
extends: default

rules:
# 200 chars should be enough, but don't fail if a line is longer
line-length:
max: 200
level: warning
truthy:
check-keys: false
level: warning
34 changes: 34 additions & 0 deletions .github/tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# End to End Tests

## Overview

The end to end tests can be found in the ./github/workflows/end-to-end-test.yml action.

- The tests are triggered by `workflow_dispatch` or on `pull_request` only when the label `PR: Safe to test 🧪` has been added to the PR.
- The tests run against the environment `CSUTF`, which requires a manual approval to deploy.
- The tests run as a matrix, targeting different OS, VCS, Terraform and Auth Methods.

## Test Process

The test follow this process:

1. Check out the module from the PR merge branch.
1. Generate an `inputs.json` file that is used to override the prompts in the `ALZ` PowerShell module.
1. Install the `ALZ` PowerShell module.
1. Get the latest version tag for the live accelerator module.
1. Copy the `boostrap` and `template` folders into a folder named by the latest version tag.
1. Run the `New-New-ALZEnvironment` function to deploy the terraform.
1. Run a `terraform destroy` to clean up the environment.

## Environment

The tests use a set of environemnts to managed by the ALZ team. These are:

- Azure:
- Tenant: CSU TF
- Subscription: csu-tf-devops
- User Assigned Managed Identity: alz-terraform-accelerator-cd-tests-identity (this has federated credentials)
- Azure DevOps
- Organisation: microsoft-azure-landing-zones-cd-tests
- GiHub
- Organisation: microsoft-azure-landingzones-cd-tests
15 changes: 15 additions & 0 deletions .github/tests/cleanup-scripts/cleanup_github-repositories.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# This file can be used to clean up GitHub repositories if there has been an issue with the End to End tests.
# CAUTION: Make sure you are connected to the correct organization before running this script!
$repos = gh repo list microsoft-azure-landing-zones-cd-tests --json name,owner | ConvertFrom-Json

$repos | ForEach-Object -Parallel {
$match = "*229*"
$repoName = "$($_.owner.login)/$($_.name)"

if($repoName -like $match)
{
Write-Host "Deleting repo: $repoName"
gh repo delete $repoName --yes

}
} -ThrottleLimit 10
9 changes: 9 additions & 0 deletions .github/tests/cleanup-scripts/cleanup_resouce_groups.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# This file can be used to clean up Resource Groups if there has been an issue with the End to End tests.
# CAUTION: Make sure you are connected to the correct subscription before running this script!
az account show
$resourceGroups = az group list --query "[?contains(name, '254-')]" | ConvertFrom-Json

$resourceGroups | ForEach-Object -Parallel {
Write-Host "Deleting resource group: $($_.name)"
az group delete --name $_.name --yes
} -ThrottleLimit 10
13 changes: 13 additions & 0 deletions .github/tests/scripts/Install-AcceleratorModule.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
param (
[string]$ModuleUrl = "https://github.com/Azure/ALZ-PowerShell-Module",
[string]$ModuleBranch = "main"
)

$targetDirectory = "./accelerator-powershell-module"

if(!(Test-Path $targetDirectory)) {
git clone -b $ModuleBranch $ModuleUrl $targetDirectory
}

./accelerator-powershell-module/actions_bootstrap_for_e2e_tests.ps1 | Out-String | Write-Verbose
Invoke-Build -File ./accelerator-powershell-module/src/ALZ.build.ps1 BuildAndInstallOnly | Out-String | Write-Verbose
Loading

0 comments on commit 65f3d95

Please sign in to comment.