Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add e2e tests #1

Merged
merged 25 commits into from
Mar 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 111 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
name: "Bug report \U0001F41B"
description: Report errors or unexpected behaviour
title: "bug:"
labels: [bug]
body:
- type: markdown
attributes:
value: |
Thank you for taking the time to fill out a bug report.

If you are not running the latest version of this module, please try to reproduce your bug with the latest version before opening an issue.
- type: checkboxes
attributes:
label: Is there an existing issue for this?
description: Please search to see if an issue already exists for the bug you encountered.
options:
- label: I have searched the existing issues
required: true
- type: dropdown
attributes:
label: Bootstrap or Starter Module Deployment? (Required)
description: Do you have a problem with deploying the Bootstrap Resources or the Starter Module through CI/CD?
multiple: false
options:
- bootstrap
- starter_module
validations:
required: true
- type: input
id: ps_module
attributes:
label: ALZ-PowerShell-Module Version (Required)
description: Which ALZ-PowerShell-Module version are you using?
placeholder: Example value, 1.1.6
validations:
required: true
- type: input
id: alz_tf_acc
attributes:
label: alz-terraform-accelerator Version (Required)
description: Which alz-terraform-accelerator version are you using?
placeholder: Example value, 1.1.0
validations:
required: true
- type: input
id: starter_module
attributes:
label: Starter Module (Required)
description: Which Starter Module are you using?
placeholder: Example value, complete
validations:
required: true
- type: textarea
id: inputs
attributes:
label: Input arguments of the ALZ-PowerShell-Module (Required)
description: |
Please provide the input arguments of the ALZ-PowerShell-Module that can reproduce the issue. If not stated explicitly please find these details in the `cache-bootstrap-<VCS>.json` and `cache-starter-<STARTER MODULE NAME>.json` files in the copy of the alz-terraform-accelerator that the ALZ-PowerShell-Module downloaded on execution.
render: YAML
validations:
required: true
- type: textarea
id: config
attributes:
label: Contents of the configuration file used for the `complete` starter module. (Optional)
description: |
Please provide contents of the configuration file used for the `complete` starter module.
render: YAML
- type: textarea
id: debug
attributes:
label: Debug Output/Panic Output (Optional)
description: |
For long debug logs please provide a link to a GitHub Gist containing the complete debug output. Please do NOT paste the debug output in the issue; just paste a link to the Gist.

To obtain the debug output, see the [Terraform documentation on debugging](https://www.terraform.io/docs/internals/debugging.html).
render: shell
- type: textarea
id: expected
attributes:
label: Expected Behaviour (Required)
description: What should have happened?
validations:
required: true
- type: textarea
id: actual
attributes:
label: Actual Behaviour (Required)
description: What actually happened?
validations:
required: true
- type: textarea
id: reproduce
attributes:
label: Steps to Reproduce (Optional)
description: |
Please list the steps required to reproduce the issue, e.g.
- type: input
id: facts
attributes:
label: Important Factoids (Optional)
description: |
Are there anything atypical about your accounts that we should know? For example: Running in a Azure China/Germany/Government?
- type: textarea
id: references
attributes:
label: References (Optional)
description: |
Information about referencing Github Issues: https://help.github.com/articles/basic-writing-and-formatting-syntax/#referencing-issues-and-pull-requests

Are there any other GitHub issues (open or closed) or pull requests that should be linked here? Such as vendor documentation?
2 changes: 2 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
---
blank_issues_enabled: false
31 changes: 31 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
name: "Feature request \U0001F680"
about: Suggest an idea for this project
title: 'Feature Request'
labels: 'needs triage :warning:'
assignees: ''
---

### Community Note

<!--- Please keep this note for the community --->

- Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment

<!--- Thank you for keeping this note for the community --->

### Description

**Is your feature request related to a problem?**

<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->

**Describe the solution you'd like**

<!-- A clear and concise description of what you want to happen. -->

<!-- A clear and concise description of any alternative solutions or features you've considered. -->

**Additional context**
27 changes: 27 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
<!-- Thank you for submitting a Pull Request. Please fill out the template below.-->
## Overview/Summary

Replace this with a brief description of what this Pull Request fixes, changes, etc.

## This PR fixes/adds/changes/removes

1. *Replace me*
2. *Replace me*
3. *Replace me*

### Breaking Changes

1. *Replace me*
2. *Replace me*

## Testing Evidence

Please provide any testing evidence to show that your Pull Request works/fixes as described and planned (include screenshots, if appropriate).

## As part of this Pull Request I have

- [ ] Checked for duplicate [Pull Requests](https://github.com/Azure/alz-terraform-accelerator/pulls)
- [ ] Associated it with relevant [issues](https://github.com/Azure/alz-terraform-accelerator/issues), for tracking and closure.
- [ ] Ensured my code/branch is up-to-date with the latest changes in the `main` [branch](https://github.com/Azure/alz-terraform-accelerator/tree/main)
- [ ] Performed testing and provided evidence.
- [ ] Updated relevant and associated documentation.
12 changes: 12 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates

version: 2
updates:
- package-ecosystem: "terraform" # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "weekly"
36 changes: 36 additions & 0 deletions .github/linters/.markdown-lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
###########################
###########################
## Markdown Linter rules ##
###########################
###########################

# Linter rules doc:
# - https://github.com/DavidAnson/markdownlint
#
# Note:
# To comment out a single error:
# <!-- markdownlint-disable -->
# any violations you want
# <!-- markdownlint-restore -->
#

###############
# Rules by id #
###############
MD004: false # ul-style - Unordered list style
MD007:
indent: 2 # ul-indent - Unordered list indentation
MD013:
line_length: 400 # line-length - Line length
MD026:
punctuation: ".,;:!。,;:" # no-trailing-punctuation - Trailing punctuation in heading
MD029: false # ol-prefix - Ordered list item prefix
MD033: false # no-inline-html - Inline HTML
MD036: false # no-emphasis-as-heading/no-emphasis-as-header - Emphasis used instead of a heading
MD041: false # first-line-heading/first-line-h1 - First line in a file should be a top-level heading

#################
# Rules by tags #
#################
blank_lines: false # MD012, MD022, MD031, MD032, MD047
11 changes: 11 additions & 0 deletions .github/linters/.yaml-lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
extends: default

rules:
# 200 chars should be enough, but don't fail if a line is longer
line-length:
max: 200
level: warning
truthy:
check-keys: false
level: warning
34 changes: 34 additions & 0 deletions .github/tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# End to End Tests

## Overview

The end to end tests can be found in the ./github/workflows/end-to-end-test.yml action.

- The tests are triggered by `workflow_dispatch` or on `pull_request` only when the label `PR: Safe to test 🧪` has been added to the PR.
- The tests run against the environment `CSUTF`, which requires a manual approval to deploy.
- The tests run as a matrix, targeting different OS, VCS, Terraform and Auth Methods.

## Test Process

The test follow this process:

1. Check out the module from the PR merge branch.
1. Generate an `inputs.json` file that is used to override the prompts in the `ALZ` PowerShell module.
1. Install the `ALZ` PowerShell module.
1. Get the latest version tag for the live accelerator module.
1. Copy the `boostrap` and `template` folders into a folder named by the latest version tag.
1. Run the `New-New-ALZEnvironment` function to deploy the terraform.
1. Run a `terraform destroy` to clean up the environment.

## Environment

The tests use a set of environemnts to managed by the ALZ team. These are:

- Azure:
- Tenant: CSU TF
- Subscription: csu-tf-devops
- User Assigned Managed Identity: alz-terraform-accelerator-cd-tests-identity (this has federated credentials)
- Azure DevOps
- Organisation: microsoft-azure-landing-zones-cd-tests
- GiHub
- Organisation: microsoft-azure-landingzones-cd-tests
15 changes: 15 additions & 0 deletions .github/tests/cleanup-scripts/cleanup_github-repositories.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# This file can be used to clean up GitHub repositories if there has been an issue with the End to End tests.
# CAUTION: Make sure you are connected to the correct organization before running this script!
$repos = gh repo list microsoft-azure-landing-zones-cd-tests --json name,owner | ConvertFrom-Json

$repos | ForEach-Object -Parallel {
$match = "*229*"
$repoName = "$($_.owner.login)/$($_.name)"

if($repoName -like $match)
{
Write-Host "Deleting repo: $repoName"
gh repo delete $repoName --yes

}
} -ThrottleLimit 10
9 changes: 9 additions & 0 deletions .github/tests/cleanup-scripts/cleanup_resouce_groups.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# This file can be used to clean up Resource Groups if there has been an issue with the End to End tests.
# CAUTION: Make sure you are connected to the correct subscription before running this script!
az account show
$resourceGroups = az group list --query "[?contains(name, '254-')]" | ConvertFrom-Json

$resourceGroups | ForEach-Object -Parallel {
Write-Host "Deleting resource group: $($_.name)"
az group delete --name $_.name --yes
} -ThrottleLimit 10
13 changes: 13 additions & 0 deletions .github/tests/scripts/Install-AcceleratorModule.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
param (
[string]$ModuleUrl = "https://github.com/Azure/ALZ-PowerShell-Module",
[string]$ModuleBranch = "main"
)

$targetDirectory = "./accelerator-powershell-module"

if(!(Test-Path $targetDirectory)) {
git clone -b $ModuleBranch $ModuleUrl $targetDirectory
}

./accelerator-powershell-module/actions_bootstrap_for_e2e_tests.ps1 | Out-String | Write-Verbose
Invoke-Build -File ./accelerator-powershell-module/src/ALZ.build.ps1 BuildAndInstallOnly | Out-String | Write-Verbose
Loading
Loading