Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: update CI/CD process to use composite actions and auto-release workflow #361

Merged
merged 11 commits into from
Nov 7, 2021
Merged

fix: update CI/CD process to use composite actions and auto-release workflow #361

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
05ef0a3
fix: update CI/CD process to use composite actions and auto-release w…
bryantbiggs Oct 26, 2021
f86d659
chore: need to have both current and future branch names
bryantbiggs Oct 26, 2021
9afa01b
chore: add a change to ensure release workflow is run
bryantbiggs Oct 26, 2021
304e5df
chore: remove comment added to trigger worfklow
bryantbiggs Oct 26, 2021
df10f44
Update .pre-commit-config.yaml
bryantbiggs Oct 26, 2021
48c770a
chore: update to remove redundant hook and set semantic-release version
bryantbiggs Oct 26, 2021
6dbc2fe
chore: update release workflow to use PAT
bryantbiggs Oct 26, 2021
588f70e
chore: trigger release workflow on any terraform or python file changes
bryantbiggs Oct 27, 2021
7c362ad
chore: bump pre-commit to latest
bryantbiggs Oct 27, 2021
4acaf8f
chore: update changelog header
bryantbiggs Oct 30, 2021
94abe7f
Update .pre-commit-config.yaml
antonbabenko Nov 7, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 0 additions & 111 deletions .chglog/CHANGELOG.tpl.md
View file
Open in desktop

This file was deleted.

10 changes: 0 additions & 10 deletions .chglog/config.yml
View file
Open in desktop

This file was deleted.

103 changes: 41 additions & 62 deletions .github/workflows/pre-commit.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,98 +2,77 @@ name: Pre-Commit

on:
pull_request:
push:
branches:
- main
- master

env:
TERRAFORM_DOCS_VERSION: v0.16.0

jobs:
# Min Terraform version(s)
getDirectories:
name: Get root directories
collectInputs:
name: Collect workflow inputs
runs-on: ubuntu-latest
outputs:
directories: ${{ steps.dirs.outputs.directories }}
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Install Python
uses: actions/setup-python@v2
- name: Build matrix
id: matrix
run: |
DIRS=$(python -c "import json; import glob; print(json.dumps([x.replace('/versions.tf', '') for x in glob.glob('./**/versions.tf', recursive=True)]))")
echo "::set-output name=directories::$DIRS"
outputs:
directories: ${{ steps.matrix.outputs.directories }}

- name: Get root directories
id: dirs
uses: clowdhaus/terraform-composite-actions/directories@v1.3.0

preCommitMinVersions:
name: Min TF validate
needs: getDirectories
name: Min TF pre-commit
needs: collectInputs
runs-on: ubuntu-latest
strategy:
matrix:
directory: ${{ fromJson(needs.getDirectories.outputs.directories) }}
directory: ${{ fromJson(needs.collectInputs.outputs.directories) }}
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Install Python
uses: actions/setup-python@v2

- name: Terraform min/max versions
id: minMax
uses: clowdhaus/terraform-min-max@v1.0.2
uses: clowdhaus/terraform-min-max@v1.0.3
with:
directory: ${{ matrix.directory }}
- name: Install Terraform v${{ steps.minMax.outputs.minVersion }}
uses: hashicorp/setup-terraform@v1
with:
terraform_version: ${{ steps.minMax.outputs.minVersion }}
- name: Install pre-commit dependencies
run: pip install pre-commit
- name: Execute pre-commit

- name: Pre-commit Terraform ${{ steps.minMax.outputs.minVersion }}
# Run only validate pre-commit check on min version supported
if: ${{ matrix.directory != '.' }}
run: pre-commit run terraform_validate --color=always --show-diff-on-failure --files ${{ matrix.directory }}/*
- name: Execute pre-commit
uses: clowdhaus/terraform-composite-actions/pre-commit@v1.3.0
with:
terraform-version: ${{ steps.minMax.outputs.minVersion }}
args: 'terraform_validate --color=always --show-diff-on-failure --files ${{ matrix.directory }}/*'

- name: Pre-commit Terraform ${{ steps.minMax.outputs.minVersion }}
# Run only validate pre-commit check on min version supported
if: ${{ matrix.directory == '.' }}
run: pre-commit run terraform_validate --color=always --show-diff-on-failure --files $(ls *.tf)
uses: clowdhaus/terraform-composite-actions/pre-commit@v1.3.0
with:
terraform-version: ${{ steps.minMax.outputs.minVersion }}
args: 'terraform_validate --color=always --show-diff-on-failure --files $(ls *.tf)'

# Max Terraform version
getBaseVersion:
name: Module max TF version
preCommitMaxVersion:
name: Max TF pre-commit
runs-on: ubuntu-latest
needs: collectInputs
steps:
- name: Checkout
uses: actions/checkout@v2
with:
ref: ${{ github.event.pull_request.head.ref }}
repository: ${{github.event.pull_request.head.repo.full_name}}

- name: Terraform min/max versions
id: minMax
uses: clowdhaus/terraform-min-max@v1.0.2
outputs:
minVersion: ${{ steps.minMax.outputs.minVersion }}
maxVersion: ${{ steps.minMax.outputs.maxVersion }}
uses: clowdhaus/terraform-min-max@v1.0.3

preCommitMaxVersion:
name: Max TF pre-commit
runs-on: ubuntu-latest
needs: getBaseVersion
strategy:
fail-fast: false
matrix:
version:
- ${{ needs.getBaseVersion.outputs.maxVersion }}
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Install Python
uses: actions/setup-python@v2
- name: Install Terraform v${{ matrix.version }}
uses: hashicorp/setup-terraform@v1
- name: Pre-commit Terraform ${{ steps.minMax.outputs.maxVersion }}
uses: clowdhaus/terraform-composite-actions/pre-commit@v1.3.0
with:
terraform_version: ${{ matrix.version }}
- name: Install pre-commit dependencies
run: |
pip install pre-commit
curl -Lo ./terraform-docs.tar.gz https://github.com/terraform-docs/terraform-docs/releases/download/v0.13.0/terraform-docs-v0.13.0-$(uname)-amd64.tar.gz && tar -xzf terraform-docs.tar.gz && chmod +x terraform-docs && sudo mv terraform-docs /usr/bin/
curl -L "$(curl -s https://api.github.com/repos/terraform-linters/tflint/releases/latest | grep -o -E "https://.+?_linux_amd64.zip")" > tflint.zip && unzip tflint.zip && rm tflint.zip && sudo mv tflint /usr/bin/
- name: Execute pre-commit
# Run all pre-commit checks on max version supported
if: ${{ matrix.version == needs.getBaseVersion.outputs.maxVersion }}
run: pre-commit run --color=always --show-diff-on-failure --all-files
terraform-version: ${{ steps.minMax.outputs.maxVersion }}
terraform-docs-version: ${{ env.TERRAFORM_DOCS_VERSION }}
29 changes: 29 additions & 0 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Release

on:
push:
branches:
- main
- master
paths:
- '*.tf'
- 'modules/**/*.tf'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few questions:

  1. Does it mean that any change in these masks will trigger a release?
  2. Can modules/**/*.tf be generalize to be **/*.tf to include changes in examples/*, too?
  3. Is it right, that this GH Action will decide on the scope based on the commit message type (feat => major, fix: => minor, chore => patch). If so, do you know where the mapping is described? I can't find it on the GH Action homepage. Found answer on https://github.com/semantic-release/commit-analyzer
  4. Is there any stop word to have in git commit message which will not trigger the release even if valid paths were changed? (e.g. [ci skip])

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Correct - any PRs that match these path patterns will trigger a release and we can tailor the paths/pattern to whatever we choose
  2. Yes - I believe the pattern **/*.tf means any *.tf file anywhere in the repo. However, if someone makes only a change to an example, I don't think that would be something "release worthy" since there are no source changes
  3. 👍🏽
  4. Yes - there are a few patterns which do not trigger releases. I generally use commit messages like chore: <something> and this pattern does not trigger a release (skips it)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. If anyone changes tf files in examples and the git commit message starts with fix:, we should still release a patch version. To prevent this we can set the message to start with chore:.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated in 588f70e and added '**/*.py' since that will come up in a few repos and this keeps the file uniform across the repos

also, I tried to update to pre-commit 1.54 but it seems to be broken on terraform-docs now

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the report on broken terraform_docs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Please use v1.55.0.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, updated to v1.55.0 🎉


jobs:
release:
name: Release
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0

- name: Release
uses: cycjimmy/semantic-release-action@v2
with:
bryantbiggs marked this conversation as resolved.
Show resolved Hide resolved
extra_plugins: |
@semantic-release/changelog@6.0.0
@semantic-release/git@10.0.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

persist-credentials: false should be used somewhere in the config, too, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we use a personal access token then yes, we don't want to persist creds because they will conflict with the default token

env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another "strange questions" from me :)

  1. Will this token be available for others to read? (eg. by adding another GH action YAML config and outputting ${{ secrets.GITHUB_TOKEN }} there?)
  2. What is the minimum scope of the token should we have? Reading this IMPORTANT note and I am not sure I understand if Personal Access Token should be used (we have master as a protected branch).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. This token is already there by default https://docs.github.com/en/actions/security-guides/automatic-token-authentication
  2. Let me get back to you on that, I'll have to create a token and do a bunch of test runs to see what permissions are required

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok so it looks like we only need a token with public_repo scope for this to work
image

I would store this as an org secret under the name SEMANTIC_RELEASE_TOKEN. You can elect to select repos one by one as we add the workflow or allow all repos - up to you (will come up when you create the org secret as an option)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I would make sure that the actions are locked down as well on the org. Here is what I have on my org - you can check by going to settings -> actions.

image

6 changes: 5 additions & 1 deletion .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
repos:
- repo: git://github.com/antonbabenko/pre-commit-terraform
bryantbiggs marked this conversation as resolved.
Show resolved Hide resolved
rev: v1.50.0
rev: v1.53.0
bryantbiggs marked this conversation as resolved.
Show resolved Hide resolved
hooks:
- id: terraform_fmt
- id: terraform_validate
- id: terraform_docs
args:
- '--args=--lockfile=false'
- id: terraform_tflint
args:
- '--args=--only=terraform_deprecated_interpolation'
Expand All @@ -24,3 +26,5 @@ repos:
rev: v4.0.1
hooks:
- id: check-merge-conflict
- id: trailing-whitespace
bryantbiggs marked this conversation as resolved.
Show resolved Hide resolved
- id: end-of-file-fixer
28 changes: 28 additions & 0 deletions .releaserc.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"branches": [
"main",
"master"
],
"ci": false,
"plugins": [
"@semantic-release/commit-analyzer",
"@semantic-release/release-notes-generator",
"@semantic-release/github",
[
"@semantic-release/changelog",
{
"changelogFile": "CHANGELOG.md",
"changelogTitle": "# Changelog\n\nAll notable changes to this project will be documented in this file. See\n[Conventional Commits](https://conventionalcommits.org) for commit guidelines."
bryantbiggs marked this conversation as resolved.
Show resolved Hide resolved
}
],
[
"@semantic-release/git",
{
"assets": [
"CHANGELOG.md"
],
"message": "chore(release): version ${nextRelease.version} [skip ci]\n\n${nextRelease.notes}"
}
]
]
}
9 changes: 0 additions & 9 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,3 @@
# Change Log
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are removed since they will be managed by the semantic-release plugin going forward. They will get added back in when the first release kicks off, just like it did here following this same process https://github.com/clowdhaus/terraform-aws-rds/blob/master/CHANGELOG.md#changelog


All notable changes to this project will be documented in this file.

<a name="unreleased"></a>
## [Unreleased]



<a name="v3.4.0"></a>
## [v3.4.0] - 2021-09-07

Expand Down
7 changes: 0 additions & 7 deletions Makefile
View file
Open in desktop

This file was deleted.

8 changes: 4 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -220,10 +220,10 @@ Users have the ability to:

| Name | Source | Version |
|------|--------|---------|
| <a name="module_db_instance"></a> [db\_instance](#module\_db\_instance) | ./modules/db_instance | |
| <a name="module_db_option_group"></a> [db\_option\_group](#module\_db\_option\_group) | ./modules/db_option_group | |
| <a name="module_db_parameter_group"></a> [db\_parameter\_group](#module\_db\_parameter\_group) | ./modules/db_parameter_group | |
| <a name="module_db_subnet_group"></a> [db\_subnet\_group](#module\_db\_subnet\_group) | ./modules/db_subnet_group | |
| <a name="module_db_instance"></a> [db\_instance](#module\_db\_instance) | ./modules/db_instance | n/a |
| <a name="module_db_option_group"></a> [db\_option\_group](#module\_db\_option\_group) | ./modules/db_option_group | n/a |
| <a name="module_db_parameter_group"></a> [db\_parameter\_group](#module\_db\_parameter\_group) | ./modules/db_parameter_group | n/a |
| <a name="module_db_subnet_group"></a> [db\_subnet\_group](#module\_db\_subnet\_group) | ./modules/db_subnet_group | n/a |

## Resources

Expand Down
4 changes: 2 additions & 2 deletions examples/complete-mssql/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,8 @@ Note that this example may create resources which cost money. Run `terraform des

| Name | Source | Version |
|------|--------|---------|
| <a name="module_db"></a> [db](#module\_db) | ../../ | |
| <a name="module_db_disabled"></a> [db\_disabled](#module\_db\_disabled) | ../../ | |
| <a name="module_db"></a> [db](#module\_db) | ../../ | n/a |
| <a name="module_db_disabled"></a> [db\_disabled](#module\_db\_disabled) | ../../ | n/a |
| <a name="module_security_group"></a> [security\_group](#module\_security\_group) | terraform-aws-modules/security-group/aws | ~> 4 |
| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 2 |

Expand Down
Loading