Feat: added support for distinct variable files per region per accoun…

…t in Terraform pipelines (#662)

* Feature: added support for distinct variable files per region per account in Terraform pipelines

* Fixed Markdown for linting

* Spell typo

* Update src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/helpers/terraform/adf_terraform.sh

Co-authored-by: Simon Kok

docs/user-guide.md

@@ -1164,7 +1164,7 @@ pipelines:
 4. Add variable definition to `tf/variables.tf` file and variable values to
    `tfvars/global.auto.tfvars`.
 
-   - Local variables (per account) can be configured using the following
+   - Local variables (per account and per region) can be configured using the following
      naming convention
 
      ```txt
@@ -1176,8 +1176,11 @@ pipelines:
      │
      └───111111111111 <-- this folders contains variable files related to
      │   │                the account
-     │   └──────│   local.auto.tfvars <-- this file contains variables related
-     │          │                         to the account
+     │   └──────│   local.auto.tfvars <-- this file contains variables
+     │          │                         related to the account
+     │          └───eu-west-1
+     │              └────── region.auto.tfvars <-- this file contains
+     │                      variables related to the account and the region
      │
      └───222222222222
          └──────│   local.auto.tfvars
@@ -1190,6 +1193,21 @@ pipelines:
 7. Pipeline contains a manual approval step between Terraform plan and
    Terraform apply. Confirm to proceed.
 
+**Note**:
+The pipeline leverages the terraform behavior on reading *.local.tfvars files,
+so the latter file in lexicographical order overrides variables already defined
+in preceding files.
+
+Given the structure above, terraform can possibly find 3 files:
+
+- `global.auto.tfvars`
+- `local.auto.tfvars`
+- `region.auto.tfvars`
+
+and it means that `region.auto.tfvars` can override variables passed in
+`local.auto.tfvars`, that in turn can override variables passed in
+`global.auto.tfvars`.
+
 Terraform state files are stored in the regional S3 buckets in the deployment
 account. One state file per account/region/module is created.
 

src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/helpers/terraform/adf_terraform.sh

@@ -15,6 +15,9 @@ tfinit(){
     if [ -d "${CURRENT}/tfvars/${TF_VAR_TARGET_ACCOUNT_ID}" ]; then
         cp -R "${CURRENT}/tfvars/${TF_VAR_TARGET_ACCOUNT_ID}/." "${CURRENT}/tmp/${TF_VAR_TARGET_ACCOUNT_ID}-${AWS_REGION}"
     fi
+    if [ -d "${CURRENT}/tfvars/${TF_VAR_TARGET_ACCOUNT_ID}/${AWS_REGION}" ]; then
+        cp -R "${CURRENT}/tfvars/${TF_VAR_TARGET_ACCOUNT_ID}/${AWS_REGION}"/. "${CURRENT}/tmp/${TF_VAR_TARGET_ACCOUNT_ID}-${AWS_REGION}"
+    fi
     if [ -f "${CURRENT}/tfvars/global.auto.tfvars" ]; then
         cp -R "${CURRENT}/tfvars/global.auto.tfvars" "${CURRENT}/tmp/${TF_VAR_TARGET_ACCOUNT_ID}-${AWS_REGION}"
     fi