example install run 20220915 - to validate terraform 1.2.8 project-factory removal of impersonate_service_account #47

obriensystems · 2022-09-15T23:22:09Z

admin at guardrails.gcp.zone clean identity org from scratch - full onboarding prep

note the "trust" checkbox - critical - #41

<img width="1767" alt="Screen Shot 2022-09-15 at 19 08 43" src="https://user-images.githubusercontent.com/24765473/190526445-9e2aa520-a894-4223-a5df-685be

9fa0f25.png">

Notice that there is a pending jira I forgot about with the checkout in cloud shell on my branch

Welcome to Cloud Shell! Type "help" to get started.
To set your Cloud Platform project in this session use “gcloud config set project [PROJECT_ID]”
admin_@cloudshell:~$ cloudshell_open --repo_url "https://github.com/fmichaelobrien/accelerators_accelerateurs-gcp" --page "editor" --tutorial "README.md" --force_new_clone
2022/09/15 23:06:10 Cloning https://github.com/fmichaelobrien/accelerators_accelerateurs-gcp into /home/admin_/cloudshell_open/accelerators_accelerateurs-gcp
Cloning into '/home/admin_/cloudshell_open/accelerators_accelerateurs-gcp'...
remote: Enumerating objects: 2698, done.
remote: Counting objects: 100% (368/368), done.
remote: Compressing objects: 100% (142/142), done.
remote: Total 2698 (delta 148), reused 332 (delta 145), pack-reused 2330
Receiving objects: 100% (2698/2698), 1.09 MiB | 9.18 MiB/s, done.
Resolving deltas: 100% (1687/1687), done.
admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp$ gcloud config set project gr-bootstrap-ggz
Updated property [core/project].

admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp (gr-bootstrap-ggz)$ cd deployment-templates/Terraform/guardrails/
admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails (gr-bootstrap-ggz)$ cd 0-bootstrap/
admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/0-bootstrap (gr-bootstrap-ggz)$ ./bootstrap.sh -d ggz -p gr-bootstrap-gg

before

as expected - we will add missing roles - these will be automated tomorrow+ in #42

admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/0-bootstrap (gr-bootstrap-ggz)$ ./bootstrap.sh -d ggz -p gr-bootstrap-ggz
seed project id: ggz-seed-project
boostrap project id: gr-bootstrap-ggz
org id: 743091813895
billing id: 016706-67373C-2417D0
Updated property [core/project].
checking roles of current account: admin@guardrails.gcp.zone
iam.serviceAccountTokenCreator
roles/iam.serviceAccountTokenCreator role missing
admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/0-bootstrap (gr-bootstrap-ggz)$

create service account token creator at the org level

admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/0-bootstrap (gr-bootstrap-ggz)$ ./bootstrap.sh -d ggz -p gr-bootstrap-ggz
seed project id: ggz-seed-project
boostrap project id: gr-bootstrap-ggz
org id: 743091813895
billing id: 016706-67373C-2417D0
Updated property [core/project].
checking roles of current account: admin@guardrails.gcp.zone
iam.serviceAccountTokenCreator
ROLE: roles/iam.serviceAccountTokenCreator role set OK on super admin account
roles/resourcemanager.folderAdmin
ROLE: roles/resourcemanager.folderAdmin role set OK on super admin account
roles/resourcemanager.organizationAdmin
ROLE: roles/resourcemanager.organizationAdmin role set OK on super admin account
orgpolicy.policyAdmin
roles/orgpolicy.policyAdmin role missing

add org policy admin

add project creator role

admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/0-bootstrap (gr-bootstrap-ggz)$ ./bootstrap.sh -d ggz -p gr-bootstrap-ggz
seed project id: ggz-seed-project
boostrap project id: gr-bootstrap-ggz
org id: 743091813895
billing id: 016706-67373C-2417D0
Updated property [core/project].
checking roles of current account: admin@guardrails.gcp.zone
iam.serviceAccountTokenCreator
ROLE: roles/iam.serviceAccountTokenCreator role set OK on super admin account
roles/resourcemanager.folderAdmin
ROLE: roles/resourcemanager.folderAdmin role set OK on super admin account
roles/resourcemanager.organizationAdmin
ROLE: roles/resourcemanager.organizationAdmin role set OK on super admin account
orgpolicy.policyAdmin
ROLE: roles/orgpolicy.policyAdmin role set OK on super admin account
resourcemanager.projectCreator
roles/resourcemanager.projectCreator role missing

add billing project manager role

admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/0-bootstrap (gr-bootstrap-ggz)$ ./bootstrap.sh -d ggz -p gr-bootstrap-ggz
seed project id: ggz-seed-project
boostrap project id: gr-bootstrap-ggz
org id: 743091813895
billing id: 016706-67373C-2417D0
Updated property [core/project].
checking roles of current account: admin@guardrails.gcp.zone
iam.serviceAccountTokenCreator
ROLE: roles/iam.serviceAccountTokenCreator role set OK on super admin account
roles/resourcemanager.folderAdmin
ROLE: roles/resourcemanager.folderAdmin role set OK on super admin account
roles/resourcemanager.organizationAdmin
ROLE: roles/resourcemanager.organizationAdmin role set OK on super admin account
orgpolicy.policyAdmin
ROLE: roles/orgpolicy.policyAdmin role set OK on super admin account
resourcemanager.projectCreator
ROLE: roles/resourcemanager.projectCreator role set OK on super admin account
billing.projectManager
roles/billing.projectManager role missing

full bootstrap


admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/0-bootstrap (gr-bootstrap-ggz)$ ./bootstrap.sh -d ggz -p gr-bootstrap-ggz
seed project id: ggz-seed-project
boostrap project id: gr-bootstrap-ggz
org id: 743091813895
billing id: 016706-67373C-2417D0
Updated property [core/project].
checking roles of current account: admin@guardrails.gcp.zone
iam.serviceAccountTokenCreator
ROLE: roles/iam.serviceAccountTokenCreator role set OK on super admin account
roles/resourcemanager.folderAdmin
ROLE: roles/resourcemanager.folderAdmin role set OK on super admin account
roles/resourcemanager.organizationAdmin
ROLE: roles/resourcemanager.organizationAdmin role set OK on super admin account
orgpolicy.policyAdmin
ROLE: roles/orgpolicy.policyAdmin role set OK on super admin account
resourcemanager.projectCreator
ROLE: roles/resourcemanager.projectCreator role set OK on super admin account
billing.projectManager
ROLE: roles/billing.projectManager role set OK on super admin account
all roles set OK on super admin account:  admin@guardrails.gcp.zone - proceeding
enabling pubsub.googleapis.com identitytoolkit cloudresourcemanager iam cloudbilling on gr-bootstrap-ggz project
Operation "operations/acf.p2-502392433631-0b13dfb5-fcb9-4d89-974d-99bdd21b7354" finished successfully.
Listed 0 items.
Create in progress for [https://cloudresourcemanager.googleapis.com/v1/projects/ggz-seed-project].
Waiting for [operations/cp.7485999651336448021] to finish...done.    
Enabling service [cloudapis.googleapis.com] on project [ggz-seed-project]...
Operation "operations/acat.p2-350624459886-83c7c08d-b991-4032-8a68-7477a3a55ed3" finished successfully.
billingAccountName: billingAccounts/016706-67373C-2417D0
billingEnabled: true
name: projects/ggz-seed-project/billingInfo
projectId: ggz-seed-project
Listed 0 items.
Created service account [tfadmin-ggz].
tfadmin-ggz@ggz-seed-project.iam.gserviceaccount.com
Updated IAM policy for organization [743091813895].
bindings:
- members:
  - serviceAccount:tfadmin-ggz@ggz-seed-project.iam.gserviceaccount.com
  role: roles/billing.admin
- members:
  - domain:guardrails.gcp.zone
  role: roles/billing.creator
- members:
  - user:admin@guardrails.gcp.zone
  role: roles/billing.projectManager
- members:
  - user:admin@guardrails.gcp.zone
  role: roles/iam.serviceAccountTokenCreator
- members:
  - user:admin@guardrails.gcp.zone
  role: roles/orgpolicy.policyAdmin
- members:
  - user:admin@guardrails.gcp.zone
  role: roles/owner
- members:
  - user:admin@guardrails.gcp.zone
  role: roles/resourcemanager.folderAdmin
- members:
  - user:admin@guardrails.gcp.zone
  role: roles/resourcemanager.organizationAdmin
- members:
  - domain:guardrails.gcp.zone
  - user:admin@guardrails.gcp.zone
  role: roles/resourcemanager.projectCreator
  
  
  - members:
  - serviceAccount:tfadmin-ggz@ggz-seed-project.iam.gserviceaccount.com
  role: roles/storage.admin
etag: BwXowAXavIQ=
version: 1
gs://ggz-seed-project-guardrails
Creating gs://ggz-seed-project-guardrails/...
Replace backend.tf bucketname
Your active configuration is: [cloudshell-22665]
Updated property [core/project].
wrote TF SA to provider.tf and variables.tfvar along with the bucket, billing account and org id - verify them
enabling pubsub identitytoolkit cloudresourcemanager iam cloudbilling on ggz-seed-project project
Operation "operations/acf.p2-350624459886-b48f9e33-bcad-4ec1-953e-fe213a2d356b" finished successfully.
NAME: cloudresourcemanager.googleapis.com
NAME: identitytoolkit.googleapis.com
NAME: pubsub.googleapis.com
NAME: cloudbilling.googleapis.com
NAME: iam.googleapis.com
if you get an iam permission on the guardrails-aaaa project - run gcloud services enable iam.googleapis.com --project guardrails-nnnn
Status: 0
GCP seed project created project id: ggz-seed-project \n
 Terraform Service account to be used for creating GCP landing zone =  tfadmin-ggz@ggz-seed-project.iam.gserviceaccount.com \n
 Terraform Backend Storage Bucket: gs://ggz-seed-project-guardrails

check the 0 items returns and change the output from the iam role additions to not print out

after

admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/0-bootstrap (ggz-seed-project)$ gcloud services list --enabled --project  ggz-seed-project | grep NAME
NAME: bigquery.googleapis.com
NAME: bigquerymigration.googleapis.com
NAME: bigquerystorage.googleapis.com
NAME: cloudapis.googleapis.com
NAME: cloudbilling.googleapis.com
NAME: clouddebugger.googleapis.com
NAME: cloudresourcemanager.googleapis.com
NAME: cloudtrace.googleapis.com
NAME: datastore.googleapis.com
NAME: iam.googleapis.com
NAME: iamcredentials.googleapis.com
NAME: identitytoolkit.googleapis.com
NAME: logging.googleapis.com
NAME: monitoring.googleapis.com
NAME: pubsub.googleapis.com
NAME: servicemanagement.googleapis.com
NAME: serviceusage.googleapis.com
NAME: sql-component.googleapis.com
NAME: storage-api.googleapis.com
NAME: storage-component.googleapis.com
NAME: storage.googleapis.com

diff

admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/0-bootstrap (ggz-seed-project)$ git diff
diff --git a/deployment-templates/Terraform/guardrails/1-guardrails/backend.tf b/deployment-templates/Terraform/guardrails/1-guardrails/backend.tf
index 3ec11a9..edbeb2e 100644
--- a/deployment-templates/Terraform/guardrails/1-guardrails/backend.tf
+++ b/deployment-templates/Terraform/guardrails/1-guardrails/backend.tf
@@ -1,6 +1,6 @@
 terraform {
   backend "gcs" {
-    bucket = "BUCKETNAME"
+    bucket = "ggz-seed-project-guardrails"
     prefix = "/orgadmin/seeding/"
   }
 }
\ No newline at end of file
diff --git a/deployment-templates/Terraform/guardrails/1-guardrails/provider.tf b/deployment-templates/Terraform/guardrails/1-guardrails/provider.tf
index 155a52f..c21f9ca 100644
--- a/deployment-templates/Terraform/guardrails/1-guardrails/provider.tf
+++ b/deployment-templates/Terraform/guardrails/1-guardrails/provider.tf
@@ -36,5 +36,5 @@ data "google_service_account_access_token" "default" {
   lifetime               = "3600s"
 }

-# written from bootstrap.sh via YOUR_SERVICE_ACCOUNT in form SERVICE_ACCOUNT@PROJECT.iam.gserviceaccount.com
-locals { terraform_service_account = "YOUR_SERVICE_ACCOUNT" }
\ No newline at end of file
+# written from bootstrap.sh via tfadmin-ggz@ggz-seed-project.iam.gserviceaccount.com in form SERVICE_ACCOUNT@PROJECT.iam.gserviceaccount.com
+locals { terraform_service_account = "tfadmin-ggz@ggz-seed-project.iam.gserviceaccount.com" }
\ No newline at end of file

variables.tfvar

audit_data_users="audit_data_users@guardrails.gcp.zone"
ssc_broker_users="ssc_broker_users@guardrails.gcp.zone"
org_id="743091813895"
terraform_service_account="tfadmin-ggz@ggz-seed-project.iam.gserviceaccount.com"
billing_account="016706-67373C-2417D0"
billing_data_users="billing_data_users@guardrails.gcp.zone"
audit_logs_table_delete_contents_on_destroy=true
log_export_storage_force_destroy=true
allowed_regions=["northamerica-northeast1", "northamerica-northeast2"]
bucket_name="ggz-guardrails-assets"

admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/0-bootstrap (ggz-seed-project)$ cd ../1-guardrails/
admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/1-guardrails (ggz-seed-project)$ terraform version
Terraform v1.2.8
on linux_amd64

Your version of Terraform is out of date! The latest version
is 1.2.9. You can update by downloading from https://www.terraform.io/downloads.html
admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/1-guardrails (ggz-seed-project)$ terraform init

terraform init - initializes provider plugins - verify this

admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/1-guardrails (ggz-seed-project)$ terraform init
Initializing modules...
Downloading registry.terraform.io/terraform-google-modules/project-factory/google 10.4.0 for administration...
- administration in .terraform/modules/administration
- administration.budget in .terraform/modules/administration/modules/budget
- administration.gsuite_group in .terraform/modules/administration/modules/gsuite_group
- administration.project-factory in .terraform/modules/administration/modules/core_project_factory
- administration.project-factory.project_services in .terraform/modules/administration/modules/project_services
- administration.quotas in .terraform/modules/administration/modules/quota_manager
- administration.shared_vpc_access in .terraform/modules/administration/modules/shared_vpc_access
Downloading registry.terraform.io/terraform-google-modules/log-export/google 7.4.2 for bigquery_destination...
- bigquery_destination in .terraform/modules/bigquery_destination/modules/bigquery
Downloading registry.terraform.io/terraform-google-modules/log-export/google 7.4.2 for log_export_to_biqquery...
- log_export_to_biqquery in .terraform/modules/log_export_to_biqquery
Downloading registry.terraform.io/terraform-google-modules/log-export/google 7.4.2 for log_export_to_pubsub...
- log_export_to_pubsub in .terraform/modules/log_export_to_pubsub
Downloading registry.terraform.io/terraform-google-modules/log-export/google 7.4.2 for log_export_to_storage...
- log_export_to_storage in .terraform/modules/log_export_to_storage
Downloading registry.terraform.io/terraform-google-modules/org-policy/google 3.0.2 for org-policy...
- org-policy in .terraform/modules/org-policy
Downloading registry.terraform.io/terraform-google-modules/log-export/google 7.4.2 for pubsub_destination...
- pubsub_destination in .terraform/modules/pubsub_destination/modules/pubsub
Downloading registry.terraform.io/terraform-google-modules/log-export/google 7.4.2 for storage_destination...
- storage_destination in .terraform/modules/storage_destination/modules/storage

Initializing the backend...

Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Finding hashicorp/google versions matching ">= 2.5.0, >= 3.43.0, >= 3.50.0, >= 3.53.0, < 4.0.0, < 5.0.0"...
- Finding hashicorp/random versions matching ">= 2.2.0"...
- Finding hashicorp/google-beta versions matching ">= 3.1.0, >= 3.43.0, >= 3.50.0, < 4.0.0"...
- Finding hashicorp/null versions matching ">= 2.1.0"...
- Installing hashicorp/google v3.90.1...
- Installed hashicorp/google v3.90.1 (signed by HashiCorp)
- Installing hashicorp/random v3.4.3...
- Installed hashicorp/random v3.4.3 (signed by HashiCorp)
- Installing hashicorp/google-beta v3.90.1...
- Installed hashicorp/google-beta v3.90.1 (signed by HashiCorp)
- Installing hashicorp/null v3.1.1...
- Installed hashicorp/null v3.1.1 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/1-guardrails (ggz-seed-project)$

apply

admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/1-guardrails (ggz-seed-project)$ terraform apply -var-file variables.tfvar

reproduced

admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/1-guardrails (ggz-seed-project)$ terraform apply -var-file variables.tfvar
╷
│ Error: Unsupported argument
│
│   on projects.tf line 5, in module "administration":
│    5:   impersonate_service_account = var.terraform_service_account
│
│ An argument named "impersonate_service_account" is not expected here.
╵

above worked in last admin-root@cloud-nuage.info in mid June 2022
Since then there may have been terraform changes to module, less likely code changes - verifying both before tomorrows's 2 guardrails installs

admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/1-guardrails (ggz-seed-project)$ terraform apply -var-file variables.tfvar
╷
│ Error: Unsupported argument
│
│   on projects.tf line 5, in module "administration":
│    5:   impersonate_service_account = var.terraform_service_account
│
│ An argument named "impersonate_service_account" is not expected here.

triage

https://registry.terraform.io/modules/terraform-google-modules/project-factory/google/latest

in projects.tf
  #impersonate_service_account = var.terraform_service_account

admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/1-guardrails (ggz-seed-project)$ terraform init
Initializing modules...

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/google-beta from the dependency lock file
- Reusing previous version of hashicorp/null from the dependency lock file
- Reusing previous version of hashicorp/google from the dependency lock file
- Using previously-installed hashicorp/random v3.4.3
- Using previously-installed hashicorp/google-beta v3.90.1
- Using previously-installed hashicorp/null v3.1.1
- Using previously-installed hashicorp/google v3.90.1

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.


admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/1-guardrails (ggz-seed-project)$ terraform init
Initializing modules...

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/google-beta from the dependency lock file
- Reusing previous version of hashicorp/null from the dependency lock file
- Reusing previous version of hashicorp/google from the dependency lock file
- Using previously-installed hashicorp/random v3.4.3
- Using previously-installed hashicorp/google-beta v3.90.1
- Using previously-installed hashicorp/null v3.1.1
- Using previously-installed hashicorp/google v3.90.1

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/1-guardrails (ggz-seed-project)$ terraform apply -var-file variables.tfvar
data.google_service_account_access_token.default: Reading...
data.google_service_account_access_token.default: Read complete after 0s [id=projects/-/serviceAccounts/tfadmin-ggz@ggz-seed-project.iam.gserviceaccount.com]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create
 <= read (data resources)

Terraform will perform the following actions:

  # google_bigquery_dataset.billing_dataset will be created
  + resource "google_bigquery_dataset" "billing_dataset" {
      + creation_time              = (known after apply)
      + dataset_id                 = "billing_data"
      + delete_contents_on_destroy = false
      + etag                       = (known after apply)
      + friendly_name              = "GCP Billing Data"
      + id                         = (known after apply)
      + last_modified_time         = (known after apply)
      + location                   = "northamerica-northeast1"
      + project                    = (known after apply)
      + self_link                  = (known after apply)

      + access {
          + domain         = (known after apply)
          + group_by_email = (known after apply)
          + role           = (known after apply)
          + special_group  = (known after apply)
          + user_by_email  = (known after apply)

          + view {
              + dataset_id = (known after apply)
              + project_id = (known after apply)
              + table_id   = (known after apply)
            }
        }
    }

  # google_organization_iam_audit_config.org_config[0] will be created
  + resource "google_organization_iam_audit_config" "org_config" {
      + etag    = (known after apply)
      + id      = (known after apply)
      + org_id  = "743091813895"
      + service = "allServices"

      + audit_log_config {
          + exempted_members = []
          + log_type         = "ADMIN_READ"
        }
      + audit_log_config {
          + exempted_members = []
          + log_type         = "DATA_READ"
        }
      + audit_log_config {
          + exempted_members = []
          + log_type         = "DATA_WRITE"
        }
    }

  # google_organization_iam_member.asset_inventory_viewer will be created
  + resource "google_organization_iam_member" "asset_inventory_viewer" {
      + etag   = (known after apply)
      + id     = (known after apply)
      + member = "group:ssc_broker_users@guardrails.gcp.zone"
      + org_id = "743091813895"
      + role   = "roles/cloudasset.viewer"
    }

  # google_organization_iam_member.billing_viewer will be created
  + resource "google_organization_iam_member" "billing_viewer" {
      + etag   = (known after apply)
      + id     = (known after apply)
      + member = "group:billing_data_users@guardrails.gcp.zone"
      + org_id = "743091813895"
      + role   = "roles/billing.viewer"
    }

  # google_organization_iam_member.ssc-billing will be created
  + resource "google_organization_iam_member" "ssc-billing" {
      + etag   = (known after apply)
      + id     = (known after apply)
      + member = "group:ssc_broker_users@guardrails.gcp.zone"
      + org_id = "743091813895"
      + role   = "roles/billing.viewer"
    }

  # google_project_iam_member.audit_log_bq_data_viewer will be created
  + resource "google_project_iam_member" "audit_log_bq_data_viewer" {
      + etag    = (known after apply)
      + id      = (known after apply)
      + member  = "group:audit_data_users@guardrails.gcp.zone"
      + project = (known after apply)
      + role    = "roles/bigquery.dataViewer"
    }

  # google_project_iam_member.audit_log_bq_user will be created
  + resource "google_project_iam_member" "audit_log_bq_user" {
      + etag    = (known after apply)
      + id      = (known after apply)
      + member  = "group:audit_data_users@guardrails.gcp.zone"
      + project = (known after apply)
      + role    = "roles/bigquery.user"
    }

  # google_project_iam_member.billing_bq_user will be created
  + resource "google_project_iam_member" "billing_bq_user" {
      + etag    = (known after apply)
      + id      = (known after apply)
      + member  = "group:billing_data_users@guardrails.gcp.zone"
      + project = (known after apply)
      + role    = "roles/bigquery.user"
    }

  # google_project_iam_member.billing_bq_viewer will be created
  + resource "google_project_iam_member" "billing_bq_viewer" {
      + etag    = (known after apply)
      + id      = (known after apply)
      + member  = "group:billing_data_users@guardrails.gcp.zone"
      + project = (known after apply)
      + role    = "roles/bigquery.dataViewer"
    }

  # google_storage_bucket.guardrails-bucket will be created
  + resource "google_storage_bucket" "guardrails-bucket" {
      + bucket_policy_only          = (known after apply)
      + force_destroy               = true
      + id                          = (known after apply)
      + location                    = "NORTHAMERICA-NORTHEAST1"
      + name                        = "ggz-guardrails-assets"
      + project                     = (known after apply)
      + self_link                   = (known after apply)
      + storage_class               = "STANDARD"
      + uniform_bucket_level_access = true
      + url                         = (known after apply)
    }

  # random_string.suffix will be created
  + resource "random_string" "suffix" {
      + id          = (known after apply)
      + length      = 4
      + lower       = true
      + min_lower   = 0
      + min_numeric = 0
      + min_special = 0
      + min_upper   = 0
      + number      = true
      + numeric     = true
      + result      = (known after apply)
      + special     = false
      + upper       = false
    }

  # module.bigquery_destination.google_bigquery_dataset.dataset will be created
  + resource "google_bigquery_dataset" "dataset" {
      + creation_time               = (known after apply)
      + dataset_id                  = "audit_logs"
      + default_table_expiration_ms = 2592000000
      + delete_contents_on_destroy  = true
      + description                 = "Log export dataset"
      + etag                        = (known after apply)
      + id                          = (known after apply)
      + last_modified_time          = (known after apply)
      + location                    = "northamerica-northeast1"
      + project                     = (known after apply)
      + self_link                   = (known after apply)

      + access {
          + domain         = (known after apply)
          + group_by_email = (known after apply)
          + role           = (known after apply)
          + special_group  = (known after apply)
          + user_by_email  = (known after apply)

          + view {
              + dataset_id = (known after apply)
              + project_id = (known after apply)
              + table_id   = (known after apply)
            }
        }
    }

  # module.bigquery_destination.google_project_iam_member.bigquery_sink_member will be created
  + resource "google_project_iam_member" "bigquery_sink_member" {
      + etag    = (known after apply)
      + id      = (known after apply)
      + member  = (known after apply)
      + project = (known after apply)
      + role    = "roles/bigquery.dataEditor"
    }

  # module.bigquery_destination.google_project_service.enable_destination_api will be created
  + resource "google_project_service" "enable_destination_api" {
      + disable_on_destroy = false
      + id                 = (known after apply)
      + project            = (known after apply)
      + service            = "bigquery.googleapis.com"
    }

  # module.log_export_to_biqquery.google_logging_organization_sink.sink[0] will be created
  + resource "google_logging_organization_sink" "sink" {
      + destination      = (known after apply)
      + filter           = <<-EOT
                logName: /logs/cloudaudit.googleapis.com%2Factivity OR
                logName: /logs/cloudaudit.googleapis.com%2Fsystem_event OR
                logName: /logs/cloudaudit.googleapis.com%2Fdata_access OR
                logName: /logs/compute.googleapis.com%2Fvpc_flows OR
                logName: /logs/compute.googleapis.com%2Ffirewall OR
                logName: /logs/cloudaudit.googleapis.com%2Faccess_transparency
        EOT
      + id               = (known after apply)
      + include_children = true
      + name             = "log_sink-bq"
      + org_id           = "743091813895"
      + writer_identity  = (known after apply)

      + bigquery_options {
          + use_partitioned_tables = (known after apply)
        }
    }

  # module.log_export_to_pubsub.google_logging_organization_sink.sink[0] will be created
  + resource "google_logging_organization_sink" "sink" {
      + destination      = (known after apply)
      + filter           = <<-EOT
                logName: /logs/cloudaudit.googleapis.com%2Factivity OR
                logName: /logs/cloudaudit.googleapis.com%2Fsystem_event OR
                logName: /logs/cloudaudit.googleapis.com%2Fdata_access OR
                logName: /logs/compute.googleapis.com%2Fvpc_flows OR
                logName: /logs/compute.googleapis.com%2Ffirewall OR
                logName: /logs/cloudaudit.googleapis.com%2Faccess_transparency
        EOT
      + id               = (known after apply)
      + include_children = true
      + name             = "sk-c-logging-pub"
      + org_id           = "743091813895"
      + writer_identity  = (known after apply)

      + bigquery_options {
          + use_partitioned_tables = (known after apply)
        }
    }

  # module.log_export_to_storage.google_logging_organization_sink.sink[0] will be created
  + resource "google_logging_organization_sink" "sink" {
      + destination      = (known after apply)
      + id               = (known after apply)
      + include_children = true
      + name             = "org_log_sink"
      + org_id           = "743091813895"
      + writer_identity  = (known after apply)

      + bigquery_options {
          + use_partitioned_tables = (known after apply)
        }
    }

  # module.org-policy.google_organization_policy.org_policy_list_allow_values[0] will be created
  + resource "google_organization_policy" "org_policy_list_allow_values" {
      + constraint  = "constraints/gcp.resourceLocations"
      + etag        = (known after apply)
      + id          = (known after apply)
      + org_id      = "743091813895"
      + update_time = (known after apply)
      + version     = (known after apply)

      + list_policy {
          + suggested_value = (known after apply)

          + allow {
              + all    = false
              + values = [
                  + "northamerica-northeast1",
                  + "northamerica-northeast2",
                ]
            }
        }
    }

  # module.pubsub_destination.google_project_service.enable_destination_api will be created
  + resource "google_project_service" "enable_destination_api" {
      + disable_on_destroy = false
      + id                 = (known after apply)
      + project            = (known after apply)
      + service            = "pubsub.googleapis.com"
    }

  # module.pubsub_destination.google_pubsub_subscription.pubsub_subscription[0] will be created
  + resource "google_pubsub_subscription" "pubsub_subscription" {
      + ack_deadline_seconds       = (known after apply)
      + id                         = (known after apply)
      + message_retention_duration = "604800s"
      + name                       = (known after apply)
      + path                       = (known after apply)
      + project                    = (known after apply)
      + topic                      = (known after apply)

      + expiration_policy {
          + ttl = (known after apply)
        }
    }

  # module.pubsub_destination.google_pubsub_subscription_iam_member.pubsub_subscriber_role[0] will be created
  + resource "google_pubsub_subscription_iam_member" "pubsub_subscriber_role" {
      + etag         = (known after apply)
      + id           = (known after apply)
      + member       = (known after apply)
      + project      = (known after apply)
      + role         = "roles/pubsub.subscriber"
      + subscription = (known after apply)
    }

  # module.pubsub_destination.google_pubsub_topic.topic will be created
  + resource "google_pubsub_topic" "topic" {
      + id      = (known after apply)
      + name    = (known after apply)
      + project = (known after apply)

      + message_storage_policy {
          + allowed_persistence_regions = (known after apply)
        }

      + schema_settings {
          + encoding = (known after apply)
          + schema   = (known after apply)
        }
    }

  # module.pubsub_destination.google_pubsub_topic_iam_member.pubsub_sink_member will be created
  + resource "google_pubsub_topic_iam_member" "pubsub_sink_member" {
      + etag    = (known after apply)
      + id      = (known after apply)
      + member  = (known after apply)
      + project = (known after apply)
      + role    = "roles/pubsub.publisher"
      + topic   = (known after apply)
    }

  # module.pubsub_destination.google_pubsub_topic_iam_member.pubsub_viewer_role[0] will be created
  + resource "google_pubsub_topic_iam_member" "pubsub_viewer_role" {
      + etag    = (known after apply)
      + id      = (known after apply)
      + member  = (known after apply)
      + project = (known after apply)
      + role    = "roles/pubsub.viewer"
      + topic   = (known after apply)
    }

  # module.pubsub_destination.google_service_account.pubsub_subscriber[0] will be created
  + resource "google_service_account" "pubsub_subscriber" {
      + account_id   = (known after apply)
      + disabled     = false
      + display_name = (known after apply)
      + email        = (known after apply)
      + id           = (known after apply)
      + name         = (known after apply)
      + project      = (known after apply)
      + unique_id    = (known after apply)
    }

  # module.storage_destination.google_project_service.enable_destination_api will be created
  + resource "google_project_service" "enable_destination_api" {
      + disable_on_destroy = false
      + id                 = (known after apply)
      + project            = (known after apply)
      + service            = "storage-component.googleapis.com"
    }

  # module.storage_destination.google_storage_bucket.bucket will be created
  + resource "google_storage_bucket" "bucket" {
      + bucket_policy_only          = (known after apply)
      + force_destroy               = true
      + id                          = (known after apply)
      + location                    = "NORTHAMERICA-NORTHEAST1"
      + name                        = (known after apply)
      + project                     = (known after apply)
      + self_link                   = (known after apply)
      + storage_class               = "STANDARD"
      + uniform_bucket_level_access = true
      + url                         = (known after apply)

      + versioning {
          + enabled = true
        }
    }

  # module.storage_destination.google_storage_bucket_iam_member.storage_sink_member will be created
  + resource "google_storage_bucket_iam_member" "storage_sink_member" {
      + bucket = (known after apply)
      + etag   = (known after apply)
      + id     = (known after apply)
      + member = (known after apply)
      + role   = "roles/storage.objectCreator"
    }

  # module.administration.module.budget.data.google_project.project[0] will be read during apply
  # (config refers to values not yet known)
 <= data "google_project" "project" {
      + auto_create_network = (known after apply)
      + billing_account     = (known after apply)
      + folder_id           = (known after apply)
      + id                  = (known after apply)
      + labels              = (known after apply)
      + name                = (known after apply)
      + number              = (known after apply)
      + org_id              = (known after apply)
      + project_id          = (known after apply)
      + skip_delete         = (known after apply)
    }

  # module.administration.module.project-factory.google_project.main will be created
  + resource "google_project" "main" {
      + auto_create_network = false
      + billing_account     = "016706-67373C-2417D0"
      + folder_id           = (known after apply)
      + id                  = (known after apply)
      + labels              = {
          + "application_name"  = "org-logging"
          + "billing_code"      = "1234"
          + "business_code"     = "abcd"
          + "env_code"          = "p"
          + "environment"       = "production"
          + "primary_contact"   = "example1"
          + "secondary_contact" = "example2"
        }
      + name                = "guardrails"
      + number              = (known after apply)
      + org_id              = "743091813895"
      + project_id          = (known after apply)
      + skip_delete         = (known after apply)
    }

  # module.administration.module.project-factory.google_project_default_service_accounts.default_service_accounts[0] will be created
  + resource "google_project_default_service_accounts" "default_service_accounts" {
      + action           = "DEPRIVILEGE"
      + id               = (known after apply)
      + project          = (known after apply)
      + restore_policy   = "REVERT_AND_IGNORE_FAILURE"
      + service_accounts = (known after apply)
    }

  # module.administration.module.project-factory.google_service_account.default_service_account[0] will be created
  + resource "google_service_account" "default_service_account" {
      + account_id   = "project-service-account"
      + disabled     = false
      + display_name = "guardrails Project Service Account"
      + email        = (known after apply)
      + id           = (known after apply)
      + name         = (known after apply)
      + project      = (known after apply)
      + unique_id    = (known after apply)
    }

  # module.administration.module.project-factory.random_id.random_project_id_suffix will be created
  + resource "random_id" "random_project_id_suffix" {
      + b64_std     = (known after apply)
      + b64_url     = (known after apply)
      + byte_length = 2
      + dec         = (known after apply)
      + hex         = (known after apply)
      + id          = (known after apply)
    }

  # module.administration.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"] will be created
  + resource "google_project_service" "project_services" {
      + disable_dependent_services = true
      + disable_on_destroy         = true
      + id                         = (known after apply)
      + project                    = (known after apply)
      + service                    = "bigquery.googleapis.com"
    }

  # module.administration.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"] will be created
  + resource "google_project_service" "project_services" {
      + disable_dependent_services = true
      + disable_on_destroy         = true
      + id                         = (known after apply)
      + project                    = (known after apply)
      + service                    = "billingbudgets.googleapis.com"
    }

  # module.administration.module.project-factory.module.project_services.google_project_service.project_services["cloudasset.googleapis.com"] will be created
  + resource "google_project_service" "project_services" {
      + disable_dependent_services = true
      + disable_on_destroy         = true
      + id                         = (known after apply)
      + project                    = (known after apply)
      + service                    = "cloudasset.googleapis.com"
    }

  # module.administration.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"] will be created
  + resource "google_project_service" "project_services" {
      + disable_dependent_services = true
      + disable_on_destroy         = true
      + id                         = (known after apply)
      + project                    = (known after apply)
      + service                    = "logging.googleapis.com"
    }

Plan: 36 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value:

yes

2042:30

module.administration.module.project-factory.random_id.random_project_id_suffix: Creating...
module.administration.module.project-factory.random_id.random_project_id_suffix: Creation complete after 0s [id=i_0]
random_string.suffix: Creating...
random_string.suffix: Creation complete after 0s [id=isb1]
google_organization_iam_member.ssc-billing: Creating...
google_organization_iam_member.billing_viewer: Creating...
module.administration.module.project-factory.google_project.main: Creating...
module.org-policy.google_organization_policy.org_policy_list_allow_values[0]: Creating...
google_organization_iam_member.asset_inventory_viewer: Creating...
google_organization_iam_audit_config.org_config[0]: Creating...
module.org-policy.google_organization_policy.org_policy_list_allow_values[0]: Creation complete after 1s [id=743091813895/constraints/gcp.resourceLocations]


google_organization_iam_member.asset_inventory_viewer: Creation complete after 5s [id=743091813895/roles/cloudasset.viewer/group:ssc_broker_users@guardrails.gcp.zone]
google_organization_iam_member.billing_viewer: Still creating... [10s elapsed]
google_organization_iam_member.ssc-billing: Still creating... [10s elapsed]
module.administration.module.project-factory.google_project.main: Still creating... [10s elapsed]
google_organization_iam_audit_config.org_config[0]: Still creating... [10s elapsed]
google_organization_iam_member.ssc-billing: Creation complete after 16s [id=743091813895/roles/billing.viewer/group:ssc_broker_users@guardrails.gcp.zone]
google_organization_iam_member.billing_viewer: Creation complete after 16s [id=743091813895/roles/billing.viewer/group:billing_data_users@guardrails.gcp.zone]
google_organization_iam_audit_config.org_config[0]: Creation complete after 17s [id=743091813895/audit_config/allServices]
module.administration.module.project-factory.google_project.main: Still creating... [20s elapsed]
module.administration.module.project-factory.google_project.main: Still creating... [30s elapsed]
module.administration.module.project-factory.google_project.main: Still creating... [40s elapsed]
module.administration.module.project-factory.google_project.main: Still creating... [50s elapsed]
module.administration.module.project-factory.google_project.main: Still creating... [1m0s elapsed]
module.administration.module.project-factory.google_project.main: Still creating... [1m10s elapsed]
module.administration.module.project-factory.google_project.main: Still creating... [1m20s elapsed]
module.administration.module.project-factory.google_project.main: Still creating... [1m30s elapsed]

20:44:30

module.administration.module.project-factory.google_project.main: Still creating... [1m40s elapsed]
module.administration.module.project-factory.google_project.main: Still creating... [1m50s elapsed]
module.administration.module.project-factory.google_project.main: Still creating... [2m0s elapsed]
module.administration.module.project-factory.google_project.main: Still creating... [2m10s elapsed]
module.administration.module.project-factory.google_project.main: Still creating... [2m20s elapsed]
module.administration.module.project-factory.google_project.main: Still creating... [2m30s elapsed]
module.administration.module.project-factory.google_project.main: Still creating... [2m40s elapsed]
module.administration.module.project-factory.google_project.main: Still creating... [2m50s elapsed]
module.administration.module.project-factory.google_project.main: Creation complete after 2m52s [id=projects/guardrails-8bfd]
module.administration.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Creating...
module.administration.module.project-factory.google_service_account.default_service_account[0]: Creating...
module.administration.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Creating...
module.administration.module.project-factory.module.project_services.google_project_service.project_services["cloudasset.googleapis.com"]: Creating...
module.administration.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Creating...
module.administration.module.project-factory.google_service_account.default_service_account[0]: Creation complete after 1s [id=projects/guardrails-8bfd/serviceAccounts/project-service-account@guardrails-8bfd.iam.gserviceaccount.com]


20:46

module.administration.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Still creating... [10s elapsed]
module.administration.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Still creating... [10s elapsed]
module.administration.module.project-factory.module.project_services.google_project_service.project_services["cloudasset.googleapis.com"]: Still creating... [10s elapsed]
module.administration.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Still creating... [10s elapsed]
module.administration.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Creation complete after 18s [id=guardrails-8bfd/bigquery.googleapis.com]
module.administration.module.project-factory.module.project_services.google_project_service.project_services["cloudasset.googleapis.com"]: Creation complete after 18s [id=guardrails-8bfd/cloudasset.googleapis.com]
module.administration.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Creation complete after 18s [id=guardrails-8bfd/billingbudgets.googleapis.com]
module.administration.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Creation complete after 18s [id=guardrails-8bfd/logging.googleapis.com]
module.administration.module.budget.data.google_project.project[0]: Reading...
module.administration.module.project-factory.google_project_default_service_accounts.default_service_accounts[0]: Creating...
module.pubsub_destination.google_project_service.enable_destination_api: Creating...
google_project_iam_member.audit_log_bq_data_viewer: Creating...
google_project_iam_member.billing_bq_viewer: Creating...
google_project_iam_member.billing_bq_user: Creating...
google_project_iam_member.audit_log_bq_user: Creating...
module.storage_destination.google_project_service.enable_destination_api: Creating...
google_bigquery_dataset.billing_dataset: Creating...
google_storage_bucket.guardrails-bucket: Creating...
module.administration.module.budget.data.google_project.project[0]: Read complete after 0s [id=projects/guardrails-8bfd]
module.bigquery_destination.google_project_service.enable_destination_api: Creating...
module.administration.module.project-factory.google_project_default_service_accounts.default_service_accounts[0]: Creation complete after 0s [id=projects/guardrails-8bfd]
google_storage_bucket.guardrails-bucket: Creation complete after 0s [id=ggz-guardrails-assets]
google_bigquery_dataset.billing_dataset: Creation complete after 1s [id=projects/guardrails-8bfd/datasets/billing_data]
module.bigquery_destination.google_project_service.enable_destination_api: Creation complete after 3s [id=guardrails-8bfd/bigquery.googleapis.com]
module.bigquery_destination.google_bigquery_dataset.dataset: Creating...
module.bigquery_destination.google_bigquery_dataset.dataset: Creation complete after 1s [id=projects/guardrails-8bfd/datasets/audit_logs]
module.log_export_to_biqquery.google_logging_organization_sink.sink[0]: Creating...
module.log_export_to_biqquery.google_logging_organization_sink.sink[0]: Creation complete after 1s [id=organizations/743091813895/sinks/log_sink-bq]
module.bigquery_destination.google_project_iam_member.bigquery_sink_member: Creating...
google_project_iam_member.audit_log_bq_data_viewer: Creation complete after 7s [id=guardrails-8bfd/roles/bigquery.dataViewer/group:audit_data_users@guardrails.gcp.zone]
google_project_iam_member.billing_bq_viewer: Creation complete after 7s [id=guardrails-8bfd/roles/bigquery.dataViewer/group:billing_data_users@guardrails.gcp.zone]
google_project_iam_member.audit_log_bq_user: Creation complete after 7s [id=guardrails-8bfd/roles/bigquery.user/group:audit_data_users@guardrails.gcp.zone]
google_project_iam_member.billing_bq_user: Creation complete after 7s [id=guardrails-8bfd/roles/bigquery.user/group:billing_data_users@guardrails.gcp.zone]
module.pubsub_destination.google_project_service.enable_destination_api: Still creating... [10s elapsed]
module.storage_destination.google_project_service.enable_destination_api: Still creating... [10s elapsed]
module.bigquery_destination.google_project_iam_member.bigquery_sink_member: Creation complete after 7s [id=guardrails-8bfd/roles/bigquery.dataEditor/serviceAccount:o743091813895-912373@gcp-sa-logging.iam.gserviceaccount.com]


module.storage_destination.google_project_service.enable_destination_api: Creation complete after 20s [id=guardrails-8bfd/storage-component.googleapis.com]
module.pubsub_destination.google_project_service.enable_destination_api: Creation complete after 20s [id=guardrails-8bfd/pubsub.googleapis.com]
module.pubsub_destination.google_pubsub_topic.topic: Creating...
module.storage_destination.google_storage_bucket.bucket: Creating...
module.storage_destination.google_storage_bucket.bucket: Creation complete after 1s [id=bkt-guardrails-8bfd-org-logs-isb1]
module.log_export_to_storage.google_logging_organization_sink.sink[0]: Creating...
module.log_export_to_storage.google_logging_organization_sink.sink[0]: Creation complete after 2s [id=organizations/743091813895/sinks/org_log_sink]
module.storage_destination.google_storage_bucket_iam_member.storage_sink_member: Creating...
module.pubsub_destination.google_pubsub_topic.topic: Creation complete after 4s [id=projects/guardrails-8bfd/topics/tp-org-logs-isb1]
module.log_export_to_pubsub.google_logging_organization_sink.sink[0]: Creating...
module.pubsub_destination.google_pubsub_subscription.pubsub_subscription[0]: Creating...
module.pubsub_destination.google_service_account.pubsub_subscriber[0]: Creating...
module.pubsub_destination.google_service_account.pubsub_subscriber[0]: Creation complete after 1s [id=projects/guardrails-8bfd/serviceAccounts/tp-org-logs-isb1-subscriber@guardrails-8bfd.iam.gserviceaccount.com]
module.pubsub_destination.google_pubsub_topic_iam_member.pubsub_viewer_role[0]: Creating...
module.log_export_to_pubsub.google_logging_organization_sink.sink[0]: Creation complete after 1s [id=organizations/743091813895/sinks/sk-c-logging-pub]
module.pubsub_destination.google_pubsub_topic_iam_member.pubsub_sink_member: Creating...
module.pubsub_destination.google_pubsub_subscription.pubsub_subscription[0]: Creation complete after 2s [id=projects/guardrails-8bfd/subscriptions/tp-org-logs-isb1-subscription]
module.pubsub_destination.google_pubsub_subscription_iam_member.pubsub_subscriber_role[0]: Creating...
module.storage_destination.google_storage_bucket_iam_member.storage_sink_member: Creation complete after 3s [id=b/bkt-guardrails-8bfd-org-logs-isb1/roles/storage.objectCreator/serviceAccount:o743091813895-073944@gcp-sa-logging.iam.gserviceaccount.com]
module.pubsub_destination.google_pubsub_subscription_iam_member.pubsub_subscriber_role[0]: Creation complete after 4s [id=projects/guardrails-8bfd/subscriptions/tp-org-logs-isb1-subscription/roles/pubsub.subscriber/serviceAccount:tp-org-logs-isb1-subscriber@guardrails-8bfd.iam.gserviceaccount.com]
module.pubsub_destination.google_pubsub_topic_iam_member.pubsub_sink_member: Creation complete after 8s [id=projects/guardrails-8bfd/topics/tp-org-logs-isb1/roles/pubsub.publisher/serviceAccount:o743091813895-086501@gcp-sa-logging.iam.gserviceaccount.com]
module.pubsub_destination.google_pubsub_topic_iam_member.pubsub_viewer_role[0]: Creation complete after 8s [id=projects/guardrails-8bfd/topics/tp-org-logs-isb1/roles/pubsub.viewer/serviceAccount:tp-org-logs-isb1-subscriber@guardrails-8bfd.iam.gserviceaccount.com]

Apply complete! Resources: 36 added, 0 changed, 0 destroyed.


20:47

after 5 min check

asset inventory

The text was updated successfully, but these errors were encountered:

see full clean org testing in canada-ca#47

fmichaelobrien · 2022-09-16T01:59:46Z

Summary: the unsupported argument error - is an expected result of an API change
Fix is usually to adapt the module or inherit the variable - here is is just to remove the reference in projects.tf

admin_@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/1-guardrails (ggz-seed-project)$ terraform apply -var-file variables.tfvar
╷
│ Error: Unsupported argument
│
│   on projects.tf line 5, in module "administration":
│    5:   impersonate_service_account = var.terraform_service_account
│
│ An argument named "impersonate_service_account" is not expected here.

#49

fmichaelobrien · 2022-09-16T17:50:45Z

Validation

https://github.com/canada-ca/cloud-guardrails-gcp/blob/main/guardrails-validation/README.md

create a separate validation-nnn project
turn off the location restriction - to allow for us buckets (or adjust the mb) on the project

Do you want to continue (Y/n)?  ^C

Command killed by keyboard interrupt


admin_root@cloudshell:~$ gcloud config set project validator-ncinfo
admin_root@cloudshell:~ (validator-ncinfo)$ mkdir validator
admin_root@cloudshell:~ (validator-ncinfo)$ cd validator/
admin_root@cloudshell:~/validator (validator-ncinfo)$ export MY_BUCKET_NAME=gr-validator-ncinfo
set billing
admin_root@cloudshell:~/validator (validator-ncinfo)$ gsutil mb gs://$MY_BUCKET_NAME
Creating gs://gr-validator-ncinfo/...

admin_root@cloudshell:~/validator (validator-ncinfo)$ gcloud asset export --output-path=gs://$MY_BUCKET_NAME/resource_inventory.json --content-type=resource --project=guardrails-eaba
ERROR: (gcloud.asset.export) code: 403
message: service-65453940734@gcp-sa-cloudasset.iam.gserviceaccount.com does not have
  storage.objects.create access to the Google Cloud Storage object.
status: PERMISSION_DENIED

https://cloud.google.com/asset-inventory/docs/export-asset-metadata
https://cloud.google.com/asset-inventory/docs/configuring-permissions
add security reviewer to SA user (no)

admin_root@cloudshell:~/validator (validator-ncinfo)$ gcloud projects add-iam-policy-binding validator-nfinfo      --member user:admin-root@cloud-nuage.info      --role roles/cloudasset.viewer
ERROR: (gcloud.projects.add-iam-policy-binding) User [admin-root@nuage-cloud.info] does not have permission to access projects instance [validator-nfinfo:getIamPolicy] (or it may not exist): The caller does not have permission

obriensystems · 2022-09-17T12:31:59Z

Billing Summary

see (needs update for shared billing)
https://cloud.google.com/billing/docs/how-to/billing-access

type 1: shared billing account where account owner in other org adds the super admin account in this org as a Billing Account Administrator and (usually missed) the terraform service account
type 2: direct billing credit card on this account (all tests above so far are this case)

State of billing id associations for type 2 are the following - notice that the terraform service account is in the list as well as the user sa

billing account administrator
billing account user
logs configuration writer (via tf)
guardrails gcp zone

nuage-cloud info

Reproduced issue where a 3rd party billing id in org 2 is used by org 1 - by adding org 1 sa as BAA in org 2. What happens is the normally inherited roles (beyond BAA) are not properly set in billing when the IAM role is set. As a result Billing Account User must be set manually for the terraform SA in the billing view or we get the following

terraform destroy - the deployments, then change the billing id, terraform init and apply

google_organization_iam_audit_config.org_config[0]: Creation complete after 17s [id=93413315325/audit_config/allServices]
╷
│ Error: failed pre-requisites: missing permission on "billingAccounts/019283-6F1AB5-7AD576": billing.resourceAssociations.create
│
│   with module.administration.module.project-factory.google_project.main,
│   on .terraform/modules/administration/modules/core_project_factory/main.tf line 65, in resource "google_project" "main":
│   65: resource "google_project" "main" {
│
╵
admin_root@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/1-guardrails (sscncinfo-seed-project)$ terraform apply -var-file variables.tfvar

manually set the role on both IAM and billing (why?)

If the billing account is from another org - the role will not get inherited automatically from IAM - you must also set it in billing

rerun

Plan: 30 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.org-policy.google_organization_policy.org_policy_list_allow_values[0]: Modifying... [id=93413315325/constraints/gcp.resourceLocations]
module.administration.module.project-factory.google_project.main: Creating...
module.org-policy.google_organization_policy.org_policy_list_allow_values[0]: Modifications complete after 1s [id=93413315325/constraints/gcp.resourceLocations]
module.administration.module.project-factory.google_project.main: Still creating... [10s elapsed]

module.pubsub_destination.google_pubsub_topic_iam_member.pubsub_sink_member: Creation complete after 9s [id=projects/guardrails-5def/topics/tp-org-logs-zvw4/roles/pubsub.publisher/serviceAccount:o93413315325-642767@gcp-sa-logging.iam.gserviceaccount.com]

Apply complete! Resources: 30 added, 1 changed, 0 destroyed.

TL;DR; Shared billing accounts do not get shared IAM roles - they need to be set separately

Keep in mind that we need a workaround (see #47) for the fact that if the billing account is of type "shared" where it comes in under "No Organization, ID=0" then any service account created will not get inherited links from IAM set in Billing - these like Billing Account User - need to be set manually.

Example

michael@cloudshell:~$ gcloud config set project gcp-zone-landing-stg
Updated property [core/project].
michael@cloudshell:~ (gcp-zone-landing-stg)$ export PROJECT_ID=$(gcloud config list --format 'value(core.project)')
michael@cloudshell:~ (gcp-zone-landing-stg)$ export ORG_ID=$(gcloud projects get-ancestors $PROJECT_ID --format='get(id)' | tail -1)
michael@cloudshell:~ (gcp-zone-landing-stg)$ export SA_PREFIX=tfsa-example
michael@cloudshell:~ (gcp-zone-landing-stg)$ gcloud iam service-accounts create "${SA_PREFIX}" --display-name "Terraform example service account" --project=${PROJECT_ID}
Created service account [tfsa-example].
michael@cloudshell:~ (gcp-zone-landing-stg)$ export SA_EMAIL=`gcloud iam service-accounts list --project="${PROJECT_ID}" --filter=tfsa --format="value(email)"`
michael@cloudshell:~ (gcp-zone-landing-stg)$ echo $SA_EMAIL
tfsa-example@gcp-zone-landing-stg.iam.gserviceaccount.com

check existing roles
michael@cloudshell:~ (gcp-zone-landing-stg)$ gcloud organizations get-iam-policy $ORG_ID --filter="bindings.members:$SA_EMAIL" --flatten="bindings[].members" --format="table(bindings.role)"

Set the billing role
gcloud organizations add-iam-policy-binding ${ORG_ID}  --member=serviceAccount:${SA_EMAIL} --role=roles/billing.user

check again

michael@cloudshell:~ (gcp-zone-landing-stg)$ gcloud organizations add-iam-policy-binding ${ORG_ID}  --member=serviceAccount:${SA_EMAIL} --role=roles/billing.user
Updated IAM policy for organization [925207728429].
...
michael@cloudshell:~ (gcp-zone-landing-stg)$ gcloud organizations get-iam-policy $ORG_ID --filter="bindings.members:$SA_EMAIL" --flatten="bindings[].members" --format="table(bindings.role)"

ROLE: roles/billing.user

It may take a couple min to show in IAM

Checking billing on the shared account

expected on billing accounts belonging to this org - via IAM inheritance in billing

not expected on billing accounts shared from other orgs

#47 - terraform 1.2.8 breaks without SA removal - critical fix

fmichaelobrien · 2022-10-13T01:55:28Z

Notes on rerunning the script after any type of inadvertent typo in variables.tfvar

 You will need to get back into the project (with associated re-auth that will kick in), and cd into the 1- directory before running init and apply again

gcloud config set project gr-glc
cd deployment-templates/Terraform/guardrails/
cd ../1-guardrails/
terraform init
terraform apply -var-file variables.tfvar

Notes
https://github.com/canada-ca/accelerators_accelerateurs-gcp/tree/main/deployment-templates/Terraform/guardrails#stage-1---common-resources

Example install
#47

fmichaelobrien · 2022-11-18T22:52:23Z

Guardrails install instructions
We will need the following artifacts at the meeting to proceed. (essentially admin level - domain and BID access)
1 - a BID (Billing ID) - the shared billing account
Note: this BID will need at least 3 open project/billing association quota left - new accounts get 5 default - not normally an issue
2 - the GCP org owning the BID above - where we will add the new SA as a BAA (billing account administrator)
3 - any Billing Account Administrator cloud identity user on the GCP domain that owns the BID above - to be able to add the new cloud identity account (Admin: super admin, GCP: organization administrator) that bootstraps the new HC org
we will need the account below to be added as a BAA on the BID org
4 - a person with any email that will be the new bootstrap cloud identity super admin (root) account on the new HC org - this user will need to configure MFA (defaulting to a phone text)
example: super-admin@guardrails.gcp.zone
5 - Pick a domain name for the GCP org - note that this is just an org identifier not a future FQDN frontend for later - it does not have to be used for A and CNAME records later - something like guardrails.gcp.zone
6 - Domain validation: an IT/OPS person or anyone who can add a TXT domain record to the root zone (synchronously - in the meeting) - GCP org onboarding takes 2 min after the TXT record DNS propagates - the record can be removed after we are done - or left
example: guardrails.gcp.zone TXT record on the sub domain gcp.zone

 References:

Determine cloud profile 1=sandbox: https://github.com/canada-ca/cloud-guardrails/blob/master/EN/00_Applicable-Scope.md#applicability-of-guardrails-to-cloud-usage-profiles
Shared Billing structure: https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/main/docs/google-cloud-onboarding.md#billing
Identity Onboarding steps: https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/main/docs/google-cloud-onboarding.md#onboarding-category-3b1-3rd-party-email-account---3rd-party-aws-route53-domain-validation---reuse-existing-billing-account
Example Guardrails install run: #47
Guardrails entrypoint to clone code: https://github.com/canada-ca/accelerators_accelerateurs-gcp#gc-accelerators-gcp
Guardrails instructions we will follow: https://github.com/canada-ca/accelerators_accelerateurs-gcp/blob/main/deployment-templates/Terraform/guardrails/README.md#setting-up-your-environment

obriensystems changed the title ~~example install run 20220915~~ example install run 20220915 - to validate terraform 1.2.8 project-factory removal of impersonate_service_account Sep 16, 2022

fmichaelobrien added a commit to fmichaelobrien/accelerators_accelerateurs-gcp that referenced this issue Sep 16, 2022

canada-ca#47 - terraform 1.2.8 breaks without SA removal

182c84d

see full clean org testing in canada-ca#47

fmichaelobrien mentioned this issue Sep 16, 2022

#47 - terraform 1.2.8 breaks without SA removal - critical fix #49

Merged

obriensystems mentioned this issue Sep 18, 2022

Document and add to bootstrap.sh 6 super admin required roles required to prepare for terraform service account permissions #42

Open

ptd-tbs added a commit that referenced this issue Sep 21, 2022

Merge pull request #49 from fmichaelobrien/patch-5

2adb2ae

#47 - terraform 1.2.8 breaks without SA removal - critical fix

fmichaelobrien added a commit to fmichaelobrien/accelerators_accelerateurs-gcp that referenced this issue Nov 18, 2022

canada-ca#47 - add guardrails install prereq

30e8fd7

fmichaelobrien mentioned this issue Feb 16, 2023

Adjust Guardrails Onboarding Documentation for recent changes - ongoing shared billing procedures #67

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

example install run 20220915 - to validate terraform 1.2.8 project-factory removal of impersonate_service_account #47

example install run 20220915 - to validate terraform 1.2.8 project-factory removal of impersonate_service_account #47

obriensystems commented Sep 15, 2022 •

edited

Loading

fmichaelobrien commented Sep 16, 2022

fmichaelobrien commented Sep 16, 2022

obriensystems commented Sep 17, 2022 •

edited

Loading

fmichaelobrien commented Oct 13, 2022

fmichaelobrien commented Nov 18, 2022 •

edited

Loading

example install run 20220915 - to validate terraform 1.2.8 project-factory removal of impersonate_service_account #47

example install run 20220915 - to validate terraform 1.2.8 project-factory removal of impersonate_service_account #47

Comments

obriensystems commented Sep 15, 2022 • edited Loading

fmichaelobrien commented Sep 16, 2022

fmichaelobrien commented Sep 16, 2022

obriensystems commented Sep 17, 2022 • edited Loading

fmichaelobrien commented Oct 13, 2022

fmichaelobrien commented Nov 18, 2022 • edited Loading

obriensystems commented Sep 15, 2022 •

edited

Loading

obriensystems commented Sep 17, 2022 •

edited

Loading

fmichaelobrien commented Nov 18, 2022 •

edited

Loading