Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kpt live apply needs to be 10-15 min not 2 for the reconcile timeout loop - updating readme and script #802

Closed
fmichaelobrien opened this issue Jan 31, 2024 · 0 comments
Closed

kpt live apply needs to be 10-15 min not 2 for the reconcile timeout loop - updating readme and script #802

fmichaelobrien opened this issue Jan 31, 2024 · 0 comments
Assignees
@fmichaelobrien
Labels
automation

Comments

@fmichaelobrien
Copy link
Contributor

fmichaelobrien commented Jan 31, 2024 •
edited
Loading

core-landing-zone deploys ok with a longer timeout of 15 min not the default 2 in the readme
see wiki https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/wiki/DevOps#core-landing-zone--hub-env
see script #766
change: https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/blob/gh766-script/solutions/setup.sh#L311

kpt live apply $REL_SUB_PACKAGE --reconcile-timeout=15m --output=table

see remaining logging-sa missing Storage Admin in #801

see testing of core-landing-zone in #766

increment prefix in vars.sh - for projects and buckets

export PREFIX=cso2

setup.sh code
  echo "kpt live init"
  kpt live init $REL_SUB_PACKAGE --namespace config-control
  # --force
  echo "kpt fn render"
  kpt fn render $REL_SUB_PACKAGE --truncate-output=false
  echo "kpt live apply after 60s wait"
  sleep 60
  kpt live apply $REL_SUB_PACKAGE --reconcile-timeout=15m --output=table
  echo "check status"
  kpt live status --inv-type remote --statuses InProgress,NotFound

rerun

michael@cloudshell:~/kcc-cso/github/pubsec-declarative-toolkit/solutions (kcc-cso)$ ./setup.sh -b kcc-cso -u cso2 -n false -c false -l true -h false -r false -d false -j false -p kcc-cso-4380

wait 60 sec to let the GKE cluster stabilize 15 workloads
KCC_PROJECT_NUMBER: 343139601407
DIRECTORY_CUSTOMER_ID: C02w06bdi
generated derived setters-core-landing-zone.yaml
Directory kpt exists - using it
deploying core-landing-zone
get kpt release package solutions/core-landing-zone version 0.7.1
Package "core-landing-zone":
Fetching https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit@0.7.1
From https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit
 * tag               solutions/core-landing-zone/0.7.1 -> FETCH_HEAD
Adding package "solutions/core-landing-zone".

Fetched 1 package(s).
copy over generated setters.yaml
kpt live init
initializing "resourcegroup.yaml" data (namespace: config-control)...success
kpt fn render
Package "core-landing-zone": 
[RUNNING] "gcr.io/kpt-fn/apply-setters:v0.2"
[PASS] "gcr.io/kpt-fn/apply-setters:v0.2" in 600ms
  Results:
    [info] spec.folderRef.external: set field value to "276061734969"
    [info] metadata.annotations.config.kubernetes.io/depends-on: set field value to "resourcemanager.cnrm.cloud.google.com/namespaces/projects/Project/logging-project-cso2"
    [info] spec.projectRef.name: set field value to "logging-project-cso2"
    [info] spec.locked: set field value to "false"
    [info] spec.retentionDays: set field value to "1"
    [info] metadata.name: set field value to "platform-and-component-log-bucket-cso2"
    [info] metadata.annotations.config.kubernetes.io/depends-on: set field value to "resourcemanager.cnrm.cloud.google.com/namespaces/projects/Project/logging-project-cso2"
    [info] spec.projectRef.name: set field value to "logging-project-cso2"
    [info] spec.locked: set field value to "false"
    [info] spec.retentionDays: set field value to "1"
    [info] metadata.annotations.config.kubernetes.io/depends-on: set field value to "resourcemanager.cnrm.cloud.google.com/namespaces/projects/Project/logging-project-cso2"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "logging-project-cso2"
    [info] metadata.name: set field value to "kcc-cso-4380"
    [info] spec.metricsScope: set field value to "location/global/metricsScopes/logging-project-cso2"
    [info] metadata.annotations.config.kubernetes.io/depends-on: set field value to "resourcemanager.cnrm.cloud.google.com/namespaces/projects/Project/logging-project-cso2"
    [info] spec.resourceRef.name: set field value to "logging-project-cso2"
    [info] spec.bindings[0].members[0].memberFrom.logSinkRef.name: set field value to "org-log-sink-security-logging-project-cso2"
    [info] metadata.annotations.config.kubernetes.io/depends-on: set field value to "resourcemanager.cnrm.cloud.google.com/namespaces/projects/Project/logging-project-cso2"
    [info] spec.resourceRef.name: set field value to "logging-project-cso2"
    [info] metadata.annotations.config.kubernetes.io/depends-on: set field value to "resourcemanager.cnrm.cloud.google.com/namespaces/projects/Project/logging-project-cso2"
    [info] spec.resourceRef.name: set field value to "logging-project-cso2"
    [info] metadata.annotations.config.kubernetes.io/depends-on: set field value to "resourcemanager.cnrm.cloud.google.com/namespaces/projects/Project/logging-project-cso2"
    [info] spec.resourceRef.name: set field value to "logging-project-cso2"
    [info] metadata.annotations.config.kubernetes.io/depends-on: set field value to "resourcemanager.cnrm.cloud.google.com/namespaces/projects/Project/logging-project-cso2"
    [info] spec.resourceRef.name: set field value to "logging-project-cso2"
    [info] metadata.name: set field value to "logging-project-cso2-data-access-sink"
    [info] spec.projectRef.name: set field value to "logging-project-cso2"
    [info] spec.destination.loggingLogBucketRef.external: set field value to "logging.googleapis.com/projects/logging-project-cso2/locations/northamerica-northeast1/buckets/security-log-bucket"
    [info] metadata.name: set field value to "logging-project-cso2"
    [info] spec.name: set field value to "logging-project-cso2"
    [info] spec.billingAccountRef.external: set field value to "01B35D-D56E1A-BAE17A"
    [info] metadata.name: set field value to "logging-project-cso2-logging"
    [info] metadata.annotations.config.kubernetes.io/depends-on: set field value to "resourcemanager.cnrm.cloud.google.com/namespaces/projects/Project/logging-project-cso2"
    [info] spec.projectRef.external: set field value to "logging-project-cso2"
    [info] metadata.name: set field value to "logging-project-cso2-monitoring"
    [info] metadata.annotations.config.kubernetes.io/depends-on: set field value to "resourcemanager.cnrm.cloud.google.com/namespaces/projects/Project/logging-project-cso2"
    [info] spec.projectRef.external: set field value to "logging-project-cso2"
    [info] spec.folderRef.external: set field value to "276061734969"
    [info] metadata.annotations.config.kubernetes.io/depends-on: set field value to "logging.cnrm.cloud.google.com/namespaces/logging/LoggingLogBucket/platform-and-component-log-bucket-cso2"
    [info] spec.destination.loggingLogBucketRef.external: set field value to "logging.googleapis.com/projects/logging-project-cso2/locations/northamerica-northeast1/buckets/platform-and-component-log-bucket-cso2"
    [info] spec.folderRef.external: set field value to "276061734969"
    [info] metadata.name: set field value to "dns-project-cso2-standard-core-public-dns"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "dns-project-cso2"
    [info] metadata.annotations.config.kubernetes.io/depends-on: set field value to "resourcemanager.cnrm.cloud.google.com/namespaces/projects/Project/dns-project-cso2"
    [info] spec.dnsName: set field value to "cloud-setup.org."
    [info] metadata.name: set field value to "dns-project-cso2"
    [info] spec.name: set field value to "dns-project-cso2"
    [info] spec.billingAccountRef.external: set field value to "01B35D-D56E1A-BAE17A"
    [info] metadata.name: set field value to "dns-project-cso2-dns"
    [info] metadata.annotations.config.kubernetes.io/depends-on: set field value to "resourcemanager.cnrm.cloud.google.com/namespaces/projects/Project/dns-project-cso2"
    [info] spec.projectRef.external: set field value to "dns-project-cso2"
    [info] metadata.annotations.config.kubernetes.io/depends-on: set field value to "logging.cnrm.cloud.google.com/namespaces/logging/LoggingLogBucket/platform-and-component-log-bucket-cso2"
    [info] spec.destination.loggingLogBucketRef.external: set field value to "logging.googleapis.com/projects/logging-project-cso2/locations/northamerica-northeast1/buckets/platform-and-component-log-bucket-cso2"
    [info] spec.folderRef.external: set field value to "276061734969"
    [info] metadata.name: set field value to "compute-disable-serial-port-logging-except-kcc-cso-4380"
    [info] spec.projectRef.external: set field value to "kcc-cso-4380"
    [info] metadata.name: set field value to "compute-require-shielded-vm-except-kcc-cso-4380"
    [info] spec.projectRef.external: set field value to "kcc-cso-4380"
    [info] metadata.name: set field value to "compute-restrict-cloud-nat-usage-except-kcc-cso-4380"
    [info] spec.listPolicy.allow.values[0]: set field value to "under:projects/kcc-cso-4380"
    [info] spec.projectRef.external: set field value to "kcc-cso-4380"
    [info] metadata.annotations.config.kubernetes.io/depends-on: set field value to "logging.cnrm.cloud.google.com/namespaces/logging/LoggingLogBucket/platform-and-component-log-bucket-cso2"
    [info] spec.projectRef.external: set field value to "kcc-cso-4380"
    [info] spec.destination.loggingLogBucketRef.external: set field value to "logging.googleapis.com/projects/logging-project-cso2/locations/northamerica-northeast1/buckets/platform-and-component-log-bucket-cso2"
    [info] metadata.name: set field value to "kcc-cso-4380-cloudbilling"
    [info] metadata.namespace: set field value to "config-control"
    [info] spec.projectRef.external: set field value to "kcc-cso-4380"
    [info] metadata.name: set field value to "kcc-cso-4380-cloudresourcemanager"
    [info] metadata.namespace: set field value to "config-control"
    [info] spec.projectRef.external: set field value to "kcc-cso-4380"
    [info] metadata.name: set field value to "kcc-cso-4380-serviceusage"
    [info] metadata.namespace: set field value to "config-control"
    [info] spec.projectRef.external: set field value to "kcc-cso-4380"
    [info] metadata.name: set field value to "kcc-cso-4380-accesscontextmanager"
    [info] metadata.namespace: set field value to "config-control"
    [info] spec.projectRef.external: set field value to "kcc-cso-4380"
    [info] metadata.name: set field value to "kcc-cso-4380-anthos"
    [info] metadata.namespace: set field value to "config-control"
    [info] spec.projectRef.external: set field value to "kcc-cso-4380"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "kcc-cso-4380"
    [info] spec.member: set field value to "serviceAccount:config-mgmt-mon-default-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.bindings[0].members[0].member: set field value to "serviceAccount:kcc-cso-4380.svc.id.goog[config-management-monitoring/default]"
    [info] spec.googleServiceAccount: set field value to "config-mgmt-mon-default-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "kcc-cso-4380"
    [info] spec.member: set field value to "serviceAccount:gatekeeper-admin-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.bindings[0].members[0].member: set field value to "serviceAccount:kcc-cso-4380.svc.id.goog[gatekeeper-system/gatekeeper-admin]"
    [info] spec.googleServiceAccount: set field value to "gatekeeper-admin-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "276061734969"
    [info] spec.member: set field value to "serviceAccount:hierarchy-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.bindings[0].members[0].member: set field value to "serviceAccount:kcc-cso-4380.svc.id.goog[cnrm-system/cnrm-controller-manager-hierarchy]"
    [info] spec.googleServiceAccount: set field value to "hierarchy-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "734065690346"
    [info] spec.member: set field value to "serviceAccount:logging-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.name: set field value to "logging-sa-monitoring-admin-kcc-cso-4380-permissions"
    [info] metadata.namespace: set field value to "config-control"
    [info] spec.resourceRef.external: set field value to "kcc-cso-4380"
    [info] spec.member: set field value to "serviceAccount:logging-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.name: set field value to "logging-sa-monitoring-admin-logging-project-cso2-permissions"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "logging-project-cso2"
    [info] spec.member: set field value to "serviceAccount:logging-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.name: set field value to "logging-sa-storageadmin-logging-project-cso2-permissions"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "logging-project-cso2"
    [info] spec.resourceRef.name: set field value to "logging-project-cso2"
    [info] spec.member: set field value to "serviceAccount:logging-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.bindings[0].members[0].member: set field value to "serviceAccount:kcc-cso-4380.svc.id.goog[cnrm-system/cnrm-controller-manager-logging]"
    [info] spec.googleServiceAccount: set field value to "logging-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "734065690346"
    [info] spec.member: set field value to "serviceAccount:service-343139601407@gcp-sa-yakima.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "kcc-cso-4380"
    [info] spec.member: set field value to "serviceAccount:service-343139601407@gcp-sa-yakima.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "kcc-cso-4380"
    [info] spec.member: set field value to "serviceAccount:service-343139601407@gcp-sa-yakima.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "276061734969"
    [info] spec.member: set field value to "serviceAccount:networking-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "276061734969"
    [info] spec.member: set field value to "serviceAccount:networking-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "276061734969"
    [info] spec.member: set field value to "serviceAccount:networking-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "734065690346"
    [info] spec.member: set field value to "serviceAccount:networking-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "734065690346"
    [info] spec.member: set field value to "serviceAccount:networking-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "276061734969"
    [info] spec.member: set field value to "serviceAccount:networking-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.bindings[0].members[0].member: set field value to "serviceAccount:kcc-cso-4380.svc.id.goog[cnrm-system/cnrm-controller-manager-networking]"
    [info] spec.googleServiceAccount: set field value to "networking-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "734065690346"
    [info] spec.member: set field value to "serviceAccount:policies-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.bindings[0].members[0].member: set field value to "serviceAccount:kcc-cso-4380.svc.id.goog[cnrm-system/cnrm-controller-manager-policies]"
    [info] spec.googleServiceAccount: set field value to "policies-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "276061734969"
    [info] spec.member: set field value to "serviceAccount:projects-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "276061734969"
    [info] spec.member: set field value to "serviceAccount:projects-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "276061734969"
    [info] spec.member: set field value to "serviceAccount:projects-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "276061734969"
    [info] spec.member: set field value to "serviceAccount:projects-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "276061734969"
    [info] spec.member: set field value to "serviceAccount:projects-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.resourceRef.external: set field value to "734065690346"
    [info] spec.member: set field value to "serviceAccount:projects-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/project-id: set field value to "kcc-cso-4380"
    [info] spec.bindings[0].members[0].member: set field value to "serviceAccount:kcc-cso-4380.svc.id.goog[cnrm-system/cnrm-controller-manager-projects]"
    [info] spec.googleServiceAccount: set field value to "projects-sa@kcc-cso-4380.iam.gserviceaccount.com"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/organization-id: set field value to "734065690346"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/organization-id: set field value to "734065690346"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/organization-id: set field value to "734065690346"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/organization-id: set field value to "734065690346"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/organization-id: set field value to "734065690346"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/organization-id: set field value to "734065690346"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/organization-id: set field value to "734065690346"
    [info] metadata.namespace: set field value to "config-control"
    [info] metadata.annotations.cnrm.cloud.google.com/organization-id: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.listPolicy.allow.values: set field value to "- \"under:organizations/734065690346\"\n"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.listPolicy.allow.values: set field value to "- \"projects/cos-cloud\"\n"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.listPolicy.allow.values: set field value to "- \"@cloud-setup.org\"\n"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.listPolicy.allow.values: set field value to "- \"C02w06bdi\"\n"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] metadata.name: set field value to "org-log-sink-security-logging-project-cso2"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.destination.loggingLogBucketRef.external: set field value to "logging.googleapis.com/projects/logging-project-cso2/locations/northamerica-northeast1/buckets/security-log-bucket"
    [info] metadata.name: set field value to "org-log-sink-data-access-logging-project-cso2"
    [info] spec.organizationRef.external: set field value to "734065690346"
    [info] spec.destination.loggingLogBucketRef.external: set field value to "logging.googleapis.com/projects/logging-project-cso2/locations/northamerica-northeast1/buckets/security-log-bucket"

Successfully executed 1 function(s) in 1 package(s).
kpt live apply after 60s wait

apiVersion: v1
kind: ConfigMap
metadata: # kpt-merge: /setters
  name: setters
  annotations:
    config.kubernetes.io/local-config: "true"
    internal.kpt.dev/upstream-identifier: '|ConfigMap|default|setters'
data:
  org-id: "734..46"
  lz-folder-id: "27..9"
  billing-id: "01B...7A"
  management-project-id: "kcc-cso-4380"
  management-project-number: "34...07"
  management-namespace: config-control
  allowed-trusted-image-projects: |
    - "projects/cos-cloud"
  allowed-contact-domains: |
    - "@cloud-setup.org"
  allowed-policy-domain-members: |
    - "C02w06bdi"
  allowed-vpc-peering: |
    - "under:organizations/73...6"
  logging-project-id: logging-project-cso2
  security-log-bucket: security-log-bucket-cso2
  platform-and-component-log-bucket: platform-and-component-log-bucket-cso2
  retention-locking-policy: "false"
  retention-in-days: "1"
  dns-project-id: dns-project-cso2
  dns-name: "cloud-setup.org."

1202 - better

hierarchy   RoleBinding/allow-hierarchy-resource-ref  Successful    Current                 <None>                                    17m     Resource is current                     
hierarchy   RoleBinding/allow-hierarchy-resource-ref  Successful    Current                 <None>                                    17m     Resource is current                     
hierarchy   RoleBinding/allow-hierarchy-resource-ref  Successful    Current                 <None>                                    17m     Resource is current                     
hierarchy   Folder/audits                             Successful    Current                 Ready                                     17m     Resource is Current                     
hierarchy   Folder/clients                            Successful    Current                 Ready                                     17m     Resource is Current                     
hierarchy   Folder/services                           Successful    Current                 Ready                                     17m     Resource is Current                     
hierarchy   Folder/services-infrastructure            Successful    Current                 Ready                                     17m     Resource is Current                     
logging     ConfigConnectorContext/configconnectorco  Successful    Current                 <None>                                    17m     status.healthy is true                  
logging     LoggingLogBucket/platform-and-component-  Successful    Current                 Ready                                     15m     Resource is Current                     
logging     LoggingLogBucket/security-log-bucket      Successful    Current                 Ready                                     15m     Resource is Current                     
logging     LoggingLogSink/logging-project-cso2-data  Successful    Current                 Ready                                     27s     Resource is Current                     
logging     LoggingLogSink/mgmt-project-cluster-plat  Successful    Current                 Ready                                     27s     Resource is Current                     
logging     LoggingLogSink/org-log-sink-data-access-  Successful    Current                 Ready                                     27s     Resource is Current                     
logging     LoggingLogSink/org-log-sink-security-log  Successful    Current                 Ready                                     27s     Resource is Current                     
logging     LoggingLogSink/platform-and-component-se  Successful    Current                 Ready                                     26s     Resource is Current                     
logging     LoggingLogSink/platform-and-component-se  Successful    Current                 Ready                                     26s     Resource is Current                     
logging     MonitoringMonitoredProject/kcc-cso-4380   Successful    Current                 Ready                                     17m     Resource is Current                     
logging     RoleBinding/allow-logging-resource-refer  Successful    Current                 <None>                                    17m     Resource is current                     
logging     StorageBucket/security-incident-log-buck  Successful    Failed                  Ready                                     15m     Update call failed: error fetching live 
networking  ConfigConnectorContext/configconnectorco  Successful    Current                 <None>                                    17m     status.healthy is true                  
networking  DNSManagedZone/dns-project-cso2-standard  Successful    Failed                  Ready                                     28s     Update call failed: error applying desir
policies    ConfigConnectorContext/configconnectorco  Successful    Current                 <None>                                    17m     status.healthy is true                  
policies    ResourceManagerPolicy/compute-disable-gu  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-disable-ne  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-disable-se  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-disable-se  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-disable-se  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-disable-vp  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-require-os  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-require-sh  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-require-sh  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-restrict-c  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-restrict-c  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-restrict-l  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-restrict-s  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-restrict-v  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-restrict-v  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-skip-defau  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-trusted-im  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-vm-can-ip-  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/compute-vm-externa  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/essentialcontacts-  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/gcp-restrict-resou  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/iam-allowed-policy  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/iam-automatic-iam-  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/iam-disable-audit-  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/iam-disable-servic  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/iam-disable-servic  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/sql-restrict-publi  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/storage-public-acc  Successful    Current                 Ready                                     17m     Resource is Current                     
policies    ResourceManagerPolicy/storage-uniform-bu  Successful    Current                 Ready                                     17m     Resource is Current                     
projects    ConfigConnectorContext/configconnectorco  Successful    Current                 <None>                                    17m     status.healthy is true                  
projects    IAMAuditConfig/logging-project-data-acce  Successful    Current                 Ready                                     15m     Resource is Current                     
projects    IAMPartialPolicy/mgmt-project-cluster-pl  Successful    Current                 Ready                                     15m     Resource is Current                     
projects    IAMPartialPolicy/platform-and-component-  Successful    InProgress              Ready                                     15m     reference LoggingLogSink logging/platfor
projects    IAMPartialPolicy/platform-and-component-  Successful    Current                 Ready                                     15m     Resource is Current                     
projects    IAMPartialPolicy/security-log-bucket-wri  Successful    Current                 Ready                                     15m     Resource is Current                     
projects    IAMPolicyMember/logging-sa-monitoring-ad  Successful    Current                 Ready                                     17m     Resource is Current                     
projects    IAMPolicyMember/logging-sa-storageadmin-  Successful    Current                 Ready                                     17m     Resource is Current                     
projects    RoleBinding/allow-projects-resource-refe  Successful    Current                 <None>                                    17m     Resource is current                     
projects    RoleBinding/allow-projects-resource-refe  Successful    Current                 <None>                                    17m     Resource is current                     
projects    RoleBinding/allow-projects-resource-refe  Successful    Current                 <None>                                    17m     Resource is current                     
projects    Project/dns-project-cso2                  Successful    Current                 Ready                                     15m     Resource is Current                     
projects    Project/logging-project-cso2              Successful    Current                 Ready                                     17m     Resource is Current                     
projects    Service/dns-project-cso2-dns              Successful    Current                 Ready                                     26s     Resource is Current                     
projects    Service/logging-project-cso2-logging      Successful    Current                 Ready                                     15m     Resource is Current                     
projects    Service/logging-project-cso2-monitoring   Successful    Current                 Ready                                     15m     Resource is Current                     


michael@cloudshell:~/kcc-cso/kpt (kcc-cso-4380)$ kpt live status core-landing-zone --inv-type remote --statuses InProgress,NotFound
michael@cloudshell:~/kcc-cso/kpt (kcc-cso-4380)$ kpt live status core-landing-zone | grep error
inventory-36537147/storagebucket.storage.cnrm.cloud.google.com/logging/security-incident-log-bucket is Failed: Update call failed: error fetching live state: error reading underlying resource: summary: Error when reading or editing Storage Bucket "security-incident-log-bucket": googleapi: Error 403: logging-sa@kcc-cso-4380.iam.gserviceaccount.com does not have storage.buckets.get access to the Google Cloud Storage bucket. Permission 'storage.buckets.get' denied on resource (or it may not exist)., forbidden

Same issue on redeployed cloud-setup

michael@cloudshell:~/kcc-cso/kpt (kcc-cso-4380)$ kpt live status core-landing-zone | grep error
inventory-36537147/storagebucket.storage.cnrm.cloud.google.com/logging/security-incident-log-bucket is Failed: Update call failed: error fetching live state: error reading underlying resource: summary: Error when reading or editing Storage Bucket "security-incident-log-bucket": googleapi: Error 403: logging-sa@kcc-cso-4380.iam.gserviceaccount.com does not have storage.buckets.get access to the Google Cloud Storage bucket. Permission 'storage.buckets.get' denied on resource (or it may not exist)., forbidden

michael@cloudshell:~/kcc-cso/kpt (kcc-cso-4380)$ kubectl describe storagebucket.storage.cnrm.cloud.google.com/security-incident-log-bucket -n logging
Name:         security-incident-log-bucket
Namespace:    logging
Labels:       <none>
Annotations:  cnrm.cloud.google.com/blueprint: kpt-pkg-fn-live
              cnrm.cloud.google.com/management-conflict-prevention-policy: none
              cnrm.cloud.google.com/project-id: logging-project-cso2
              cnrm.cloud.google.com/state-into-spec: merge
              config.k8s.io/owning-inventory: ec099affabc09ae4652ae62190d9b794c9ec63d1-1706718583884502216
              config.kubernetes.io/depends-on: resourcemanager.cnrm.cloud.google.com/namespaces/projects/Project/logging-project-cso2
              internal.kpt.dev/upstream-identifier: storage.cnrm.cloud.google.com|StorageBucket|logging|security-incident-log-bucket
API Version:  storage.cnrm.cloud.google.com/v1beta1
Kind:         StorageBucket
Metadata:
  Creation Timestamp:  2024-01-31T16:33:31Z
  Generation:          1
  Resource Version:    4501241
  UID:                 b6cc605b-ac0b-45ae-ab03-0854998ab193
Spec:
  Autoclass:
    Enabled:                 true
  Location:                  northamerica-northeast1
  Public Access Prevention:  enforced
  Retention Policy:
    Is Locked:                  false
    Retention Period:           86400
  Uniform Bucket Level Access:  true
Status:
  Conditions:
    Last Transition Time:  2024-01-31T16:33:31Z
    Message:               Update call failed: error fetching live state: error reading underlying resource: summary: Error when reading or editing Storage Bucket "security-incident-log-bucket": googleapi: Error 403: logging-sa@kcc-cso-4380.iam.gserviceaccount.com does not have storage.buckets.get access to the Google Cloud Storage bucket. Permission 'storage.buckets.get' denied on resource (or it may not exist)., forbidden
    Reason:                UpdateFailed
    Status:                False
    Type:                  Ready
  Observed Generation:     1
Events:
  Type     Reason        Age                 From                      Message
  ----     ------        ----                ----                      -------
  Warning  UpdateFailed  93s (x22 over 33m)  storagebucket-controller  Update call failed: error fetching live state: error reading underlying resource: summary: Error when reading or editing Storage Bucket "security-incident-log-bucket": googleapi: Error 403: logging-sa@kcc-cso-4380.iam.gserviceaccount.com does not have storage.buckets.get access to the Google Cloud Storage bucket. Permission 'storage.buckets.get' denied on resource (or it may not exist)., forbidden
  
  however the logging-sa is missing Storage Admin
  
logging-sa@kcc-cso-4380.iam.gserviceaccount.com | logging-sa | Logging AdminMonitoring Admin

https://cloud.google.com/storage/docs/access-control/iam-roles


Storage Admin (roles/storage.admin) | Grants full control of buckets, managed folders, and objects, including getting and setting object ACLs or IAM policies.When applied to an individual bucket, control applies only to the specified bucket and the managed folders and objects within the bucket. | firebase.projects.getorgpolicy.policy.get1resourcemanager.projects.get2resourcemanager.projects.list2storage.buckets.*storage.managedFolders.*storage.objects.*storage.multipartUploads.*
-- | -- | --

added to #801
@fmichaelobrien fmichaelobrien self-assigned this Jan 31, 2024
@fmichaelobrien fmichaelobrien mentioned this issue Jan 31, 2024
Open
fmichaelobrien added a commit that referenced this issue Jan 31, 2024
@fmichaelobrien
#802 - kpt live apply reconcile-timeout from 2 to 15 min
1e51c21 
see testing in #766
@fmichaelobrien fmichaelobrien mentioned this issue Jan 31, 2024
Merged
fmichaelobrien added a commit that referenced this issue Feb 5, 2024
@fmichaelobrien
#802 - kpt live apply reconcile-timeout from 2 to 15 min (#803)
8fb4775 
see testing in #766
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fmichaelobrien fmichaelobrien

Labels
automation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@fmichaelobrien