Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix inconsistent locations in some resources #4377

Conversation

upodroid
Copy link
Contributor

@upodroid upodroid commented Jan 8, 2021

Fixes: hashicorp/terraform-provider-google#8027

If this PR is for Terraform, I acknowledge that I have:

  • Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
  • Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
  • Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
  • Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
  • Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

compute: fixed an issue where `google_compute_region_per_instance_config`, `google_compute_per_instance_config`, `google_compute_region_instance_group_manager` resources were not inferring the region/zone from the provider. 

@google-cla google-cla bot added the cla: yes label Jan 8, 2021
@modular-magician
Copy link
Collaborator

Hello! I am a robot who works on Magic Modules PRs.

I have detected that you are a community contributor, so your PR will be assigned to someone with a commit-bit on this repo for initial review.

Thanks for your contribution! A human will be with you soon.

@melinath, please review this PR or find an appropriate assignee.

@modular-magician
Copy link
Collaborator

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 16 files changed, 80 insertions(+), 90 deletions(-))
Terraform Beta: Diff ( 19 files changed, 88 insertions(+), 106 deletions(-))
Ansible: Diff ( 2 files changed, 9 insertions(+), 8 deletions(-))
TF OiCS: Diff ( 4 files changed, 5 deletions(-))
Inspec: Diff ( 1 file changed, 1 insertion(+), 1 deletion(-))

@upodroid
Copy link
Contributor Author

upodroid commented Jan 8, 2021

Getting weird behavior with per_instance_config resources.

TF is complaining about a missing MIG but it actually exists and it deletes the same MIG right after the failure.

---[ REQUEST ]---------------------------------------
GET /compute/beta/projects/projects/REDACTED/zones/us-central1-c/instanceGroupManagers/tf-test-igm-0qkad7ek8e?alt=json&prettyPrint=false HTTP/1.1
Host: www.googleapis.com
User-Agent: google-api-go-client/0.5 Terraform/0.12.29 (+https://www.terraform.io) Terraform-Plugin-SDK/2.1.0 terraform-provider-google-beta/dev
X-Goog-Api-Client: gl-go/1.15.5 gdcl/20201023
Accept-Encoding: gzip


-----------------------------------------------------
2021/01/08 17:52:55 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Connection: close
Transfer-Encoding: chunked
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Fri, 08 Jan 2021 17:52:55 GMT
Etag: fpswSYt1yq5LYdDayrNCoYWRBho=/I5Ryb6HHUNhsxXLCqv-7jL-09sg=
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

617
{
 "id": "1089190630260024580",
 "creationTimestamp": "2021-01-08T09:52:43.648-08:00",
 "name": "tf-test-igm-0qkad7ek8e",
 "description": "Terraform test instance group manager",
 "zone": "https://www.googleapis.com/compute/beta/projects/REDACTED/zones/us-central1-c",
 "instanceTemplate": "https://www.googleapis.com/compute/beta/projects/REDACTED/global/instanceTemplates/igm-temp-rkjmx8ptxa",
 "versions": [
  {
   "name": "prod",
   "instanceTemplate": "https://www.googleapis.com/compute/beta/projects/REDACTED/global/instanceTemplates/igm-temp-rkjmx8ptxa",
   "targetSize": {
    "calculated": 0
   }
  }
 ],
 "instanceGroup": "https://www.googleapis.com/compute/beta/projects/REDACTED/zones/us-central1-c/instanceGroups/tf-test-igm-0qkad7ek8e",
 "baseInstanceName": "igm-no-tp",
 "fingerprint": "ydypp55Ongk=",
 "currentActions": {
  "none": 0,
  "creating": 0,
  "creatingWithoutRetries": 0,
  "verifying": 0,
  "recreating": 0,
  "deleting": 0,
  "abandoning": 0,
  "restarting": 0,
  "refreshing": 0
 },
 "status": {
  "isStable": true,
  "versionTarget": {
   "isReached": true
  },
  "stateful": {
   "isStateful": false,
   "hasStatefulConfig": false,
   "perInstanceConfigs": {
    "allEffective": true
   }
  }
 },
 "targetSize": 0,
 "selfLink": "https://www.googleapis.com/compute/beta/projects/REDACTED/zones/us-central1-c/instanceGroupManagers/tf-test-igm-0qkad7ek8e",
 "updatePolicy": {
  "type": "OPPORTUNISTIC",
  "minimalAction": "REPLACE",
  "maxSurge": {
   "fixed": 1,
   "calculated": 1
  },
  "maxUnavailable": {
   "fixed": 1,
   "calculated": 1
  },
  "minReadySec": 0,
  "replacementMethod": "SUBSTITUTE"
 },
 "serviceAccount": "550924169191@cloudservices.gserviceaccount.com",
 "kind": "compute#instanceGroupManager"
}
0


-----------------------------------------------------
2021/01/08 17:52:55 [DEBUG] Retry Transport: Stopping retries, last request was successful
2021/01/08 17:52:55 [DEBUG] Retry Transport: Returning after 1 attempts
2021/01/08 17:52:55 [DEBUG] Locking "instanceGroupManager/REDACTED/us-central1-a/tf-test-igm-0qkad7ek8e"
2021/01/08 17:52:55 [DEBUG] Locked "instanceGroupManager/REDACTED/us-central1-a/tf-test-igm-0qkad7ek8e"
2021/01/08 17:52:55 [DEBUG] Creating new PerInstanceConfig: map[string]interface {}{"instances":[]interface {}{map[string]interface {}{"name":"instance-6sxl3o7wew", "preservedState":map[string]interface {}{"metadata":map[string]string{"asdf":"asdf"}}}}}
2021/01/08 17:52:55 [DEBUG] Waiting for state to become: [success]
2021/01/08 17:52:55 [DEBUG] Retry Transport: starting RoundTrip retry loop
2021/01/08 17:52:55 [DEBUG] Retry Transport: request attempt 0
2021/01/08 17:52:55 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
POST /compute/beta/projects/REDACTED/zones/us-central1-a/instanceGroupManagers/tf-test-igm-0qkad7ek8e/createInstances?alt=json HTTP/1.1
Host: compute.googleapis.com
User-Agent: Terraform/0.12.29 (+https://www.terraform.io) Terraform-Plugin-SDK/2.1.0 terraform-provider-google-beta/dev
Content-Length: 93
Content-Type: application/json
Accept-Encoding: gzip

{
 "instances": [
  {
   "name": "instance-6sxl3o7wew",
   "preservedState": {
    "metadata": {
     "asdf": "asdf"
    }
   }
  }
 ]
}

-----------------------------------------------------
2021/01/08 17:52:56 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 404 Not Found
Connection: close
Transfer-Encoding: chunked
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Fri, 08 Jan 2021 17:52:56 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

19b
{
  "error": {
    "code": 404,
    "message": "The resource 'projects/REDACTED/zones/us-central1-a/instanceGroupManagers/tf-test-igm-0qkad7ek8e' was not found",
    "errors": [
      {
        "message": "The resource 'projects/REDACTED/zones/us-central1-a/instanceGroupManagers/tf-test-igm-0qkad7ek8e' was not found",
        "domain": "global",
        "reason": "notFound"
      }
    ]
  }
}

0


-----------------------------------------------------
2021/01/08 17:52:56 [DEBUG] Retry Transport: Stopping retries, last request failed with non-retryable error: googleapi: got HTTP response code 404 with body: HTTP/1.1 404 Not Found
Connection: close
Transfer-Encoding: chunked
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Fri, 08 Jan 2021 17:52:56 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

19b
{
  "error": {
    "code": 404,
    "message": "The resource 'projects/REDACTED/zones/us-central1-a/instanceGroupManagers/tf-test-igm-0qkad7ek8e' was not found",
    "errors": [
      {
        "message": "The resource 'projects/REDACTED/zones/us-central1-a/instanceGroupManagers/tf-test-igm-0qkad7ek8e' was not found",
        "domain": "global",
        "reason": "notFound"
      }
    ]
  }
}

0

2021/01/08 17:52:56 [DEBUG] Retry Transport: Returning after 1 attempts
2021/01/08 17:52:56 [DEBUG] Unlocking "instanceGroupManager/REDACTED/us-central1-a/tf-test-igm-0qkad7ek8e"
2021/01/08 17:52:56 [DEBUG] Unlocked "instanceGroupManager/REDACTED/us-central1-a/tf-test-igm-0qkad7ek8e"
2021/01/08 17:52:56 [WARN] Got error running Terraform: 2021/01/08 17:52:28 [DEBUG] Using modified User-Agent: Terraform/0.12.29 HashiCorp-terraform-exec/0.10.0

Error: Error creating PerInstanceConfig: googleapi: Error 404: The resource 'projects/REDACTED/zones/us-central1-a/instanceGroupManagers/tf-test-igm-0qkad7ek8e' was not found, notFound


    provider_test.go:270: Step 1/2 error: Error running apply: 2021/01/08 17:52:28 [DEBUG] Using modified User-Agent: Terraform/0.12.29 HashiCorp-terraform-exec/0.10.0
        
        Error: Error creating PerInstanceConfig: googleapi: Error 404: The resource 'projects/REDACTED/zones/us-central1-a/instanceGroupManagers/tf-test-igm-0qkad7ek8e' was not found, notFound
        
        
2021/01/08 17:52:59 [WARN] Truncating attribute path of 0 diagnostics for TypeSet
2021/01/08 17:52:59 [WARN] Truncating attribute path of 0 diagnostics for TypeSet
2021/01/08 17:52:59 [INFO] Authenticating using DefaultClient...
2021/01/08 17:52:59 [INFO]   -- Scopes: [https://www.googleapis.com/auth/compute https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/cloud-identity https://www.googleapis.com/auth/ndev.clouddns.readwrite https://www.googleapis.com/auth/devstorage.full_control https://www.googleapis.com/auth/userinfo.email]
2021/01/08 17:52:59 [INFO] Instantiating GCE Beta client for path https://www.googleapis.com/compute/beta/projects/
2021/01/08 17:52:59 [DEBUG] Retry Transport: starting RoundTrip retry loop
2021/01/08 17:52:59 [DEBUG] Retry Transport: request attempt 0
2021/01/08 17:52:59 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
DELETE /compute/beta/projects/projects/REDACTED/zones/us-central1-c/instanceGroupManagers/tf-test-igm-0qkad7ek8e?alt=json&prettyPrint=false HTTP/1.1
Host: www.googleapis.com
User-Agent: google-api-go-client/0.5 Terraform/0.12.29 (+https://www.terraform.io) Terraform-Plugin-SDK/2.1.0 terraform-provider-google-beta/dev
X-Goog-Api-Client: gl-go/1.15.5 gdcl/20201023
Accept-Encoding: gzip


-----------------------------------------------------
2021/01/08 17:53:00 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Connection: close
Transfer-Encoding: chunked
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Fri, 08 Jan 2021 17:53:00 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

302
{
 "id": "4249920173635872019",
 "name": "operation-1610128379511-5b8673a1a8493-c680daf5-969e407b",
 "zone": "https://www.googleapis.com/compute/beta/projects/REDACTED/zones/us-central1-c",
 "operationType": "compute.instanceGroupManagers.delete",
 "targetLink": "https://www.googleapis.com/compute/beta/projects/REDACTED/zones/us-central1-c/instanceGroupManagers/tf-test-igm-0qkad7ek8e",
 "targetId": "1089190630260024580",
 "status": "RUNNING",
 "user": "maha4472@mgcp.rackspace.com",
 "progress": 0,
 "insertTime": "2021-01-08T09:53:00.424-08:00",
 "startTime": "2021-01-08T09:53:00.455-08:00",
 "selfLink": "https://www.googleapis.com/compute/beta/projects/REDACTED/zones/us-central1-c/operations/operation-1610128379511-5b8673a1a8493-c680daf5-969e407b",
 "kind": "compute#operation"
}
0

@melinath
Copy link
Member

melinath commented Jan 8, 2021

@upodroid it looks like the projects are not quite the same:

# missing?
projects/REDACTED/zones/us-central1-a/instanceGroupManagers/tf-test-igm-0qkad7ek8e
# getting deleted
projects/REDACTED/zones/us-central1-c/instanceGroupManagers/tf-test-igm-0qkad7ek8e

Specifically, the object that exists is in us-central1-c, not in us-central1-a. I hope that helps!

@upodroid
Copy link
Contributor Author

upodroid commented Jan 8, 2021

Ah, rookie mistake. All of them are working except the per_instance_config resources. I'll get them fixed for Monday.

@modular-magician
Copy link
Collaborator

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 22 files changed, 108 insertions(+), 117 deletions(-))
Terraform Beta: Diff ( 24 files changed, 115 insertions(+), 132 deletions(-))
Ansible: Diff ( 2 files changed, 9 insertions(+), 8 deletions(-))
TF OiCS: Diff ( 4 files changed, 5 deletions(-))
Inspec: Diff ( 1 file changed, 1 insertion(+), 1 deletion(-))

@modular-magician
Copy link
Collaborator

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 22 files changed, 108 insertions(+), 113 deletions(-))
Terraform Beta: Diff ( 24 files changed, 115 insertions(+), 128 deletions(-))
Ansible: Diff ( 2 files changed, 9 insertions(+), 8 deletions(-))
TF OiCS: Diff ( 3 files changed, 3 deletions(-))
Inspec: Diff ( 1 file changed, 1 insertion(+), 1 deletion(-))

@melinath
Copy link
Member

@upodroid could you split this up into separate PRs based on product? That would make it a lot easier to review & merge. If you ping me on the new PRs I can have a look at them tomorrow.

@upodroid
Copy link
Contributor Author

Hmm, that is long. How about:

  • 1 for Bigtable. That needs a manual run by a human since VCR is disabled.
  • 1 for all MM resources except GCE.
  • 1 for Compute Engine.

@melinath
Copy link
Member

melinath commented Jan 11, 2021

@upodroid sure, that sounds good to me. Thanks!

Co-authored-by: upodroid <cy@borg.dev>
@upodroid upodroid force-pushed the fix-inconsistent-location-requirements branch from fa3d8fa to f7a7f08 Compare January 12, 2021 21:29
@upodroid
Copy link
Contributor Author

It is done. This one has the GCE resources only.

@melinath
Copy link
Member

Thanks! /gcbrun

@modular-magician
Copy link
Collaborator

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=166693"

@modular-magician
Copy link
Collaborator

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 10 files changed, 49 insertions(+), 47 deletions(-))
Terraform Beta: Diff ( 10 files changed, 49 insertions(+), 47 deletions(-))

1 similar comment
@modular-magician
Copy link
Collaborator

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 10 files changed, 49 insertions(+), 47 deletions(-))
Terraform Beta: Diff ( 10 files changed, 49 insertions(+), 47 deletions(-))

@modular-magician
Copy link
Collaborator

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccInstanceTemplateDatasource_filter|TestAccInstanceTemplateDatasource_filter_mostRecent|TestAccDataSourceGoogleSubnetwork|TestAccDataSourceStorageBucketObjectContent_Basic|TestAccActiveDirectoryDomainTrust_activeDirectoryDomainTrustBasicExample|TestAccApigeeOrganization_apigeeOrganizationCloudFullTestExample|TestAccCloudRunService_cloudRunServiceSqlExample|TestAccComputePerInstanceConfig_update|TestAccContainerCluster_withPrivateClusterConfigMissingCidrBlock You can view the result here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=166696"

region: !ruby/object:Overrides::Terraform::PropertyOverride
required: false
ignore_read: true
default_from_api: true
Copy link
Member

@melinath melinath Jan 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect default_from_api to conflict with ignore_read - if the api provides a default but we ignore reading it, will it be available locally? This also seems to be different than the treatment in #4394 which moved things from ignore_read to default_from_api - why is there a difference?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure. It seems to work though.

@melinath
Copy link
Member

/gcbrun to see if the failures are transient.

@modular-magician
Copy link
Collaborator

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=167109"

@modular-magician
Copy link
Collaborator

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccInstanceTemplateDatasource_filter|TestAccInstanceTemplateDatasource_filter_mostRecent|TestAccDataSourceStorageBucketObjectContent_Basic|TestAccCloudRunServiceIamMemberGenerated|TestAccCloudRunServiceIamBindingGenerated|TestAccCloudRunServiceIamPolicyGenerated|TestAccActiveDirectoryDomainTrust_activeDirectoryDomainTrustBasicExample|TestAccApigeeOrganization_apigeeOrganizationCloudFullTestExample|TestAccApigeeOrganization_apigeeOrganizationCloudBasicTestExample|TestAccCloudRunService_cloudRunServiceBasicExample|TestAccCloudRunDomainMapping_cloudRunDomainMappingBasicExample|TestAccCloudRunService_cloudRunServiceNoauthExample|TestAccComputeHealthCheck_typeTransition|TestAccComputeHealthCheck_tcp_update|TestAccComputePerInstanceConfig_update|TestAccComputeRegionBackendService_basic|TestAccComputeRegionNetworkEndpointGroup_regionNetworkEndpointGroupCloudrunExample|TestAccContainerCluster_withPrivateClusterConfigMissingCidrBlock You can view the result here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=167119"

@modular-magician
Copy link
Collaborator

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 10 files changed, 49 insertions(+), 46 deletions(-))
Terraform Beta: Diff ( 10 files changed, 49 insertions(+), 46 deletions(-))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

google_compute_instance documentation reports the zone as being required
5 participants