resource/aws_glue_dev_endpoint: Catch additional IAM eventual consistency error and fix VPC testing

[bflad]

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #16170
Reference: https://docs.aws.amazon.com/glue/latest/dg/set-up-vpc-dns.html

Release note for CHANGELOG:

* resource/aws_glue_dev_endpoint: Retry on additional IAM eventual consistency error on creation

The FAILED status in GovCloud was from this reason: "FailureReason":"Reverse dns resolution of ip 10.0.1.153 failed", because Glue requires VPCs to have DNS hostnames support enabled. Added the API failure reason to the waiter error messaging when available. Also fixed up some of the testing configuration to remove extraneous IAM handling.

Previously:

=== CONT  TestAccGlueDevEndpoint_SubnetID_SecurityGroupIDs
    resource_aws_glue_dev_endpoint_test.go:447: Step 1/2 error: Error running apply: 2020/11/16 22:57:14 [DEBUG] Using modified User-Agent: Terraform/0.12.29 HashiCorp-terraform-exec/0.10.0

        Error: error creating Glue Dev Endpoint: InvalidInputException: Failed to call ec2:DescribeSubnets: User: arn:aws:sts::056518428541:assumed-role/GlueJobExecutorCoralService-adminEC2Role-1URQ1VJPS7RAI/i-0b29c5b9f851d3f90 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::187416307283:role/AWSGlueServiceRole-tf-acc-test-1350868708791253218

--- FAIL: TestAccGlueDevEndpoint_SubnetID_SecurityGroupIDs (25.32s)

=== CONT  TestAccGlueDevEndpoint_SubnetID_SecurityGroupIDs
    resource_aws_glue_dev_endpoint_test.go:447: Step 1/2 error: Error running apply: 2020/11/12 13:49:01 [DEBUG] Using modified User-Agent: Terraform/0.12.29 HashiCorp-terraform-exec/0.10.0

        Error: error while waiting for Glue Dev Endpoint (tf-acc-test-831422846705845092) to become available: unexpected state 'FAILED', wanted target 'READY'. last error: %!s(<nil>)

--- FAIL: TestAccGlueDevEndpoint_SubnetID_SecurityGroupIDs (690.11s)

=== CONT  TestAccGlueDevEndpoint_SubnetID_SecurityGroupIDs
TestAccGlueDevEndpoint_SubnetID_SecurityGroupIDs: resource_aws_glue_dev_endpoint_test.go:447: Step 1/2 error: Error running pre-apply refresh: 2020/11/16 13:33:06 [DEBUG] Using modified User-Agent: Terraform/0.12.29 HashiCorp-terraform-exec/0.10.0
Error: multiple VPC Endpoint Services matched; use additional constraints to reduce matches to a single VPC Endpoint Service
--- FAIL: TestAccGlueDevEndpoint_SubnetID_SecurityGroupIDs (2.32s)

Output from acceptance testing in AWS Commercial:

--- PASS: TestAccGlueDevEndpoint_Arguments (631.30s)
--- PASS: TestAccGlueDevEndpoint_Basic (540.64s)
--- PASS: TestAccGlueDevEndpoint_disappears (529.64s)
--- PASS: TestAccGlueDevEndpoint_ExtraJarsS3Path (524.98s)
--- PASS: TestAccGlueDevEndpoint_ExtraPythonLibsS3Path (504.74s)
--- PASS: TestAccGlueDevEndpoint_GlueVersion (975.71s)
--- PASS: TestAccGlueDevEndpoint_NumberOfNodes (575.51s)
--- PASS: TestAccGlueDevEndpoint_NumberOfWorkers (1123.20s)
--- PASS: TestAccGlueDevEndpoint_PublicKey (684.84s)
--- PASS: TestAccGlueDevEndpoint_PublicKeys (716.43s)
--- PASS: TestAccGlueDevEndpoint_SecurityConfiguration (540.04s)
--- PASS: TestAccGlueDevEndpoint_SubnetID_SecurityGroupIDs (1055.05s)
--- PASS: TestAccGlueDevEndpoint_Tags (628.04s)
--- PASS: TestAccGlueDevEndpoint_WorkerType (1040.59s)

Output from acceptance testing in AWS GovCloud (US):

--- PASS: TestAccGlueDevEndpoint_Arguments (594.05s)
--- PASS: TestAccGlueDevEndpoint_Basic (580.48s)
--- PASS: TestAccGlueDevEndpoint_disappears (483.68s)
--- PASS: TestAccGlueDevEndpoint_ExtraJarsS3Path (528.97s)
--- PASS: TestAccGlueDevEndpoint_ExtraPythonLibsS3Path (668.35s)
--- PASS: TestAccGlueDevEndpoint_GlueVersion (800.76s)
--- PASS: TestAccGlueDevEndpoint_NumberOfNodes (894.37s)
--- PASS: TestAccGlueDevEndpoint_NumberOfWorkers (828.21s)
--- PASS: TestAccGlueDevEndpoint_PublicKey (490.94s)
--- PASS: TestAccGlueDevEndpoint_PublicKeys (492.38s)
--- PASS: TestAccGlueDevEndpoint_SubnetID_SecurityGroupIDs (998.87s)
--- PASS: TestAccGlueDevEndpoint_Tags (483.63s)
--- PASS: TestAccGlueDevEndpoint_WorkerType (799.62s)

[DrFaust92]

LGTM

--- PASS: TestAccGlueDevEndpoint_disappears (489.11s)
--- PASS: TestAccGlueDevEndpoint_SecurityConfiguration (518.10s)
--- PASS: TestAccGlueDevEndpoint_Basic (564.85s)
--- PASS: TestAccGlueDevEndpoint_ExtraPythonLibsS3Path (640.74s)
--- PASS: TestAccGlueDevEndpoint_Tags (640.85s)
--- PASS: TestAccGlueDevEndpoint_ExtraJarsS3Path (649.25s)
--- PASS: TestAccGlueDevEndpoint_Arguments (657.62s)
--- PASS: TestAccGlueDevEndpoint_PublicKeys (659.56s)
--- PASS: TestAccGlueDevEndpoint_NumberOfNodes (693.43s)
--- PASS: TestAccGlueDevEndpoint_PublicKey (717.47s)
--- PASS: TestAccGlueDevEndpoint_NumberOfWorkers (770.07s)
--- PASS: TestAccGlueDevEndpoint_WorkerType (1099.08s)
--- PASS: TestAccGlueDevEndpoint_SubnetID_SecurityGroupIDs (1150.93s)
--- PASS: TestAccGlueDevEndpoint_GlueVersion (1240.63s)

[ghost]

This has been released in version 3.16.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!