Fix sql_database_instance creation to remove root user earlier

[roaks3]

b/254877908

This fixes a bug where a root user without a password would sometimes remain on an sql_database_instance after creation, creating a security risk. The logic in the provider is meant to remove this root user at the end of the creation function, but if there is a failure before then, the instance is still created but the root user remains.

The fix here simply performs the root user removal earlier, so that it is done as soon as it is not longer needed. This eliminates a subset of failure modes that would have left the root user on the instance. I believe this is the best we can do for now, short of more elaborate retry logic.

For testing, it was very difficult to come up with a test that would verify this problem has been resolved, because triggering arbitrary errors is nontrivial, and our test harness does not offer a clean way to check the state of the environment after an error occurs. I've come up with work-arounds to produce a test that I believe is adequate here.

If this PR is for Terraform, I acknowledge that I have:

• Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
• Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
• Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
• Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
• Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

sql: fixed `sql_database_instance` leaking root users

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 159 insertions(+), 27 deletions(-))
Terraform Beta: Diff ( 2 files changed, 159 insertions(+), 27 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2330
Passed tests 2083
Skipped tests: 245
Failed tests: 2

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccLoggingBucketConfigProject_cmekSettings|TestAccFirebaserulesRelease_BasicRelease

[modular-magician]

Tests passed during RECORDING mode:
TestAccLoggingBucketConfigProject_cmekSettings[Debug log]
TestAccFirebaserulesRelease_BasicRelease[Debug log]

All tests passed
View the build log or the debug log for each test

[rileykarson]

@roaks3: You don't get a reviewer assigned by the Magician anymore, btw! You can assign @GoogleCloudPlatform/terraform-team to roll one.

[shuyama1]

Only a small nitpick. The code looks good assuming test passes: https://ci-oss.hashicorp.engineering/buildConfiguration/GoogleCloudBeta_ProviderGoogleCloudBetaMmUpstream/359834

[shuyama1]

Only nit: spacing is bit off here -- should be spaces instead of tabs

[roaks3]

Whoops, thank you!

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 159 insertions(+), 27 deletions(-))
Terraform Beta: Diff ( 2 files changed, 159 insertions(+), 27 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2341
Passed tests 2094
Skipped tests: 245
Failed tests: 2

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccFirebaserulesRelease_BasicRelease|TestAccLoggingBucketConfigProject_cmekSettings

[modular-magician]

Tests passed during RECORDING mode:
TestAccFirebaserulesRelease_BasicRelease[Debug log]
TestAccLoggingBucketConfigProject_cmekSettings[Debug log]

All tests passed
View the build log or the debug log for each test

[shuyama1]

https://ci-oss.hashicorp.engineering/buildConfiguration/GoogleCloudBeta_ProviderGoogleCloudBetaMmUpstream/359847
@roaks3 Test failed on Teamcity. Would you mind taking a look? Were you able to get it pass locally? Change itself looks good to me.

[roaks3]

Yea, this test passes locally, but we are hitting an issue in CI with a number of these related tests (like TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRangeReplica and TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRangeClone). At first glance, it appears to be having trouble creating the network connection that is a prerequisite of the test.

[shuyama1]

LGTM!

[roaks3]

I did some investigating on this test failure and captured my work in hashicorp/terraform-provider-google#12902. I will hold off on merging this momentarily to see if we can resolve the root issue before adding to it with this new test, but if it looks like a long-term fix I will go ahead and move this through.

[roaks3]

It looks like the other issue could take some time, so merging this change now.