Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bootstrap storage pool in tests #11598

Merged
merged 5 commits into from
Sep 6, 2024

Conversation

shuyama1
Copy link
Member

@shuyama1 shuyama1 commented Aug 30, 2024

Given the limitations of storage pools per project, we'll need to bootstrap the storage pool resource in our testing project so it can be shared and reused across multiple tests to avoid hitting the quota limit.

This should help hashicorp/terraform-provider-google#19199 and #11391

Release Note Template for Downstream PRs (will be copied)


@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 1 file changed, 72 insertions(+))
google-beta provider: Diff ( 1 file changed, 72 insertions(+))

@shuyama1 shuyama1 force-pushed the bootstrap_storage_pool branch from 5ed6707 to dc19af4 Compare August 30, 2024 23:10
@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 1 file changed, 72 insertions(+))
google-beta provider: Diff ( 1 file changed, 72 insertions(+))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3748
Passed tests: 3339
Skipped tests: 409
Affected tests: 0

Click here to see the affected service packages

All service packages are affected

$\textcolor{red}{\textsf{Errors occurred during REPLAYING mode. Please fix them to complete your PR.}}$

View the build log

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3788
Passed tests: 3376
Skipped tests: 411
Affected tests: 1

Click here to see the affected service packages

All service packages are affected

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccComputeInstanceTemplate_withNamePrefix

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccComputeInstanceTemplate_withNamePrefix[Debug log]
$\textcolor{red}{\textsf{Tests failed when rerunning REPLAYING mode:}}$
TestAccComputeInstanceTemplate_withNamePrefix[Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.


$\textcolor{green}{\textsf{All tests passed!}}$

View the build log or the debug log for each test

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 76 insertions(+), 22 deletions(-))
google-beta provider: Diff ( 3 files changed, 76 insertions(+), 22 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3791
Passed tests: 3376
Skipped tests: 409
Affected tests: 6

Click here to see the affected service packages

All service packages are affected

Action taken

Found 6 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccBackupDRBackupVault_fullUpdate
  • TestAccComputeDisk_storagePoolSpecified
  • TestAccComputeInstanceTemplate_withNamePrefix
  • TestAccComputeInstance_bootDisk_storagePoolSpecified
  • TestAccNetappActiveDirectory_activeDirectory_FullUpdate
  • TestAccNetappBackup_NetappBackupFull_update

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccBackupDRBackupVault_fullUpdate[Debug log]
TestAccComputeDisk_storagePoolSpecified[Debug log]
TestAccComputeInstanceTemplate_withNamePrefix[Debug log]
TestAccNetappActiveDirectory_activeDirectory_FullUpdate[Debug log]
TestAccNetappBackup_NetappBackupFull_update[Debug log]
$\textcolor{red}{\textsf{Tests failed when rerunning REPLAYING mode:}}$
TestAccComputeInstanceTemplate_withNamePrefix[Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.


$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccComputeInstance_bootDisk_storagePoolSpecified[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 76 insertions(+), 126 deletions(-))
google-beta provider: Diff ( 3 files changed, 76 insertions(+), 126 deletions(-))

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 76 insertions(+), 131 deletions(-))
google-beta provider: Diff ( 3 files changed, 76 insertions(+), 131 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 2949
Passed tests: 2615
Skipped tests: 334
Affected tests: 0

Click here to see the affected service packages

All service packages are affected

$\textcolor{red}{\textsf{Errors occurred during REPLAYING mode. Please fix them to complete your PR.}}$

View the build log

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3840
Passed tests: 3428
Skipped tests: 409
Affected tests: 3

Click here to see the affected service packages

All service packages are affected

Action taken

Found 3 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccComputeInstanceTemplate_withNamePrefix
  • TestAccComputeInstance_bootDisk_storagePoolSpecified
  • TestAccProjectIamPolicy_invalidMembers

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccComputeInstanceTemplate_withNamePrefix[Debug log]
TestAccComputeInstance_bootDisk_storagePoolSpecified[Debug log]
TestAccProjectIamPolicy_invalidMembers[Debug log]
$\textcolor{red}{\textsf{Tests failed when rerunning REPLAYING mode:}}$
TestAccComputeInstanceTemplate_withNamePrefix[Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.


$\textcolor{green}{\textsf{All tests passed!}}$

View the build log or the debug log for each test

@shuyama1 shuyama1 requested a review from melinath September 3, 2024 23:14
@shuyama1
Copy link
Member Author

shuyama1 commented Sep 3, 2024

failing test TestAccComputeInstanceTemplate_withNamePrefix is not related to this PR

@shuyama1
Copy link
Member Author

shuyama1 commented Sep 5, 2024

/gcbrun

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 76 insertions(+), 131 deletions(-))
google-beta provider: Diff ( 3 files changed, 76 insertions(+), 131 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3955
Passed tests: 3543
Skipped tests: 410
Affected tests: 2

Click here to see the affected service packages

All service packages are affected

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccBackupDRBackupVault_fullUpdate
  • TestAccComputeDisk_storagePoolSpecified

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccBackupDRBackupVault_fullUpdate[Debug log]
TestAccComputeDisk_storagePoolSpecified[Debug log]

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$


$\textcolor{green}{\textsf{All tests passed!}}$

View the build log or the debug log for each test

Comment on lines 1112 to 1116
func BootstrapComputeStoragePool(t *testing.T, storagePoolName, storagePoolType string) string {
projectID := envvar.GetTestProjectFromEnv()
zone := envvar.GetTestZoneFromEnv()

storagePoolName = SharedStoragePoolPrefix + storagePoolName
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would make sense to include the storage pool type here - or only allow storagePoolType and remove storagePoolName if we only need one of each type per region.

Suggested change
func BootstrapComputeStoragePool(t *testing.T, storagePoolName, storagePoolType string) string {
projectID := envvar.GetTestProjectFromEnv()
zone := envvar.GetTestZoneFromEnv()
storagePoolName = SharedStoragePoolPrefix + storagePoolName
func BootstrapComputeStoragePool(t *testing.T, storagePoolType string) string {
projectID := envvar.GetTestProjectFromEnv()
zone := envvar.GetTestZoneFromEnv()
storagePoolName := SharedStoragePoolPrefix + storagePoolType

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll update it to storagePoolName = SharedStoragePoolPrefix + storagePoolType + storagePoolName to make it easier to identify the type by reading the resource name. I'll also keep the storagePoolName, as we might need more than one storage pool per type as test cases expand.

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 76 insertions(+), 131 deletions(-))
google-beta provider: Diff ( 3 files changed, 76 insertions(+), 131 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 3959
Passed tests: 3548
Skipped tests: 409
Affected tests: 2

Click here to see the affected service packages

All service packages are affected

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccComputeDisk_storagePoolSpecified
  • TestAccComputeInstance_bootDisk_storagePoolSpecified

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccComputeDisk_storagePoolSpecified[Debug log]
TestAccComputeInstance_bootDisk_storagePoolSpecified[Debug log]

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$


$\textcolor{green}{\textsf{All tests passed!}}$

View the build log or the debug log for each test

@shuyama1 shuyama1 merged commit ea61487 into GoogleCloudPlatform:main Sep 6, 2024
11 checks passed
@shuyama1 shuyama1 deleted the bootstrap_storage_pool branch September 6, 2024 16:05
entertvl pushed a commit to entertvl/magic-modules that referenced this pull request Sep 9, 2024
iyabchen pushed a commit to iyabchen/magic-modules that referenced this pull request Sep 14, 2024
abd-goog pushed a commit to abd-goog/magic-modules that referenced this pull request Sep 23, 2024
wyardley added a commit to wyardley/magic-modules that referenced this pull request Sep 26, 2024
Tests such as
`TestAccContainerNodePool_withDiskMachineAndStoragePoolUpdate` seem to
be failing, possibly due to being promoted from beta to v1

Rather than update the regex in extractSP, use
`tpgresource.GetRelativePath()` and return that from the bootstrap
method, as mentioned here:
GoogleCloudPlatform#11391 (comment)

See GoogleCloudPlatform#11391 and GoogleCloudPlatform#11598
niharika-98 pushed a commit to niharika-98/magic-modules that referenced this pull request Oct 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants