google_composer_env with composer-2 image both requires and disallows "zone" argument

[sceller]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

0.13.6

Affected Resource(s)

• google_composer_env

Terraform Configuration Files

resource "google_composer_environment" "composer_env" {
  provider = google-beta
  project  = var.project_id
  name     = var.composer_env_name
  region   = var.region
  labels   = var.labels

  config {
    software_config {
      image_version = "composer-2.3.5-airflow-2.5.3"
    }

    node_config {
      # zone = var.zone
      network = var.network
      subnetwork = var.subnetwork
      service_account = var.composer_service_account
    }
  }
}

Expected Behavior

A Composer V2 environment should've been created

Actual Behavior

If the "zone = " line is commented out, the following error is thrown during terraform plan:

Error: Missing required argument
    on ../../../../modules/cloud_composer_v2/main.tf line 18, in resource "google_composer_environment" "composer_env":
    18:     node_config {
The argument "zone" is required, but no definition was found.

If the "zone = " line is not commented out, the following is thrown during terraform apply:

Error: googleapi: Error 400: Found 1 problem:
    1) Configuring node location is not supported for Cloud Composer environments in versions 2.0.0 and newer.

Steps to Reproduce

1. terraform plan
2. terraform apply

Important Factoids

Building on GCP Cloud Build as part of a larger project.

References

This issue was previously filed in Jan 2022 but has received no comments: #10848

b/301065930

[edwardmedia]

b/295215918

[PeWu]

I wasn't able to reproduce this issue with a newer Terraform version. I was able to create both v1 and v2 Composer environments without the zone being specified.

Please try using a new version of Terraform.

[sceller]

Which version are you using?

…

On Tue, Aug 29, 2023 at 7:57 AM Przemek Więch ***@***.***> wrote: I wasn't able to reproduce this issue with a newer Terraform version. I was able to create both v1 and v2 Composer environments without the zone being specified. Please try using a new version of Terraform. — Reply to this email directly, view it on GitHub <#15450 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEYTBZXTIYXDC5J3ITH5ITTXXXKMBANCNFSM6AAAAAA3KP4XKY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[PeWu]

terraform v1.2.7
google-beta v4.79.0

[PeWu]

Closing after inactivity on the issue.
The issue was not reproducible in the latest version of Terraform.

Please reopen if needed.

[melinath]

I am able to reproduce this API error using Terraform by providing the zone field. However, zone was made optional + computed in GoogleCloudPlatform/magic-modules#5320, which means zone doesn't need to be supplied. If I do not supply zone then terraform apply succeeds.

Here is the configuration I used:

resource "google_composer_environment" "composer_env" {
  provider = google-beta
  name     = "tf-test-composer-env"

  config {
    software_config {
      image_version = "composer-2.3.5-airflow-2.5.3"
    }

    node_config {
      # zone = "us-central1-a"
      network    = google_compute_network.test.id
      subnetwork = google_compute_subnetwork.test.id

      service_account = google_service_account.test.name
    }
  }
}


resource "google_compute_network" "test" {
  name                    = "composer-test-network"
  auto_create_subnetworks = false
}

resource "google_compute_subnetwork" "test" {
  name          = "composer-test-subnetwork"
  ip_cidr_range = "10.2.0.0/16"
  region        = "us-central1"
  network       = google_compute_network.test.id
}

resource "google_service_account" "test" {
  account_id   = "composer-env-account"
  display_name = "Test Service Account for Composer Environment"
}

resource "google_project_iam_member" "composer-worker" {
  project = "PROJECT_ID"
  role   = "roles/composer.worker"
  member = "serviceAccount:${google_service_account.test.email}"
}

resource "google_service_account_iam_member" "custom_service_account" {
  provider = google-beta
  service_account_id = google_service_account.test.id
  role = "roles/composer.ServiceAgentV2Ext"
  member = "serviceAccount:service-PROJECT_NUMBER@cloudcomposer-accounts.iam.gserviceaccount.com"
}

[PeWu]

@melinath This behavior is correct. The zone option only makes sense for Composer 1 environments which operate zonal GKE clusters. Composer 2 environments operate regional GKE clusters.

[melinath]

@PeWu thanks for the additional context!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.