Make 'region' Argument Optional in Dataflow Configuration

[fribeiro1]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v1.2.4
on linux_amd64

• provider registry.terraform.io/hashicorp/google v4.24.0

Affected Resource(s)

• google_dataflow_job

Terraform Configuration Files

resource "google_dataflow_job" "samplejob" {
  name = "samplejob"
  template_gcs_path = "gs://dataflow-templates-us-central1/latest/Word_Count"
  temp_gcs_location = "gs://samplebucket-123/temp"
  parameters = {
    inputFile = "gs://samplebucket-123/sampleobject"
    output = "gs://samplebucket-123/counts"
  }
}

Debug Output

https://gist.github.com/fribeiro1/195629409cb26f51c133d34f67e66584

Panic Output

N/A.

Expected Behavior

The job is created.

Actual Behavior

The provider requires the optional 'region' argument to create the job.

Steps to Reproduce

1. terraform apply

Important Factoids

N/A.

References

• https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/dataflow_job#region

[edwardmedia]

@fribeiro1 do you set the region in your provider? It is optional if you have the region in the provider. This is common behavior for most resources.

[fribeiro1]

@edwardmedia Thanks for your quick reply. "region" is optional in the API and should also be optional in the provider.

[edwardmedia]

@fribeiro1 could you share the api doc that shows the region is optional?

[fribeiro1]

@edwardmedia The expected behavior when you don't provide a location is described at https://cloud.google.com/dataflow/docs/reference/rest/v1b3/projects.locations.jobs/create.

For example, the CLI, Java, and Python client libraries already implement it. I published samples at https://simplesassim.wordpress.com/2022/07/12/how-to-create-a-job-in-google-dataflow/.

[edwardmedia]

@fribeiro1 don't you need to provide {location} in the below endpoint?

POST https://dataflow.googleapis.com/v1b3/projects/{projectId}/locations/{location}/jobs

https://cloud.google.com/dataflow/docs/reference/rest/v1b3/projects.locations.jobs/create

Is this what you referred to when {location} is not provided? It is pretty clear not recommended.

Using projects.jobs.create is not recommended, as your job will always start in us-central1.

[fribeiro1]

@edwardmedia No, you don't. I know it is odd. Still, I think the provider should align with the other tools.

[edwardmedia]

It is pretty clear other tools may either provide a default {location} or choose the endpoint that is not recommended. It is not common for the provider to provide a default {region} for a resource. Based on that, I am closing this issue.

[fribeiro1]

@edwardmedia I'm afraid I have to disagree with your evaluation, but I appreciate your attention to the issue. Regards.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.