[Metricbeat] allow partial region and zone in googlecloud module config

[kaiyan-sheng]

What does this PR do?

This PR is to enable partial region and partial zone in googlecloud module config. User can choose to use wildcard * or not. For example:

- module: googlecloud
  metricsets:
    - compute
  region: "us-west"
  project_id: "elastic-observability"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 60s

This config will enable data collection from all regions start with us-west, which includes us-west1, us-west2, us-west3 and us-west4.

- module: googlecloud
  metricsets:
    - compute
  region: "us-*"
  project_id: "elastic-observability"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 60s

Similarly this config will enable data collection from all regions start with us-, which includes all regions starts with us-west, us-east and us-central.

Why is it important?

This will allow users to not copy paste region/zone names when they want to monitor several regions/zones. If user wants to monitor all regions in both US and Europe, they still need to copy paste but that's very minimum. For example:

- module: googlecloud
  metricsets:
    - compute
  region: "us-*"
  project_id: "elastic-observability"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 60s
- module: googlecloud
  metricsets:
    - compute
  region: "europe-*"
  project_id: "elastic-observability"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 60s

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

There are two parts need to be tested. One for regions and another for zones.

• Test for regions:
  Enable googlecloud with this config and you should see metrics collection from all regions start with us-.

- module: googlecloud
  metricsets:
    - compute
  region: "us-"
  project_id: "elastic-observability"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 60s

• Test for zones:
  Enable googlecloud with this config and you should see metrics collection from all zones start with europe-west3-, which includes europe-west3-a, europe-west3-b and europe-west3-c.

- module: googlecloud
  metricsets:
    - compute
  zone: "europe-west3-"
  project_id: "elastic-observability"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 60s

Related issues

• Relates [Metricbeat] Allow to specify more than one regions in googlecloud module #17835

[elasticmachine]

Pinging @elastic/integrations-platforms (Team:Platforms)

[blakerouse]

Would it be better to only enable this behavior with a wildcard?

us-* instead of us-

[kaiyan-sheng]

Good point! I will add that! Thanks @blakerouse !!

[kaiyan-sheng]

@blakerouse I added * as an option. Let me know what you think! Thanks!

[blakerouse]

I see that you allowed with and without the * that is nice. Looks like mage update might need to be called before merging. Otherwise looks good.

[kaiyan-sheng]

Yeah, gcp user who uses their API a lot will probably put region/zone without * but for others, wildcard might make more sense here. Thanks for the suggestion.