Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need to support dataset.list and jobs.list API new field feature #16987

Open
katamotokosuke opened this issue Dec 19, 2023 · 0 comments
Open

Comments

@katamotokosuke
Copy link

Hello,

I have received the following email from Google:

Hello Google BigQuery Customer,

We are writing to let you know that a new field will be added to the response of the datasets.list API and jobs.list API calls in BigQuery starting February 15, 2024.

What do you need to know?
Currently, in the rare event that one or more GCP regions become unreachable for a prolonged period, datasets.list and jobs.list API calls will fail if the response should include data from the unreachable region.

Starting February 15, 2024, these calls will succeed, but with the additional field `unreachable` included to indicate the locations where data might have been omitted from the datasets.list and jobs.list responses due to the prolonged unreachability of the entire region or multi-region.

Under normal circumstances, this field will be empty. However, if you depend on retrieving a complete list of all your datasets and jobs from the datasets.list and jobs.list APIs, you should start considering the values of this new field when present.

What do I need to do?
We recommend the following steps to make the most of the new field provided in the output:

1. Review the regions from which you are making datasets.list and jobs.list API calls.
2. After February 15, 2024, review the new "unreachable" field in the datasets.list and jobs.list API responses.
3. Examine the contents of this field to identify any unreachable locations. If your API integration with BigQuery relies on obtaining a full list of datasets/jobs from all regions/multi-regions where relevant data exists, you may need to adjust your integration based on specific scenarios.
4. Depending on the situation, you may choose to fail the workflow, issue a warning, or continue without warning if the `unreachable` field is populated.

These recommended actions will help you ensure that you are receiving a complete list of datasets from all regions.

Thank you for choosing Google Big Query.

It appears that a new unreachable field will be added to the jobs.list and datasets.list APIs. This change will modify the interfaces of these two APIs. When can I expect this SDK to be updated to reflect this change?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant