Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: warn when read_gbq / read_gbq_table uses the snapshot time cache #441

Merged
merged 2 commits into from
Mar 15, 2024

Conversation

tswast
Copy link
Collaborator

@tswast tswast commented Mar 14, 2024

I've verified that this warning appears from the command line as well as in notebooks.

warning: reading cached table

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • [N/A] Appropriate docs were updated (if necessary)

Fixes internal issue 329545805 where read_gbq gives an old version of the table unexpectedly.
🦕

@tswast tswast requested review from a team as code owners March 14, 2024 18:10
@tswast tswast requested a review from Genesis929 March 14, 2024 18:10
@product-auto-label product-auto-label bot added size: s Pull request size is small. api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. labels Mar 14, 2024
@tswast tswast requested review from milkshakeiii and removed request for Genesis929 March 14, 2024 18:14
@tswast
Copy link
Collaborator Author

tswast commented Mar 14, 2024

For context, this is a follow-up to #229 which addressed internal issue 312493159 where we would often get

BadRequest: 400 Conflicting 'FOR SYSTEM_TIME AS OF' expressions found for table ... If a 'FOR SYSTEM_TIME AS OF' expression is used, all references of a table should use the same TIMESTAMP value.

errors when folks called read_gbq on the same table more than once and combined the resulting dataframe with previous versions of the dataframe. This can possibly happen because they wanted to rerun the cells in the notebook.

@tswast tswast merged commit e16a8c0 into main Mar 15, 2024
14 of 15 checks passed
@tswast tswast deleted the b329545805-read_gbq-stale branch March 15, 2024 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. size: s Pull request size is small.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants