-
Notifications
You must be signed in to change notification settings - Fork 58
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: implement mutate rows #769
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm after comments are addressed
core_exceptions.DeadlineExceeded, | ||
core_exceptions.ServiceUnavailable, | ||
_MutateRowsIncomplete, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit. add a comment explaining the source of the errors
# RPC level errors
core_exceptions.DeadlineExceeded,
core_exceptions.ServiceUnavailable,
# Entry level errors
_MutateRowsIncomplete,
async for result_list in result_generator: | ||
for result in result_list.entries: | ||
# convert sub-request index to global index | ||
orig_idx = active_request_indices.pop(result.index) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit. pop is doing extra work here. Why not just a lookup?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When the exception is caught on line 181, we need to know which mutation entries were still pending at the time of the exception, and only add those to the retry queue. So active_request_indices needs to drop indices as they are finalized
Although looking again, I'll move the pop to the end of the block, since this could turn into a tricky bug if this function becomes more complex over time, and an exception ends up being raised in between this line and self._handle_entry_error
# add this exception to list for each mutation that wasn't | ||
# already handled |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
and update remaining_indices
if result.status.code == 0: | ||
continue | ||
else: | ||
self._handle_entry_error(orig_idx, entry_error) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add a comment that this will update remaining_indices
* feat: add new v3.0.0 API skeleton (#745) * feat: improve rows filters (#751) * feat: read rows query model class (#752) * feat: implement row and cell model classes (#753) * feat: add pooled grpc transport (#748) * feat: implement read_rows (#762) * feat: implement mutate rows (#769) * feat: literal value filter (#767) * feat: row_exists and read_row (#778) * feat: read_modify_write and check_and_mutate_row (#780) * feat: sharded read rows (#766) * feat: ping and warm with metadata (#810) * feat: mutate rows batching (#770) * chore: restructure module paths (#816) * feat: improve timeout structure (#819) * fix: api errors apply to all bulk mutations * chore: reduce public api surface (#820) * feat: improve error group tracebacks on < py11 (#825) * feat: optimize read_rows (#852) * chore: add user agent suffix (#842) * feat: optimize retries (#854) * feat: add test proxy (#836) * chore(tests): add conformance tests to CI for v3 (#870) * chore(tests): turn off fast fail for conformance tets (#882) * feat: add TABLE_DEFAULTS enum for table method arguments (#880) * fix: pass None for retry in gapic calls (#881) * feat: replace internal dictionaries with protos in gapic calls (#875) * chore: optimize gapic calls (#863) * feat: expose retryable error codes to users (#879) * chore: update api_core submodule (#897) * chore: merge main into experimental_v3 (#900) * chore: pin conformance tests to v0.0.2 (#903) * fix: bulk mutation eventual success (#909) --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Implements and tests mutate_row and bulk_mutate_rows rpc calls, along with associated models and helpers
These functions will both return nothing on success, but will raise an exception with details about the failed mutation or group of mutations on error, after retries are attempted.
Errors are exposed as an ExceptionGroup in Python3.11+, allowing users to easily digest a group of errors that are raised together, in this case for multiple mutations, with multiple retries per mutation. We use a backwards compatible implementation to expose similar functionality in Python3.10 and lower
Note that mutation batching will come in a separate PR, as it builds off of the functions added here
todo: