Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: improve rows filters #751

Merged
merged 16 commits into from
Mar 23, 2023

Conversation

daniel-sanche
Copy link
Contributor

@daniel-sanche daniel-sanche commented Mar 16, 2023

This PR builds off of the existing RowFilters classes, witha couple additions:

  • adds to_dict methods, making it easier to work with dict representations of the objects
  • adds type annotations
  • adds str and repr implementations to make filters more human readable
  • _FilterCombination types are Sequences (implement len, iteration, indexing, etc)

I also pulled in the existing unit test file into the new client's test folder, with added tests for the new functionality

TODO:

  • changes docstrings to be consistent with other classes

Note: This is merging into a new v3 branch, not main. I plan on making all PRs related to this rewrite to v3 and then merging v3 into main when development is complete

@daniel-sanche daniel-sanche requested review from a team as code owners March 16, 2023 16:29
@product-auto-label product-auto-label bot added size: xl Pull request size is extra large. api: bigtable Issues related to the googleapis/python-bigtable API. labels Mar 16, 2023
@@ -35,6 +41,19 @@ class RowFilter(object):
This class is a do-nothing base class for all row filters.
"""

def to_pb(self) -> data_v2_pb2.RowFilter:
Copy link
Contributor Author

@daniel-sanche daniel-sanche Mar 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original implementation of RowFilter had to_pb functions for each filter, but I don't think we'll need it for the v3 client. I'm leaning towards making to_pb private, but I'm curious if there are any reasons for keeping it around

Copy link
Contributor

@Mariatta Mariatta Mar 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest keeping it public, because it is already available in v2. We don't know whether users are relying on this or not. I think it will make it easier for them to upgrade to v3 if they have access to similar functions. Otherwise we need to start raising deprecation warning saying this will be removed in v3.

@daniel-sanche daniel-sanche changed the title feat: add to_dict to rows filters feat: improve rows filters Mar 17, 2023
def to_dict(self) -> dict[str, Any]:
"""Converts the row filter to a dict representation."""
# unimplemented on base class
raise NotImplementedError
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of creating an ABC using abc.ABC for RowFilter so that to_dict can be one of its @abstractmethod

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think that makes sense. The only complication is that it requires us to make other filter subclasses abstract as well (_BoolFilter, _FilterCombination, _CellCountFilter), but that's probably good to prevent users from instantiating them directly anyway

I made the changes, let me know what you think

@daniel-sanche daniel-sanche merged commit 507da99 into googleapis:v3 Mar 23, 2023
@daniel-sanche daniel-sanche deleted the v3_read_rows_filters branch March 23, 2023 18:22
daniel-sanche added a commit that referenced this pull request Feb 5, 2024
* feat: add new v3.0.0 API skeleton (#745)

* feat: improve rows filters (#751)

* feat: read rows query model class (#752)

* feat: implement row and cell model classes (#753)

* feat: add pooled grpc transport (#748)

* feat: implement read_rows (#762)

* feat: implement mutate rows (#769)

* feat: literal value filter (#767)

* feat: row_exists and read_row (#778)

* feat: read_modify_write and check_and_mutate_row (#780)

* feat: sharded read rows (#766)

* feat: ping and warm with metadata (#810)

* feat: mutate rows batching (#770)

* chore: restructure module paths (#816)

* feat: improve timeout structure (#819)

* fix: api errors apply to all bulk mutations

* chore: reduce public api surface (#820)

* feat: improve error group tracebacks on < py11 (#825)

* feat: optimize read_rows (#852)

* chore: add user agent suffix (#842)

* feat: optimize retries (#854)

* feat: add test proxy (#836)

* chore(tests): add conformance tests to CI for v3 (#870)

* chore(tests): turn off fast fail for conformance tets (#882)

* feat: add TABLE_DEFAULTS enum for table method arguments (#880)

* fix: pass None for retry in gapic calls (#881)

* feat: replace internal dictionaries with protos in gapic calls (#875)

* chore: optimize gapic calls (#863)

* feat: expose retryable error codes to users (#879)

* chore: update api_core submodule (#897)

* chore: merge main into experimental_v3 (#900)

* chore: pin conformance tests to v0.0.2 (#903)

* fix: bulk mutation eventual success (#909)

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable Issues related to the googleapis/python-bigtable API. size: xl Pull request size is extra large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants