Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feat: Threaded MutationsBatcher #722

Merged
merged 24 commits into from
May 10, 2023
Merged

Feat: Threaded MutationsBatcher #722

merged 24 commits into from
May 10, 2023

Commits on Apr 5, 2023

  1. Feat: Threaded MutationsBatcher 

    - Batch mutations in a thread to allow concurrent batching
- Flush the batch every second
    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    3010cd5 View commit details
    Browse the repository at this point in the history

  2. Make _MutationsBatchQueue private

    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    dbf80c3 View commit details
    Browse the repository at this point in the history

  3. Fix the tests.

    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    9ef3ec5 View commit details
    Browse the repository at this point in the history

  4. Unit test adjustments

    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    148d1c0 View commit details
    Browse the repository at this point in the history

  5. Add flush_interval. 

    Raise error when mutating when batcher is already closed.
    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    f312659 View commit details
    Browse the repository at this point in the history

  6. Add test for the batcher time interval

    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    e657321 View commit details
    Browse the repository at this point in the history

  7. Adjust parameters

    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    00e03e5 View commit details
    Browse the repository at this point in the history

  8. Raise errors from mutations.

    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    649cd9b View commit details
    Browse the repository at this point in the history

  9. test coverage

    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    1a5ff05 View commit details
    Browse the repository at this point in the history

  10. Use from_grpc_status 

    Add comments
    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    70f1cc8 View commit details
    Browse the repository at this point in the history

  11. Update MutationsBatchError

    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    fd06c2f View commit details
    Browse the repository at this point in the history

  12. Add flow control to control inflight requests

    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    b912792 View commit details
    Browse the repository at this point in the history

  13. Improve test coverage

    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    cc334ec View commit details
    Browse the repository at this point in the history

  14. Submit the batch when queue's empty

    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    177414f View commit details
    Browse the repository at this point in the history

  15. Update google/cloud/bigtable/batcher.py 

    Co-authored-by: Mattie Fu <mattiefu@google.com>
    @Mariatta @mutianf
    Mariatta and mutianf committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    d907180 View commit details
    Browse the repository at this point in the history

  16. Changes based on PR review 

    - Remove unneeded error
- Make some functions internal
    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    d0a6ba3 View commit details
    Browse the repository at this point in the history

  17. Update google/cloud/bigtable/batcher.py 

    Co-authored-by: Mattie Fu <mattiefu@google.com>
    @Mariatta @mutianf
    Mariatta and mutianf committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    06bc2c1 View commit details
    Browse the repository at this point in the history

  18. Update google/cloud/bigtable/batcher.py 

    Co-authored-by: Mattie Fu <mattiefu@google.com>
    @Mariatta @mutianf
    Mariatta and mutianf committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    c0584b5 View commit details
    Browse the repository at this point in the history

  19. Changes based on PR review 

    - Remove debugging variable
- Update variable names
    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    9899e45 View commit details
    Browse the repository at this point in the history

  20. Docstring updates

    @Mariatta
    Mariatta committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    51b063b View commit details
    Browse the repository at this point in the history

Commits on Apr 6, 2023

  1. Remove unused variable, is_open

    @Mariatta
    Mariatta committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    1e42c1b View commit details
    Browse the repository at this point in the history

Commits on May 5, 2023

  1. Adjust the logic for when the batch is larger than available resources. 

    Allow the batch to go through.
    @Mariatta
    Mariatta committed May 5, 2023
    Configuration menu
    Copy the full SHA
    c1c50ea View commit details
    Browse the repository at this point in the history

Commits on May 9, 2023

  1. Make _flush_async private

    @Mariatta
    Mariatta committed May 9, 2023
    Configuration menu
    Copy the full SHA
    414fe87 View commit details
    Browse the repository at this point in the history

  2. Update test case for flush async

    @Mariatta
    Mariatta committed May 9, 2023
    Configuration menu
    Copy the full SHA
    c2943ad View commit details
    Browse the repository at this point in the history