Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add PyCapsule support for Arrow import and export #825

Merged
merged 19 commits into from
Aug 30, 2024
Merged

Add PyCapsule support for Arrow import and export #825

merged 19 commits into from
Aug 30, 2024

Commits on Aug 28, 2024

  1. Support Arrow PyCapsule for reading arrow tables and for exporting da… 

    …taframes
    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    4a93356 View commit details
    Browse the repository at this point in the history

  2. Improve flow control on receiving record batches from arrow pycapsule

    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    d61621c View commit details
    Browse the repository at this point in the history

  3. Update code comment

    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    a813dc9 View commit details
    Browse the repository at this point in the history

  4. Add function to project record batch into desired schema

    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    533c915 View commit details
    Browse the repository at this point in the history

  5. Validate the pycapsule and get the pointer when successful

    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    d94885c View commit details
    Browse the repository at this point in the history

  6. Add unit test for export via pycapsule

    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    932c8c2 View commit details
    Browse the repository at this point in the history

  7. Readability

    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    a404a68 View commit details
    Browse the repository at this point in the history

  8. Remove nanoarrow so we don't add another dependency that isn't gainin… 

    …g much over pyarrow
    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    1ed4632 View commit details
    Browse the repository at this point in the history

  9. Remove unused import

    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    d1fe4df View commit details
    Browse the repository at this point in the history

  10. Add unit test to check for reading from record batch streams

    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    61473df View commit details
    Browse the repository at this point in the history

  11. Update unit test to try a variety of arrow data sources on import

    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    0bbb293 View commit details
    Browse the repository at this point in the history

  12. Simplify dataframe creation in unit test

    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    2586b3f View commit details
    Browse the repository at this point in the history

  13. Update user documenation to include usage of import and exporting arr… 

    …ow objects
    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    43a869b View commit details
    Browse the repository at this point in the history

  14. Rename from_arrow_table to from_arrow since we now support multiple t… 

    …ypes of input
    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    06e16fb View commit details
    Browse the repository at this point in the history

  15. Add admonition box to warn user

    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    afe6420 View commit details
    Browse the repository at this point in the history

  16. Mark from_arrow_table as deprecated

    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    a5b4e39 View commit details
    Browse the repository at this point in the history

  17. Clean up error handling

    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    7cc807d View commit details
    Browse the repository at this point in the history

  18. Marking was not required 

    Co-authored-by: Michael J Ward <Michael-J-Ward@users.noreply.github.com>
    @timsaucer @Michael-J-Ward
    timsaucer and Michael-J-Ward committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    babeb77 View commit details
    Browse the repository at this point in the history

  19. Apply linting after rebase

    @timsaucer
    timsaucer committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    955d7d5 View commit details
    Browse the repository at this point in the history