Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arrow: add support for null vectors #10953

Open
wants to merge 45 commits into
base: main
Choose a base branch
from

Commits on May 8, 2024

  1. apache#10275 - fix NullPointerException

    Fix NullPointerException when trying to add the vector's class name to the message for an UnsupportedOperationException
    sl255051 committed May 8, 2024
    Configuration menu
    Copy the full SHA
    ac6440a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    becf6f7 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    4e2cb86 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. Configuration menu
    Copy the full SHA
    1193d02 View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. Add new unit test

    This test more closely follows the reproduction steps described in issue apache#10275
    sl255051 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    12bc3de View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d8f3e13 View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2024

  1. Add comments to unit test

    sl255051 committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    bb4e010 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    6e7a1aa View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2024

  1. Update arrow/src/test/java/org/apache/iceberg/arrow/vectorized/ArrowR…

    …eaderTest.java
    
    Co-authored-by: Eduard Tudenhoefner <etudenhoefner@gmail.com>
    slessard and nastra committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    28451a5 View commit details
    Browse the repository at this point in the history
  2. Update arrow/src/test/java/org/apache/iceberg/arrow/vectorized/ArrowR…

    …eaderTest.java
    
    Co-authored-by: Eduard Tudenhoefner <etudenhoefner@gmail.com>
    slessard and nastra committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    24a9932 View commit details
    Browse the repository at this point in the history
  3. Address code review comments

    sl255051 committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    9bcb2b1 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    7a25b52 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2024

  1. Configuration menu
    Copy the full SHA
    a31bf94 View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. Configuration menu
    Copy the full SHA
    44a7f91 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2024

  1. Configuration menu
    Copy the full SHA
    c2eaf24 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2024

  1. DRAFT: alternate solution 2: hack in support for NullVector

    This solution hacks in a VectorHolder instance built specifically for the missing column. Implementing this hack allowed me to explore what would be needed to support vectorized reading of null columns
    slessard committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    e323db7 View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2024

  1. Configuration menu
    Copy the full SHA
    061ab02 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. Configuration menu
    Copy the full SHA
    bf0c905 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    5610dd4 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    a13415d View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2024

  1. Configuration menu
    Copy the full SHA
    2eaa63f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    62108da View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    7115e93 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2024

  1. Configuration menu
    Copy the full SHA
    08bb07c View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    442b381 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2024

  1. Configuration menu
    Copy the full SHA
    5e7668e View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. Configuration menu
    Copy the full SHA
    83913a0 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e2b428e View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2024

  1. Address code review comments

    slessard committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    7ffa7ed View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. Configuration menu
    Copy the full SHA
    cda0423 View commit details
    Browse the repository at this point in the history
  2. Update unit test to add a second row to the table being tested

    Update unit test to write a row after the schema has been altered. The test will then verify that all rows written both before and after the schema change can be correctly read.
    slessard committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    9aec9e5 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. Code cleanup

    slessard committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    0c87dc7 View commit details
    Browse the repository at this point in the history
  2. Undo adding a second row to the table

    Adding a second row was creating test complexity. The order in which the two rows are read asynchronously was creating randomness thus making it hard to predict the expected values. I'm not sure adding a second row of data was really adding any benefit anyway.
    slessard committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    e5eebd0 View commit details
    Browse the repository at this point in the history
  3. Expand calls to checkAllVectorTypes and checkAllVectorValues

    Those two test helper methods are highly tuned for a specific schema, a schema that does not exist in this test.
    slessard committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    fe60793 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    1a3896b View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2024

  1. Add unit tests for VectorHolder

    These unit tests, particularly `testIsDummy1` and `testIsDummy2`, exposed a bug in the code where the `isDummy` method no longer returned the expected value.
    slessard committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    5c3b460 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a2df95c View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2024

  1. Fix code style issues

    slessard committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    bbc776d View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2024

  1. Configuration menu
    Copy the full SHA
    2bf5b2f View commit details
    Browse the repository at this point in the history
  2. Convert to fluent assertions

    slessard committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    1edd680 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    e1b3931 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    e574623 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. Configuration menu
    Copy the full SHA
    c8bcc1c View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    da9e514 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    fe83726 View commit details
    Browse the repository at this point in the history