Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-1712: [C++] Add method to BinaryBuilder to reserve space for value data #1481

Closed
wants to merge 23 commits into from
Closed

Commits on Jan 15, 2018

  1. Merge pull request #1 from apache/master

    Syncing from original
    xuepanchen authored Jan 15, 2018
    Configuration menu
    Copy the full SHA
    c2f8dc4 View commit details
    Browse the repository at this point in the history
  2. Update BinaryBuilder::Resize(int64_t capacity) in builder.cc

    When building BinaryArrays with a known size using Resize and Reserve methods, space is also reserved for value_data_builder_ to prevent internal reallocation
    xuepanchen authored Jan 15, 2018
    Configuration menu
    Copy the full SHA
    232024e View commit details
    Browse the repository at this point in the history
  3. Merge pull request #2 from xuepanchen/xuepanchen-arrow-1712

    Update BinaryBuilder::Resize(int64_t capacity) in builder.cc
    xuepanchen authored Jan 15, 2018
    Configuration menu
    Copy the full SHA
    d021c54 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    5b73c1c View commit details
    Browse the repository at this point in the history

Commits on Jan 16, 2018

  1. Configuration menu
    Copy the full SHA
    5ebfb32 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e0434e6 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    de318f4 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b002e0b View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2018

  1. Update builder.cc

    xuepanchen authored Jan 17, 2018
    Configuration menu
    Copy the full SHA
    8dd5eaa View commit details
    Browse the repository at this point in the history
  2. Update ReserveData(int64_t) method signature for BinaryBuilder

    Replace parameter name from "bytes" to "capacity" to avoid confusion.
    xuepanchen authored Jan 17, 2018
    Configuration menu
    Copy the full SHA
    9b5e805 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    5a5593e View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2018

  1. Add test case for array-test.cc

    Add TestCapacityReserve to test space reservation for BinaryBuilder
    xuepanchen authored Jan 18, 2018
    Configuration menu
    Copy the full SHA
    15e045c View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    bbc6527 View commit details
    Browse the repository at this point in the history
  3. ARROW-1945: [C++] Add data_capacity_ to track capacity of value data

    data_capacity_ represents the indicated capacity for value_data_builder and it is always smaller than or equal to the actual capacity of underlying value_data_builder (data_capacity_ <= value_data_builder.capacity()). That's because when we say:
    
    ReserveData(capacity);
    
    The new capacity is max(data_capacity_, data length + capacity), and data_capacity_ is set to be equal to new capacity but underlying buffer size is set to BitUtil::RoundUpToMultipleOf64(new capacity) to ensure that the capacity of the buffer is a multiple of 64 bytes as defined in Layout.md.
    
    That's why data_capacity_ is needed to show the indicated capacity of the BinaryBuilder, just like ArrayBuilder::capacity_ indicates the indicated capacity of ArrayBuilder.
    
    A safety check is added in BinaryBuilder::Append() to update data_capacity_ if data length is greater than data_capacity_. The reason is that data_capacity is updated in ResearveData(). But if users make mistakes to append too much data, data length might be larger than data_capacity_ (data length <= actual capacity of underlying value_data_builder). If this happens data_capacity_ is set equal to data length to avoid confusion.
    xuepanchen authored Jan 18, 2018
    Configuration menu
    Copy the full SHA
    18f90fb View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    0b07895 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    d3c8202 View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2018

  1. Merge pull request #3 from xuepanchen/xuepanchen-arrow-1712

    Update ReserveData(int64_t) method for BinaryBuilder
    xuepanchen authored Jan 19, 2018
    Configuration menu
    Copy the full SHA
    8e4c892 View commit details
    Browse the repository at this point in the history
  2. Merge pull request #4 from apache/master

    Syncing from original
    xuepanchen authored Jan 19, 2018
    Configuration menu
    Copy the full SHA
    5a5b70e View commit details
    Browse the repository at this point in the history

Commits on Jan 20, 2018

  1. Configuration menu
    Copy the full SHA
    bc5db7d View commit details
    Browse the repository at this point in the history

Commits on Jan 22, 2018

  1. Merge pull request #5 from apache/master

    Syncing from original
    xuepanchen authored Jan 22, 2018
    Configuration menu
    Copy the full SHA
    77f8f3c View commit details
    Browse the repository at this point in the history
  2. ARROW-1712: [C++] Modify test case for BinaryBuilder::ReserveData() a…

    …nd change arguments for offsets_builder_.Resize()
    xuepanchen committed Jan 22, 2018
    Configuration menu
    Copy the full SHA
    d4bbd15 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    360e601 View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2018

  1. Configuration menu
    Copy the full SHA
    707b67b View commit details
    Browse the repository at this point in the history