Skip to content
This repository has been archived by the owner on Sep 18, 2023. It is now read-only.

Commit

Permalink
[NSE-1112] Fix Arrow array meta data validating issue when writing pa…
Browse files Browse the repository at this point in the history
…rquet files (#1113)

* Add log to troubleshoot

* change reserve to resize

* comment cout
  • Loading branch information
zhixingheyi-tian authored Sep 21, 2022
1 parent 86e1a91 commit 022ce98
Showing 1 changed file with 4 additions and 4 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -149,15 +149,15 @@ inline arrow::Status CreateArrayData(
int32_t wordoffset = int32_t(offsetAndSize >> 32);
auto value_offset = array_offset[position + 1] =
array_offset[position] + length;
uint64_t capacity = array->buffers[2]->capacity();
uint64_t size = array->buffers[2]->size();

if (ARROW_PREDICT_FALSE(value_offset >= capacity)) {
if (ARROW_PREDICT_FALSE(value_offset >= size)) {
// allocate value buffer again
// enlarge the buffer by 1.5x
capacity = capacity + std::max((capacity >> 1), (uint64_t)length);
size = size + std::max((size >> 1), (uint64_t)length);
auto value_buffer =
std::static_pointer_cast<arrow::ResizableBuffer>(array->buffers[2]);
value_buffer->Reserve(capacity);
value_buffer->Resize(size);
array_data = value_buffer->mutable_data();
}

Expand Down

0 comments on commit 022ce98

Please sign in to comment.