Faster ListArray to StringArray conversion #3593
Conversation
| @@ -663,18 +620,28 @@ mod tests { | |||
|
|
|||
| #[test] | |||
| #[should_panic( | |||
| expected = "StringArray can only be created from List<u8> arrays, mismatched data types." | |||
| expected = "BinaryArray can only be created from List<u8> arrays, mismatched data types." | |||
There was a problem hiding this comment.
This is a change, but I'm inclined to think it doesn't matter
There was a problem hiding this comment.
Might be a bit confusing but seems minor.
| )] | ||
| fn test_large_string_array_from_list_array_wrong_type() { | ||
| _test_generic_string_array_from_list_array_wrong_type::<i32>(); | ||
| _test_generic_string_array_from_list_array_wrong_type::<i64>(); |
There was a problem hiding this comment.
This was a mistake it wasn't actually testing large string array 😱
| .add_buffer(child_data.buffers()[0].slice(child_data.offset())) | ||
| .null_bit_buffer(v.data().null_buffer().cloned()); | ||
|
|
||
| Self::from(builder.build().unwrap()) |
There was a problem hiding this comment.
This is what previously performed UTF-8 validation
|
Benchmark runs are scheduled for baseline = 3f72ebb and contender = 1afefbb. 1afefbb is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
Which issue does this PR close?
Closes #.
Rationale for this change
#3168 added faster UTF-8 validation for
BinaryArraytoStringArrayconversion, we should use that logic when converting from Lists. As an added bonus this is less codeWhat changes are included in this PR?
Are there any user-facing changes?