forked from apache/arrow
-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
ARROW-13965: [C++] dynamic_casts in parquet TypedColumnWriterImpl imp…
…acting performance This PR adds observer pointers to TypedColumnWriterImpl to avoid repeated dynamic_casts upon writing many small batches. This can lead to significant performance gains. As an example, using stream_reader_writer from examples/parquet/parquet_stream_api modified to write 20M rows, an improvement of over 30% was seen in write times. ``` original: 52.919u 2.146s 0:55.81 98.6% 0+0k 0+2646928io 0pf+0w 53.328u 2.327s 0:56.38 98.6% 0+0k 0+2646928io 0pf+0w 53.048u 2.341s 0:56.06 98.7% 0+0k 0+2646928io 0pf+0w patched: 35.367u 2.012s 0:37.88 98.6% 0+0k 0+2646928io 0pf+0w 35.133u 1.717s 0:37.36 98.6% 0+0k 0+2646928io 0pf+0w 35.228u 1.950s 0:37.61 98.8% 0+0k 0+2646928io 0pf+0w ``` Closes apache#11131 from etseidl/downcast Lead-authored-by: seidl <seidl2@llnl.gov> Co-authored-by: Antoine Pitrou <antoine@python.org> Co-authored-by: etseidl <etseidl@users.noreply.github.com> Signed-off-by: Antoine Pitrou <antoine@python.org>
- Loading branch information
1 parent
1130d18
commit 5260fd5
Showing
1 changed file
with
25 additions
and
20 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters