-
Notifications
You must be signed in to change notification settings - Fork 760
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: use parquet-rs as an alternative in fuse engine #14268
Conversation
# Conflicts: # src/query/expression/src/convert_arrow_rs/record_batch.rs # src/query/expression/src/convert_arrow_rs/schema/from_table_schema.rs
# Conflicts: # tests/sqllogictests/suites/mode/standalone/explain/join_reorder/mark.test
# Conflicts: # tests/sqllogictests/suites/mode/standalone/explain/eliminate_outer_join.test # tests/sqllogictests/suites/mode/standalone/explain/explain.test # tests/sqllogictests/suites/mode/standalone/explain/join_reorder/chain.test # tests/sqllogictests/suites/mode/standalone/explain/join_reorder/star.test
can we get parquet write performance improvement after using parquet-rs in this pr? @SkyFan2002 |
No. Do you have any concerns? |
At least hopefully there is no performance reducing. |
Docker Image for PR
|
Co-authored-by: Bohu <overred.shuttler@gmail.com>
chore: add some comments and minor refactor
I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/
Summary
Migrate scope
Migrated to parquet-rs in this PR:
Still parquet2: (They read parquet in different code paths, so I decide to split into different PRs
Compatibility
Backwards compatibility is tested by
in sql logic test.
Fixes #14135
Tests
Type of change
This change is