-
Notifications
You must be signed in to change notification settings - Fork 752
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improvement: abandon internal patches of parquet2 #6064
Labels
C-improvement
Category: improvement
Comments
dantengsky
added
C-feature
Category: feature
C-improvement
Category: improvement
labels
Jun 20, 2022
dantengsky
changed the title
Feature: abandon internal patches of parquet2
Improvement: abandon internal patches of parquet2
Jun 20, 2022
Let's go upstream first! |
internal parquet2 patches are not totally abandoned yet ( for data format backward compatibility). after all the old data has been migrated, we should switch to the upstream parquet2, |
This was referenced Aug 31, 2022
It's time to remove https://github.com/datafuselabs/parquet2 and https://github.com/datafuse-extras/parquet2 ? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Summary
We have two internal patches of parqeut2, which mainly address the requirement
It works, but awkwardly: each time we sync with upstream(official parquet2), there are some extra works to do (rebase, resolve potential conflicts...)
Among the new features that parquet2 has introduced recently, the following two seem to be able to resolve the above requirement.
FileMetaData
to allow using side-car API jorgecarleitao/parquet2#148Thus,
parquet2
cargo dependency to the rev of the official parquet2 commitThe text was updated successfully, but these errors were encountered: