-
Notifications
You must be signed in to change notification settings - Fork 5.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[native] Implement bucket conversion for Hive splits #23028
Conversation
presto-native-execution/presto_cpp/main/types/PrestoToVeloxConnector.cpp
Show resolved
Hide resolved
presto-native-execution/presto_cpp/main/types/PrestoToVeloxConnector.cpp
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @Yuhta. Overall looks very good minus some documentation comments.
@Yuhta : We should have a Release note for this PR I feel. Please can you add details in the PR description. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Yuhta thanks for the fix!
presto-native-execution/presto_cpp/main/types/PrestoToVeloxConnector.cpp
Outdated
Show resolved
Hide resolved
presto-native-execution/presto_cpp/main/types/PrestoToVeloxConnector.cpp
Show resolved
Hide resolved
presto-native-execution/presto_cpp/main/types/PrestoToVeloxConnector.cpp
Outdated
Show resolved
Hide resolved
presto-native-execution/presto_cpp/main/types/tests/PrestoToVeloxSplitTest.cpp
Outdated
Show resolved
Hide resolved
e083aa8
to
5150948
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @Yuhta
When the bucket count of a table changes over time, there can be legitimate cases that multiple buckets exist in the same file. In such cases the query planner should set bucket conversion for these splits and in Velox we use extra filter to get only the rows corresponding to the bucket number requested.
5150948
to
0596214
Compare
When the bucket count of a table changes over time, there can be legitimate cases that multiple buckets exist in the same file. In such cases the query planner should set bucket conversion for these splits and in Velox we use extra filter to get only the rows corresponding to the bucket number requested.