-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: support partition prune api #119
Conversation
@xushiyan cc |
@xushiyan hello, Is there a problem with my test method in my local? |
91eb0d1
to
d1f5ffb
Compare
ignore checkstyle, fetch the latest commit, it work again, but can not report python test error. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #119 +/- ##
==========================================
+ Coverage 87.82% 89.32% +1.50%
==========================================
Files 14 15 +1
Lines 731 834 +103
==========================================
+ Hits 642 745 +103
Misses 89 89 ☔ View full report in Codecov by Sentry. |
085fbba
to
9551275
Compare
@xushiyan cc: |
@KnightChess awesome contribution! let me take a look. i might push some quick fixes just FYI to move faster. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@KnightChess good progress. i think the main functionalities are there. A major change is to re-model the Partition struct, we need to decouple it from datafusion struct
@KnightChess do you think you can address the main comment in the next few days? then i can polish further if needed and land this. Trying to get this in the upcoming release within 2 weeks 🙂 (cutting RC branch with a week) |
@xushiyan sorry for reply late, I will address these two days |
@xushiyan Hello, I couldn't find an implementation similar to ScalarValue, and I am not very familiar with Arrow yet, and. There is a certain learning curve involved, which might delay the progress of this PR. Could you please help improve this PR? |
70f14f3
to
2890fee
Compare
@xushiyan cc, I try to use arrow |
@KnightChess Thanks. I was traveling. Will take a look later today. |
I was wrapping up my vacation 😄 just now getting back to update this: Changes I've made:
There are more follow up work to do on datafusion integration side, which I'll jot down in the GH issue. |
@xushiyan thanks review |
Description
Add filtering capabilities to table API, currently only partition fields are applicable. Multiple predicates are
AND
together.Supported operators are:
>, >=, <, <=, =, !=
.For #47
How are the changes test-covered