-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use correct configuration property for orc bloom filter #10343
Use correct configuration property for orc bloom filter #10343
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Honestly, that's kind of embarrassing... Difficulty in writing tests for this is only a partial excuse :(
The thing is that hive/orc doesn't provide a easier way to expose whether bloom filter is written or not. I'll take a stab at it again to check if it is possible or not. |
4ff453a
to
c6f629e
Compare
Is it possible to write a queryrunner test where the predicate can result in no data getting read only due to bloom filter (something similar to assertRowGroupPruning in TestParquetPageSkipping). |
349ad2b
to
f38b12a
Compare
@raunaqmorarka Thanks for the pointers. Last time when I checked with the query runner, the query stats where misleading (maybe I could have a different API). Now I'm able to test if the bloom filter is created or not. Thanks a bunch !! @ksobolew I have added integration tests for it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do I understand correctly that this never worked?
Yes. |
@Praveen2112 let's consider fixing release notes for 339 |
Writing ORC bloom filter is broken for Hive connector
f38b12a
to
0e85567
Compare
@findepi Updated release notes. |
Fixes #9792