Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[pixels-core, pixels-presto] support 'is null' in query predicate. #103

Closed
bianhq opened this issue May 1, 2021 · 4 comments
Closed

[pixels-core, pixels-presto] support 'is null' in query predicate. #103

bianhq opened this issue May 1, 2021 · 4 comments
Assignees
@bianhq
Labels
bug Something isn't working
Milestone
Compatibility

Comments

@bianhq
Copy link
Contributor

bianhq commented May 1, 2021

Currently, using 'is null' in Presto may encounter the following error:

com.facebook.presto.spi.PrestoException: Failed to prepare for read.
	at io.pixelsdb.pixels.presto.PixelsPageSource.getNextPage(PixelsPageSource.java:234)
	at com.facebook.presto.operator.ScanFilterAndProjectOperator.processPageSource(ScanFilterAndProjectOperator.java:273)
	at com.facebook.presto.operator.ScanFilterAndProjectOperator.getOutput(ScanFilterAndProjectOperator.java:234)
	at com.facebook.presto.operator.Driver.processInternal(Driver.java:379)
	at com.facebook.presto.operator.Driver.lambda$processFor$8(Driver.java:278)
	at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:659)
	at com.facebook.presto.operator.Driver.processFor(Driver.java:272)
	at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:975)
	at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)
	at com.facebook.presto.execution.executor.LegacyPrioritizedSplitRunner.process(LegacyPrioritizedSplitRunner.java:23)
	at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:492)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Failed to prepare for read.
	at io.pixelsdb.pixels.core.reader.PixelsRecordReaderImpl.prepareBatch(PixelsRecordReaderImpl.java:601)
	at io.pixelsdb.pixels.presto.PixelsPageSource.getNextPage(PixelsPageSource.java:230)
	... 13 more
@bianhq bianhq added the bug Something isn't working label May 1, 2021
@bianhq bianhq self-assigned this May 1, 2021
@bianhq
Copy link
Contributor Author

bianhq commented May 1, 2021

If we use '=null' (although not meaningful) in query predicate, we encounter the following error:

java.lang.NullPointerException: undefined
	at io.pixelsdb.pixels.core.reader.PixelsRecordReaderImpl.prepareRead(PixelsRecordReaderImpl.java:254)
	at io.pixelsdb.pixels.core.reader.PixelsRecordReaderImpl.prepareBatch(PixelsRecordReaderImpl.java:599)
	at io.pixelsdb.pixels.presto.PixelsPageSource.getNextPage(PixelsPageSource.java:230)
	at com.facebook.presto.operator.ScanFilterAndProjectOperator.processPageSource(ScanFilterAndProjectOperator.java:273)
	at com.facebook.presto.operator.ScanFilterAndProjectOperator.getOutput(ScanFilterAndProjectOperator.java:234)
	at com.facebook.presto.operator.Driver.processInternal(Driver.java:379)
	at com.facebook.presto.operator.Driver.lambda$processFor$8(Driver.java:278)
	at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:659)
	at com.facebook.presto.operator.Driver.processFor(Driver.java:272)
	at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:975)
	at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)
	at com.facebook.presto.execution.executor.LegacyPrioritizedSplitRunner.process(LegacyPrioritizedSplitRunner.java:23)
	at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:492)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

@bianhq
Copy link
Contributor Author

bianhq commented May 3, 2021

Anthoer related problem:
Query select column_a from test_105 where column_b = null will never stop.

bianhq added a commit to bianhq/pixels that referenced this issue May 3, 2021
@bianhq
Issue pixelsdb#103: fix the first problem.
948766c
bianhq added a commit to bianhq/pixels that referenced this issue May 3, 2021
@bianhq
Issue pixelsdb#103: update PixelsPredicate.
4c13c34
@bianhq
Copy link
Contributor Author

bianhq commented May 3, 2021

Anthoer related problem:
Query select column_a from test_105 where column_b = null will never stop.

We have this problem after fixing the first ('is null') problem. And we confirm that this is caused by the prepareBatch behavior and the Improper action taken by Presto. So we are going to partially resolve it in another issue.

bianhq added a commit to bianhq/pixels that referenced this issue May 3, 2021
@bianhq
Issue pixelsdb#103: fix matches and add comments.
c4d625f
bianhq added a commit that referenced this issue May 4, 2021
@bianhq
[Issue #103]: fix and enhance predicate processing. (#104)
c6f5aac 
1. fix predicate processing in PixelsRecordReaderImpl and TupleDomainPixelsPredicate.
2. correctly support 'is null' and constant predicates.
3. for '=null', the query would still be endless, but this is not due to predicate processing.
@bianhq
Copy link
Contributor Author

bianhq commented May 4, 2021

Resolved in PR #104.

@bianhq bianhq closed this as completed May 4, 2021
@bianhq bianhq added this to the TPC-H milestone Oct 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bianhq bianhq

Labels
bug Something isn't working
Projects
None yet
Milestone
Compatibility
Development

No branches or pull requests
1 participant
@bianhq