Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky testAggregation in Pinot smoke test #14239

Open
findepi opened this issue Sep 21, 2022 · 10 comments
Open

Flaky testAggregation in Pinot smoke test #14239

findepi opened this issue Sep 21, 2022 · 10 comments
Assignees
Labels
bug Something isn't working test

Comments

@findepi
Copy link
Member

findepi commented Sep 21, 2022

https://github.com/trinodb/trino/actions/runs/3097813235/jobs/5015484200

Error:  io.trino.plugin.pinot.TestPinotWithoutAuthenticationIntegrationLatestVersionNoGrpcConnectorSmokeTest.testAggregation  Time elapsed: 13.462 s  <<< FAILURE!
java.lang.AssertionError: Execution of 'actual' query failed: SELECT sum(regionkey) FROM nation
	at org.testng.Assert.fail(Assert.java:83)
	at io.trino.testing.QueryAssertions.assertQuery(QueryAssertions.java:150)
	at io.trino.testing.QueryAssertions.assertQuery(QueryAssertions.java:106)
	at io.trino.testing.AbstractTestQueryFramework.assertQuery(AbstractTestQueryFramework.java:247)
	at io.trino.testing.AbstractTestQueryFramework.assertQuery(AbstractTestQueryFramework.java:242)
	at io.trino.testing.BaseConnectorSmokeTest.testAggregation(BaseConnectorSmokeTest.java:112)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1177)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: io.trino.testing.QueryFailedException: Query select sum("regionkey") from nation encountered exception org.apache.pinot.common.response.broker.QueryProcessingException@65555cc7 with query "select sum("regionkey") from nation"
	at io.trino.testing.AbstractTestingTrinoClient.execute(AbstractTestingTrinoClient.java:123)
	at io.trino.testing.DistributedQueryRunner.execute(DistributedQueryRunner.java:480)
	at io.trino.testing.QueryAssertions.assertQuery(QueryAssertions.java:147)
	... 17 more
	Suppressed: java.lang.Exception: SQL: SELECT sum(regionkey) FROM nation
		at io.trino.testing.DistributedQueryRunner.execute(DistributedQueryRunner.java:483)
		... 18 more
Caused by: io.trino.plugin.pinot.PinotException: Query select sum("regionkey") from nation encountered exception org.apache.pinot.common.response.broker.QueryProcessingException@65555cc7 with query "select sum("regionkey") from nation"
	at io.trino.plugin.pinot.client.PinotClient.lambda$submitBrokerQueryJson$6(PinotClient.java:553)
	at io.trino.plugin.pinot.client.PinotClient.doWithRetries(PinotClient.java:621)
	at io.trino.plugin.pinot.client.PinotClient.submitBrokerQueryJson(PinotClient.java:533)
	at io.trino.plugin.pinot.client.PinotClient.createResultIterator(PinotClient.java:578)
	at io.trino.plugin.pinot.PinotBrokerPageSource.getNextPage(PinotBrokerPageSource.java:112)
	at io.trino.operator.TableScanOperator.getOutput(TableScanOperator.java:311)
	at io.trino.operator.Driver.processInternal(Driver.java:411)
	at io.trino.operator.Driver.lambda$process$10(Driver.java:314)
	at io.trino.operator.Driver.tryWithLock(Driver.java:706)
	at io.trino.operator.Driver.process(Driver.java:306)
	at io.trino.operator.Driver.processForDuration(Driver.java:277)
	at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:736)
	at io.trino.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:164)
	at io.trino.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:515)
	at io.trino.$gen.Trino_testversion____20220921_142351_106.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
@findepi findepi changed the title TestPinotWithoutAuthenticationIntegrationLatestVersionNoGrpcConnectorSmokeTest Flaky test TestPinotWithoutAuthenticationIntegrationLatestVersionNoGrpcConnectorSmokeTest.testAggregation Sep 21, 2022
@findepi findepi added the bug Something isn't working label Sep 21, 2022
@findepi
Copy link
Member Author

findepi commented Sep 26, 2022

@findinpath
Copy link
Contributor

@xiangfu0
Copy link
Contributor

xiangfu0 commented Oct 27, 2022

This is a fix: #14304

@nevillelyh
Copy link
Member

Related: TestPinotWithoutAuthenticationIntegrationLatestVersionConnectorSmokeTest>BaseConnectorSmokeTest.testAggregation
https://github.com/trinodb/trino/actions/runs/3334604928/jobs/5518590450

@vlad-lyutenko
Copy link
Contributor

vlad-lyutenko commented Oct 31, 2022

I have randomly failing tests for TestPinotWithoutAuthenticationIntegrationLatestVersionConnectorSmokeTest
it varies from 10 to 17 tests like:
testTimestamp:

Expecting:
  <>
to contain exactly in any order:
  <[(1970-01-01T00:00)]>
but could not find the following elements:
  <(1970-01-01T00:00)>

testVarbinary
e.t.c
Looks like we got empty responses, when smth goes wrong:

https://github.com/trinodb/trino/actions/runs/3360648854/jobs/5570242351

It pass locally, not sure should it be separate issue or it's somehow connected with this one

My PR #14570 is about changes to SQL server only

@Dith3r
Copy link
Member

Dith3r commented Nov 9, 2022

@ebyhr
Copy link
Member

ebyhr commented Dec 12, 2022

I have randomly failing tests for TestPinotWithoutAuthenticationIntegrationLatestVersionConnectorSmokeTest

@vlad-lyutenko The stacktrace is different from the issue description. Filed #15367

@findepi
Copy link
Member Author

findepi commented Jan 5, 2023

https://github.com/trinodb/trino/actions/runs/3847136098/jobs/6553734233

Error:  io.trino.plugin.pinot.TestPinotWithoutAuthenticationIntegrationLatestVersionNoGrpcConnectorSmokeTest.testAggregation  Time elapsed: 11.127 s  <<< FAILURE!
java.lang.AssertionError: Execution of 'actual' query failed: SELECT sum(regionkey) FROM nation
	at org.testng.Assert.fail(Assert.java:83)
	at io.trino.testing.QueryAssertions.assertQuery(QueryAssertions.java:150)
	at io.trino.testing.QueryAssertions.assertQuery(QueryAssertions.java:106)
	at io.trino.testing.AbstractTestQueryFramework.assertQuery(AbstractTestQueryFramework.java:299)
	at io.trino.testing.AbstractTestQueryFramework.assertQuery(AbstractTestQueryFramework.java:294)
	at io.trino.testing.BaseConnectorSmokeTest.testAggregation(BaseConnectorSmokeTest.java:117)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1177)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: io.trino.testing.QueryFailedException: Query select sum("regionkey") from nation encountered exception code: '427' message: '1 servers [f303697005f9_R] not responded' with query "select sum("regionkey") from nation"
	at io.trino.testing.AbstractTestingTrinoClient.execute(AbstractTestingTrinoClient.java:122)
	at io.trino.testing.DistributedQueryRunner.execute(DistributedQueryRunner.java:490)
	at io.trino.testing.QueryAssertions.assertQuery(QueryAssertions.java:147)
	... 17 more
	Suppressed: java.lang.Exception: SQL: SELECT sum(regionkey) FROM nation
		at io.trino.testing.DistributedQueryRunner.execute(DistributedQueryRunner.java:493)
		... 18 more
Caused by: io.trino.plugin.pinot.PinotException: Query select sum("regionkey") from nation encountered exception code: '427' message: '1 servers [f303697005f9_R] not responded' with query "select sum("regionkey") from nation"
	at io.trino.plugin.pinot.client.PinotClient.lambda$submitBrokerQueryJson$7(PinotClient.java:563)
	at io.trino.plugin.pinot.client.PinotClient.doWithRetries(PinotClient.java:631)
	at io.trino.plugin.pinot.client.PinotClient.submitBrokerQueryJson(PinotClient.java:540)
	at io.trino.plugin.pinot.client.PinotClient.createResultIterator(PinotClient.java:588)
	at io.trino.plugin.pinot.PinotBrokerPageSource.getNextPage(PinotBrokerPageSource.java:112)
	at io.trino.operator.TableScanOperator.getOutput(TableScanOperator.java:311)
	at io.trino.operator.Driver.processInternal(Driver.java:411)
	at io.trino.operator.Driver.lambda$process$10(Driver.java:314)
	at io.trino.operator.Driver.tryWithLock(Driver.java:706)
	at io.trino.operator.Driver.process(Driver.java:306)
	at io.trino.operator.Driver.processForDuration(Driver.java:277)
	at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:752)
	at io.trino.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:164)
	at io.trino.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:519)
	at io.trino.$gen.Trino_testversion____20230105_144724_2145.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

[INFO] 
[INFO] Results:

@elonazoulay

@findepi
Copy link
Member Author

findepi commented Apr 22, 2023

https://github.com/trinodb/trino/actions/runs/4766640955/jobs/8475851653?pr=17175

Error:  io.trino.plugin.pinot.TestPinotConnectorSmokeTest.testAggregation  Time elapsed: 0.308 s  <<< FAILURE!
java.lang.AssertionError: Execution of 'actual' query 20230421_214636_00002_vyvf7 failed: SELECT sum(regionkey) FROM nation
	at org.testng.Assert.fail(Assert.java:83)
	at io.trino.testing.QueryAssertions.assertDistributedQuery(QueryAssertions.java:286)
	at io.trino.testing.QueryAssertions.assertQuery(QueryAssertions.java:184)
	at io.trino.testing.QueryAssertions.assertQuery(QueryAssertions.java:157)
	at io.trino.testing.AbstractTestQueryFramework.assertQuery(AbstractTestQueryFramework.java:334)
	at io.trino.testing.AbstractTestQueryFramework.assertQuery(AbstractTestQueryFramework.java:329)
	at io.trino.testing.BaseConnectorSmokeTest.testAggregation(BaseConnectorSmokeTest.java:119)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1177)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: io.trino.testing.QueryFailedException: Query SELECT sum("regionkey") FROM nation encountered exception code: '427' message: '1 servers [d56d168eb72a_R] not responded' with query "SELECT sum("regionkey") FROM nation"
	at io.trino.testing.AbstractTestingTrinoClient.execute(AbstractTestingTrinoClient.java:122)
	at io.trino.testing.DistributedQueryRunner.executeWithQueryId(DistributedQueryRunner.java:493)
	at io.trino.testing.QueryAssertions.assertDistributedQuery(QueryAssertions.java:277)
	... 18 more
	Suppressed: java.lang.Exception: SQL: SELECT sum(regionkey) FROM nation
		at io.trino.testing.DistributedQueryRunner.executeWithQueryId(DistributedQueryRunner.java:497)
		... 19 more
Caused by: io.trino.plugin.pinot.PinotException: Query SELECT sum("regionkey") FROM nation encountered exception code: '427' message: '1 servers [d56d168eb72a_R] not responded' with query "SELECT sum("regionkey") FROM nation"
	at io.trino.plugin.pinot.client.PinotClient.lambda$submitBrokerQueryJson$7(PinotClient.java:563)
	at io.trino.plugin.pinot.client.PinotClient.doWithRetries(PinotClient.java:631)
	at io.trino.plugin.pinot.client.PinotClient.submitBrokerQueryJson(PinotClient.java:540)
	at io.trino.plugin.pinot.client.PinotClient.createResultIterator(PinotClient.java:588)
	at io.trino.plugin.pinot.PinotBrokerPageSource.getNextPage(PinotBrokerPageSource.java:112)
	at io.trino.operator.TableScanOperator.getOutput(TableScanOperator.java:301)
	at io.trino.operator.Driver.processInternal(Driver.java:402)
	at io.trino.operator.Driver.lambda$process$8(Driver.java:305)
	at io.trino.operator.Driver.tryWithLock(Driver.java:701)
	at io.trino.operator.Driver.process(Driver.java:297)
	at io.trino.operator.Driver.processForDuration(Driver.java:268)
	at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:888)
	at io.trino.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:187)
	at io.trino.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:561)
	at io.trino.$gen.Trino_testversion____20230421_214553_2256.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

@ebyhr ebyhr changed the title Flaky test TestPinotWithoutAuthenticationIntegrationLatestVersionNoGrpcConnectorSmokeTest.testAggregation Flaky testAggregation in Pinot smoke test Jul 14, 2023
@ebyhr
Copy link
Member

ebyhr commented Jul 14, 2023

https://github.com/trinodb/trino/actions/runs/5548635127/jobs/10136844578?pr=17667

 io.trino.plugin.pinot.TestPinotSecuredConnectorSmokeTest.testAggregation  Time elapsed: 0.932 s  <<< FAILURE!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working test
Development

No branches or pull requests

9 participants