Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky io.prestosql.tests.hive.TestHiveStorageFormats.testInsertIntoPartitionedTable #4966

Closed
losipiuk opened this issue Aug 25, 2020 · 1 comment
Closed

Flaky io.prestosql.tests.hive.TestHiveStorageFormats.testInsertIntoPartitionedTable #4966

losipiuk opened this issue Aug 25, 2020 · 1 comment
Labels
bug Something isn't working test

Comments

@losipiuk
Copy link
Member 

2020-08-25T08:29:47.0110437Z tests               | 2020-08-25 14:14:46 SEVERE: Failure cause:
2020-08-25T08:29:47.0110935Z tests               | io.prestosql.tempto.query.QueryExecutionException: java.sql.SQLException: Query failed (#20200825_082946_00525_xhkq7): Error committing write to Hive
2020-08-25T08:29:47.0111448Z tests               | 	at io.prestosql.tempto.query.JdbcQueryExecutor.execute(JdbcQueryExecutor.java:114)
2020-08-25T08:29:47.0111907Z tests               | 	at io.prestosql.tempto.query.JdbcQueryExecutor.executeQuery(JdbcQueryExecutor.java:82)
2020-08-25T08:29:47.0112346Z tests               | 	at io.prestosql.tempto.query.QueryExecutor.query(QueryExecutor.java:57)
2020-08-25T08:29:47.0112858Z tests               | 	at io.prestosql.tests.hive.TestHiveStorageFormats.testInsertIntoPartitionedTable(TestHiveStorageFormats.java:197)
2020-08-25T08:29:47.0113354Z tests               | 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2020-08-25T08:29:47.0113842Z tests               | 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2020-08-25T08:29:47.0114338Z tests               | 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2020-08-25T08:29:47.0114812Z tests               | 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
2020-08-25T08:29:47.0115283Z tests               | 	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
2020-08-25T08:29:47.0115746Z tests               | 	at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
2020-08-25T08:29:47.0116169Z tests               | 	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
2020-08-25T08:29:47.0116617Z tests               | 	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1177)
2020-08-25T08:29:47.0117080Z tests               | 	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
2020-08-25T08:29:47.0117657Z tests               | 	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
2020-08-25T08:29:47.0118115Z tests               | 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
2020-08-25T08:29:47.0118600Z tests               | 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
2020-08-25T08:29:47.0119049Z tests               | 	at java.base/java.lang.Thread.run(Thread.java:834)
2020-08-25T08:29:47.0119515Z tests               | Caused by: java.sql.SQLException: Query failed (#20200825_082946_00525_xhkq7): Error committing write to Hive
2020-08-25T08:29:47.0120011Z tests               | 	at io.prestosql.jdbc.AbstractPrestoResultSet.resultsException(AbstractPrestoResultSet.java:1757)
2020-08-25T08:29:47.0120483Z tests               | 	at io.prestosql.jdbc.PrestoResultSet$ResultsPageIterator.computeNext(PrestoResultSet.java:225)
2020-08-25T08:29:47.0127462Z tests               | 	at io.prestosql.jdbc.PrestoResultSet$ResultsPageIterator.computeNext(PrestoResultSet.java:185)
2020-08-25T08:29:47.0128079Z tests               | 	at io.prestosql.jdbc.$internal.guava.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2020-08-25T08:29:47.0128726Z tests               | 	at io.prestosql.jdbc.$internal.guava.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2020-08-25T08:29:47.0129206Z tests               | 	at java.base/java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1811)
2020-08-25T08:29:47.0129726Z tests               | 	at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
2020-08-25T08:29:47.0130248Z tests               | 	at java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
2020-08-25T08:29:47.0130767Z tests               | 	at java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:161)
2020-08-25T08:29:47.0131282Z tests               | 	at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
2020-08-25T08:29:47.0131751Z tests               | 	at java.base/java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
2020-08-25T08:29:47.0132231Z tests               | 	at io.prestosql.jdbc.PrestoResultSet$AsyncIterator.lambda$new$0(PrestoResultSet.java:131)
2020-08-25T08:29:47.0132704Z tests               | 	at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)
2020-08-25T08:29:47.0133108Z tests               | 	... 3 more
2020-08-25T08:29:47.0133535Z tests               | Caused by: io.prestosql.spi.PrestoException: Error committing write to Hive
2020-08-25T08:29:47.0133986Z tests               | 	at io.prestosql.plugin.hive.RecordFileWriter.commit(RecordFileWriter.java:190)
2020-08-25T08:29:47.0134431Z tests               | 	at io.prestosql.plugin.hive.HiveWriter.commit(HiveWriter.java:86)
2020-08-25T08:29:47.0134887Z tests               | 	at io.prestosql.plugin.hive.HivePageSink.doFinish(HivePageSink.java:190)
2020-08-25T08:29:47.0135353Z tests               | 	at io.prestosql.plugin.hive.authentication.NoHdfsAuthentication.doAs(NoHdfsAuthentication.java:23)
2020-08-25T08:29:47.0135811Z tests               | 	at io.prestosql.plugin.hive.HdfsEnvironment.doAs(HdfsEnvironment.java:89)
2020-08-25T08:29:47.0136245Z tests               | 	at io.prestosql.plugin.hive.HivePageSink.finish(HivePageSink.java:181)
2020-08-25T08:29:47.0136730Z tests               | 	at io.prestosql.plugin.base.classloader.ClassLoaderSafeConnectorPageSink.finish(ClassLoaderSafeConnectorPageSink.java:77)
2020-08-25T08:29:47.0137203Z tests               | 	at io.prestosql.operator.TableWriterOperator.finish(TableWriterOperator.java:203)
2020-08-25T08:29:47.0137636Z tests               | 	at io.prestosql.operator.Driver.processInternal(Driver.java:397)
2020-08-25T08:29:47.0138167Z tests               | 	at io.prestosql.operator.Driver.lambda$processFor$8(Driver.java:283)
2020-08-25T08:29:47.0138613Z tests               | 	at io.prestosql.operator.Driver.tryWithLock(Driver.java:675)
2020-08-25T08:29:47.0139039Z tests               | 	at io.prestosql.operator.Driver.processFor(Driver.java:276)
2020-08-25T08:29:47.0139499Z tests               | 	at io.prestosql.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1076)
2020-08-25T08:29:47.0139986Z tests               | 	at io.prestosql.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)
2020-08-25T08:29:47.0140448Z tests               | 	at io.prestosql.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:484)
2020-08-25T08:29:47.0140869Z tests               | 	at io.prestosql.$gen.Presto_338943c____20200825_082256_2.run(Unknown Source)
2020-08-25T08:29:47.0141326Z tests               | 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
2020-08-25T08:29:47.0141802Z tests               | 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
2020-08-25T08:29:47.0142317Z tests               | 	at java.base/java.lang.Thread.run(Thread.java:834)
2020-08-25T08:29:47.0143942Z tests               | Caused by: org.apache.hadoop.ipc.RemoteException: File /tmp/presto-hive/06dcae4b-210b-4041-8161-19e6926b9462/returnflag=R/20200825_082946_00525_xhkq7_21eddcdd-226e-4865-9d09-4d967fded3ba.gz could only be written to 0 of the 1 minReplication nodes. There are 1 datanode(s) running and no node(s) are excluded in this operation.
2020-08-25T08:29:47.0144635Z tests               | 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:2121)
2020-08-25T08:29:47.0145167Z tests               | 	at org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.chooseTargetForNewBlock(FSDirWriteFileOp.java:286)
2020-08-25T08:29:47.0145910Z tests               | 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2706)
2020-08-25T08:29:47.0146441Z tests               | 	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:875)
2020-08-25T08:29:47.0147012Z tests               | 	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:561)
2020-08-25T08:29:47.0147625Z tests               | 	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
2020-08-25T08:29:47.0148190Z tests               | 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
2020-08-25T08:29:47.0148650Z tests               | 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1025)
2020-08-25T08:29:47.0149070Z tests               | 	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:876)
2020-08-25T08:29:47.0149524Z tests               | 	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:822)
2020-08-25T08:29:47.0149965Z tests               | 	at java.security.AccessController.doPrivileged(Native Method)
2020-08-25T08:29:47.0150396Z tests               | 	at javax.security.auth.Subject.doAs(Subject.java:422)
2020-08-25T08:29:47.0150837Z tests               | 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
2020-08-25T08:29:47.0151301Z tests               | 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2682)
2020-08-25T08:29:47.0151682Z tests               | 
2020-08-25T08:29:47.0152075Z tests               | 	at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1511)
2020-08-25T08:29:47.0152493Z tests               | 	at org.apache.hadoop.ipc.Client.call(Client.java:1457)
2020-08-25T08:29:47.0152900Z tests               | 	at org.apache.hadoop.ipc.Client.call(Client.java:1367)
2020-08-25T08:29:47.0153358Z tests               | 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
2020-08-25T08:29:47.0153961Z tests               | 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
2020-08-25T08:29:47.0154406Z tests               | 	at com.sun.proxy.$Proxy254.addBlock(Unknown Source)
2020-08-25T08:29:47.0154900Z tests               | 	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:513)
2020-08-25T08:29:47.0155383Z tests               | 	at jdk.internal.reflect.GeneratedMethodAccessor442.invoke(Unknown Source)
2020-08-25T08:29:47.0155855Z tests               | 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2020-08-25T08:29:47.0156299Z tests               | 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
2020-08-25T08:29:47.0156737Z tests               | 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
2020-08-25T08:29:47.0157235Z tests               | 	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
2020-08-25T08:29:47.0157813Z tests               | 	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
2020-08-25T08:29:47.0158276Z tests               | 	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
2020-08-25T08:29:47.0158748Z tests               | 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
2020-08-25T08:29:47.0159179Z tests               | 	at com.sun.proxy.$Proxy255.addBlock(Unknown Source)
2020-08-25T08:29:47.0159586Z tests               | 	at org.apache.hadoop.hdfs.DFSOutputStream.addBlock(DFSOutputStream.java:1081)
2020-08-25T08:29:47.0159971Z tests               | 	at org.apache.hadoop.hdfs.DataStreamer.locateFollowingBlock(DataStreamer.java:1865)
2020-08-25T08:29:47.0160458Z tests               | 	at org.apache.hadoop.hdfs.DataStreamer.nextBlockOutputStream(DataStreamer.java:1668)
@losipiuk losipiuk added the test label Aug 25, 2020
@findepi findepi added the bug Something isn't working label Aug 25, 2020
@findepi
Copy link
Member

findepi commented Aug 25, 2020

duplicates #2390

@findepi findepi closed this as completed Aug 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working test
Milestone
No milestone
Development

No branches or pull requests
2 participants
@findepi @losipiuk