[Flaky] Big query tests sometimes fail with "Server is not responding"

[losipiuk]

Example stack trace

Error:  io.trino.plugin.bigquery.TestBigQueryTaskFailureRecoveryTest.testParallel[testInsert](11)  Time elapsed: 129.017 s  <<< FAILURE!
io.trino.testing.QueryFailedException: Server is not responding
	at io.trino.testing.AbstractTestingTrinoClient.execute(AbstractTestingTrinoClient.java:122)
	at io.trino.testing.DistributedQueryRunner.executeWithQueryId(DistributedQueryRunner.java:494)
	at io.trino.testing.BaseFailureRecoveryTest$FailureRecoveryAssert.execute(BaseFailureRecoveryTest.java:568)
	at io.trino.testing.BaseFailureRecoveryTest$FailureRecoveryAssert.executeActual(BaseFailureRecoveryTest.java:553)
	at io.trino.testing.BaseFailureRecoveryTest$FailureRecoveryAssert.executeActual(BaseFailureRecoveryTest.java:533)
	at io.trino.testing.BaseFailureRecoveryTest$FailureRecoveryAssert.finishesSuccessfully(BaseFailureRecoveryTest.java:650)
	at io.trino.testing.BaseFailureRecoveryTest$FailureRecoveryAssert.finishesSuccessfully(BaseFailureRecoveryTest.java:641)
	at io.trino.testing.BaseFailureRecoveryTest$FailureRecoveryAssert.finishesSuccessfully(BaseFailureRecoveryTest.java:631)
	at io.trino.testing.BaseFailureRecoveryTest.testNonSelect(BaseFailureRecoveryTest.java:398)
	at io.trino.testing.BaseFailureRecoveryTest.testTableModification(BaseFailureRecoveryTest.java:348)
	at io.trino.testing.BaseFailureRecoveryTest.testTableModification(BaseFailureRecoveryTest.java:343)
	at io.trino.testing.BaseFailureRecoveryTest.testInsert(BaseFailureRecoveryTest.java:245)
	at io.trino.testing.BaseFailureRecoveryTest$ParallelTestRunnable.run(BaseFailureRecoveryTest.java:905)
	at io.trino.testing.BaseFailureRecoveryTest.testParallel(BaseFailureRecoveryTest.java:228)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
	at org.testng.internal.TestMethodWithDataProviderMethodWorker.call(TestMethodWithDataProviderMethodWorker.java:75)
	at org.testng.internal.TestMethodWithDataProviderMethodWorker.call(TestMethodWithDataProviderMethodWorker.java:14)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
	Suppressed: java.lang.Exception: SQL: INSERT INTO table_i4rpfi98yk SELECT * FROM orders
		at io.trino.testing.DistributedQueryRunner.executeWithQueryId(DistributedQueryRunner.java:498)
		... 27 more
Caused by: com.google.cloud.bigquery.BigQueryException: Server is not responding
	at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:115)
	at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.insertAll(HttpBigQueryRpc.java:507)
	at com.google.cloud.bigquery.BigQueryImpl.insertAll(BigQueryImpl.java:1097)
	at io.trino.plugin.bigquery.BigQueryClient.insert(BigQueryClient.java:346)
	at io.trino.plugin.bigquery.BigQueryPageSink.appendPage(BigQueryPageSink.java:85)
	at io.trino.operator.TableWriterOperator.addInput(TableWriterOperator.java:258)
	at io.trino.operator.Driver.processInternal(Driver.java:407)
	at io.trino.operator.Driver.lambda$process$8(Driver.java:305)
	at io.trino.operator.Driver.tryWithLock(Driver.java:701)
	at io.trino.operator.Driver.process(Driver.java:297)
	at io.trino.operator.Driver.processForDuration(Driver.java:268)
	at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:845)
	at io.trino.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:165)
	at io.trino.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:537)
	at io.trino.$gen.Trino_testversion____20230329_185701_914.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 503 Service Unavailable
POST https://www.googleapis.com/bigquery/v2/projects/sep-bq-cicd/datasets/tpch/tables/tmp_trino_0ec336b7_f6c64a47/insertAll?prettyPrint=false
{
  "code" : 503,
  "errors" : [ {
    "domain" : "global",
    "message" : "Server is not responding",
    "reason" : "backendError"
  } ],
  "message" : "Server is not responding",
  "status" : "UNAVAILABLE"
}
	at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
	at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
	at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:439)
	at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:525)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:466)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:576)
	at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.insertAll(HttpBigQueryRpc.java:505)
	... 16 more

[ebyhr]

Error:    TestBigQueryQueryFailureRecoveryTest>BaseFailureRecoveryTest.testParallel:227->BaseBigQueryFailureRecoveryTest.testAnalyzeTable:66 
Expecting message:
  "An internal error occurred and the request could not be completed. This is usually caused by a transient issue. Retrying the job with back-off as described in the BigQuery SLA should solve the problem: https://cloud.google.com/bigquery/sla. If the error continues to occur please contact support at https://cloud.google.com/support."
to match regex:
  "This connector does not support analyze"
but did not.

https://github.com/trinodb/trino/actions/runs/5206542855/jobs/9393320502