BigQueryException when patching a table does not contain propagated error codes

[mtagle]

It looks like there are other flows that also don’t propagate error codes, but this particular one is what I saw.

I noticed this when I was updating a table. Our code has retry logic if the returned BigQueryException has an error code of 403.

relevant code lines:
https://github.com/GoogleCloudPlatform/google-cloud-java/blob/master/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/spi/DefaultBigQueryRpc.java#L206

https://github.com/GoogleCloudPlatform/google-cloud-java/blob/master/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/spi/DefaultBigQueryRpc.java#L93

GoogleJsonResponseException is a subclass of IOException, but the error code there isn't ending up in the wrapping BigQueryException.

The part of my stack trace that lead me to this conclusion:

Caused by: com.google.cloud.bigquery.BigQueryException: Exceeded rate limits: Your table exceeded quota for table.insert or table.update per table. For more information, see https://cloud.google.com/bigquery/troubleshooting-errors

at com.google.cloud.bigquery.spi.DefaultBigQueryRpc.translate(DefaultBigQueryRpc.java:93)
at com.google.cloud.bigquery.spi.DefaultBigQueryRpc.patch(DefaultBigQueryRpc.java:218)
at com.google.cloud.bigquery.BigQueryImpl$10.call(BigQueryImpl.java:329)
at com.google.cloud.bigquery.BigQueryImpl$10.call(BigQueryImpl.java:326)
at com.google.cloud.RetryHelper.doRetry(RetryHelper.java:179)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:244)
at com.google.cloud.bigquery.BigQueryImpl.update(BigQueryImpl.java:325)
at com.wepay.kafka.connect.bigquery.SchemaManager.updateSchema(SchemaManager.java:57)
at com.wepay.kafka.connect.bigquery.write.row.AdaptiveBigQueryWriter.performWriteRequest(AdaptiveBigQueryWriter.java:86)
at com.wepay.kafka.connect.bigquery.write.row.BigQueryWriter.writeRows(BigQueryWriter.java:158) <— catching the 403 was expected to happen around here.
at com.wepay.kafka.connect.bigquery.write.batch.DynamicBatchWriter.seekingWriteAll(DynamicBatchWriter.java:131)
... 7 more
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
{
  "code" : 403,
  "errors" : [ {
    "domain" : "global",
    "location" : "table.write",
    "locationType" : "other",
    "message" : "Exceeded rate limits: Your table exceeded quota for table.insert or table.update per table. For more information, see https://cloud.google.com/bigquery/troubleshooting-errors",
    "reason" : "rateLimitExceeded"
  } ],
  "message" : "Exceeded rate limits: Your table exceeded quota for table.insert or table.update per table. For more information, see https://cloud.google.com/bigquery/troubleshooting-errors"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.bigquery.spi.DefaultBigQueryRpc.patch(DefaultBigQueryRpc.java:213)
... 16 more

[mziccard]

Hi @mtagle!

I am not able to reproduce this on the latest version of google-cloud (0.5.1). Which version are you using?

I tried the following code:

String tableName = "test_update_table";
StandardTableDefinition tableDefinition = StandardTableDefinition.of(TABLE_SCHEMA);
TableInfo tableInfo = TableInfo.of(TableId.of(DATASET, tableName), tableDefinition);
Table createdTable = bigquery.create(tableInfo);
assertNotNull(createdTable);
for (int i = 0; i < 1001; i++) {
  try {
    Table updatedTable =
      bigquery.update(tableInfo.toBuilder().setDescription("newDescription").build());
    } catch (BigQueryException ex) {
    System.out.println("MESSAGE IS " + ex.getMessage());
    System.out.println("REASON IS " + ex.getReason());
    System.out.println("ERROR IS " + ex.getError());
    System.out.println("CODE IS " + ex.getCode());
    break;
  }
}

And I get the following output:

MESSAGE IS Exceeded rate limits: Your table exceeded quota for table.insert or table.update per table. For more information, see https://cloud.google.com/bigquery/troubleshooting-errors
REASON IS rateLimitExceeded
ERROR IS BigQueryError{reason=rateLimitExceeded, location=table.write, message=Exceeded rate limits: Your table exceeded quota for table.insert or table.update per table. For more information, see https://cloud.google.com/bigquery/troubleshooting-errors}
CODE IS 403

As you can see CODE is correctly populated.

[mtagle]

I'm using 0.2.7, but I think you are correct. I had assumed the issue was with the code but it appears that I was actually making incorrect assumptions about what the reason would be. I'll close this ticket. Thank you for your help!