spanner.cloud-client.backup_sample_test: test_delete_backup failed

[flaky-bot]

Note: #3242 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.

---

commit: c9bf218
buildURL: Build Status, Sponge
status: failed

Test output

Traceback (most recent call last):
  File "/workspace/spanner/cloud-client/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 57, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/workspace/spanner/cloud-client/.nox/py-3-7/lib/python3.7/site-packages/grpc/_channel.py", line 826, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/workspace/spanner/cloud-client/.nox/py-3-7/lib/python3.7/site-packages/grpc/_channel.py", line 729, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.NOT_FOUND
	details = "Backup not found: projects/python-docs-samples-tests/instances/test-instance-916d2de66a/backups/test-backup-475a65b3b5"
	debug_error_string = "{"created":"@1594982191.742561967","description":"Error received from peer ipv4:142.250.107.95:443","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Backup not found: projects/python-docs-samples-tests/instances/test-instance-916d2de66a/backups/test-backup-475a65b3b5","grpc_status":5}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):

File "/workspace/spanner/cloud-client/backup_sample_test.py", line 99, in test_delete_backup

backup_sample.delete_backup(INSTANCE_ID, BACKUP_ID)

File "/workspace/spanner/cloud-client/backup_sample.py", line 206, in delete_backup

backup.reload()

File "/workspace/spanner/cloud-client/.nox/py-3-7/lib/python3.7/site-packages/google/cloud/spanner_v1/backup.py", line 228, in reload

pb = api.get_backup(self.name, metadata=metadata)

File "/workspace/spanner/cloud-client/.nox/py-3-7/lib/python3.7/site-packages/google/cloud/spanner_admin_database_v1/gapic/database_admin_client.py", line 1208, in get_backup

request, retry=retry, timeout=timeout, metadata=metadata

File "/workspace/spanner/cloud-client/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/gapic_v1/method.py", line 143, in call

return wrapped_func(*args, **kwargs)

File "/workspace/spanner/cloud-client/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/retry.py", line 286, in retry_wrapped_func

on_error=on_error,

File "/workspace/spanner/cloud-client/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/retry.py", line 184, in retry_target

return target()

File "/workspace/spanner/cloud-client/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/timeout.py", line 214, in func_with_timeout

return func(*args, **kwargs)

File "/workspace/spanner/cloud-client/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable

six.raise_from(exceptions.from_grpc_error(exc), exc)

File "", line 3, in raise_from

google.api_core.exceptions.NotFound: 404 Backup not found: projects/python-docs-samples-tests/instances/test-instance-916d2de66a/backups/test-backup-475a65b3b5

[flaky-bot]

Looks like this issue is flaky. 😟

I'm going to leave this open and stop commenting.

A human should fix and close this.

---

When run at the same commit (c9bf218), this test passed in one build (Build Status, Sponge) and failed in another build (Build Status, Sponge).

[tmatsuo]

@larkee Can you take a look?

There are several of similar issues, but the root cause is the same. DeadlineExceeded on CreateBackup

[tmatsuo]

Let's close this and bump the deadline if it happens too often.