Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(serverless-hub) Added re-try logic to gcp calls #2111

Merged
merged 1 commit into from
Oct 27, 2020

Conversation

chrismaree
Copy link
Member

Motivation

GCP sometimes returns flaky errors when calling GCP datastore. This occurs when reading to and writing from the last used blocknumber for the monitor bot & normally produced the following error:

image

The issue is currently discussed on GCP GitHub issues here: googleapis/nodejs-datastore#679

For now, this PR adds in a small change to the monitor hub to wrap the data retrieval and stage for the block number within re-try logic to try and mitigate this error. This is not a long term fix but aims to make the error less irritating in the interim until there is an official fix for GCP data store within GCP cloud run.

Summary

Adds async-retry logic to the datastore module.

Signed-off-by: Christopher Maree <christopher.maree@gmail.com>
@chrismaree chrismaree requested review from daywiss and mrice32 October 26, 2020 10:33
@coveralls
Copy link

Coverage Status

Coverage remained the same at 93.829% when pulling 98bfe33 on chrismaree/serverless-hub-retry-logic into 892042c on master.

Copy link
Contributor

@daywiss daywiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hope this works!

Copy link
Member

@mrice32 mrice32 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! This is awesome! And a pretty elegant change tbh

@chrismaree chrismaree merged commit 3e6051b into master Oct 27, 2020
@chrismaree chrismaree deleted the chrismaree/serverless-hub-retry-logic branch October 27, 2020 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants