Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add retry on trace not found #57

Merged
merged 2 commits into from
Oct 17, 2023
Merged

Add retry on trace not found #57

merged 2 commits into from
Oct 17, 2023

Conversation

jysheng123
Copy link
Contributor

@jysheng123 jysheng123 commented Oct 12, 2023

Issue number:

Summary

Add functionality for retrying on trace not found for fetch_trace_tree_until

Changes

Add a generic exception class called retryable exceptions that will be called so the retry decorator can retry on those exceptions. When we encounter a the common error trace not found it raises the retryable error so functions that are retrying, it will instead continue retrying instead of exiting because of the error

User experience

Mandatory Checklist

If your change doesn't seem to apply, please leave them unchecked.

  • I have performed a self-review of this change
  • Changes have been tested
  • Changes are documented

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@jysheng123 jysheng123 requested a review from a team as a code owner October 12, 2023 23:01
)
return response
except ZionException as e:
if "trace not found" in str(e):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may move this functionality into the RPC side in the future, for now this will work for the specific error for trace not found which is a commonly encountered error. This is a stop gap measure until we decide to refactor our codebase to accommodate all the error types for retry.

self.assertNotEqual(self.counter, 10)
self.assertGreater(end - start, 10)
self.assertEqual(e, "Method not found")
self.assertNotEqual(self.counter, 10)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some previous integration tests did not actually check error message because they were inside the error checking flow, moved them outside so it properly checks now

Co-authored-by: Wing Fung Lau <4760060+hawflau@users.noreply.github.com>
self.assertEqual(e, "condition is not a callable function")
self.assertEqual(str(e.value), "condition is not a callable function")

def test_retry_trace_not_found(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how is this test testing trace not found?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it because the tracing_heading is non-existent?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I just put in a random valid trace id format but it doesnt actually exist.

@jysheng123 jysheng123 added this pull request to the merge queue Oct 17, 2023
Merged via the queue into develop with commit 3c72828 Oct 17, 2023
jysheng123 added a commit that referenced this pull request Oct 17, 2023
* add retry on trace not found

* Update python-client/src/zion/__init__.py

Co-authored-by: Wing Fung Lau <4760060+hawflau@users.noreply.github.com>

---------

Co-authored-by: Wing Fung Lau <4760060+hawflau@users.noreply.github.com>
@jfuss jfuss deleted the retry-trace-not-found branch October 31, 2023 19:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants