Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(meta): show recovery cause when request is rejected #13836

Merged
merged 1 commit into from
Dec 6, 2023

Conversation

BugenZhao
Copy link
Member

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

When a DDL request is sent to the meta service which is under failure recovery, the failure cause will be included in the error message as well. Thanks to the progress made on #11443, all we need to do is to correctly return the error in meta.

ERROR:  Failed to run the query

Caused by these errors (recent errors listed first):
  1: gRPC request to meta service failed: Internal error
  2: The cluster is recovering
  3: gRPC request to stream service failed: Internal error
  4: Actor 4 exited unexpectedly
  5: Executor error
  6: Chunk operation error
  7: Division by zero

It's worth noting that after #12999, the message will rarely be seen by users as DROP is not rejected. But I guess it's still a good-to-have.

Also I don't find a way to test it in CI, as we cannot manually trigger a failure recovery now (or we have to depend on some unresolved bugs 😄, like the examples above). Perhaps we can introduce a debug-only statement to manually trigger one.

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • All checks passed in ./risedev check (or alias, ./risedev c)

Documentation

  • My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Copy link
Member

@yezizp2012 yezizp2012 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

codecov bot commented Dec 6, 2023

Codecov Report

Attention: 12 lines in your changes are missing coverage. Please review.

Comparison is base (a8aa905) 68.28% compared to head (33d9b0e) 68.28%.
Report is 3 commits behind head on main.

Files Patch % Lines
src/meta/src/barrier/mod.rs 33.33% 8 Missing ⚠️
src/meta/service/src/scale_service.rs 0.00% 3 Missing ⚠️
src/meta/src/rpc/ddl_controller.rs 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #13836      +/-   ##
==========================================
- Coverage   68.28%   68.28%   -0.01%     
==========================================
  Files        1525     1525              
  Lines      262324   262313      -11     
==========================================
- Hits       179137   179118      -19     
- Misses      83187    83195       +8     
Flag Coverage Δ
rust 68.28% <25.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants