-
Notifications
You must be signed in to change notification settings - Fork 8.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve degradation mechanism of RT-based circuit breaking #2123
base: master
Are you sure you want to change the base?
Improve degradation mechanism of RT-based circuit breaking #2123
Conversation
…ing previous buckets when the entry tries to pass (alibaba#1405) Signed-off-by: Jerry.Zhong <15951609026@163.com>
Signed-off-by: Jerry.Zhong <15951609026@163.com>
|
…edRt is too lesser than statIntervalMs. Signed-off-by: Jerry.Zhong <15951609026@163.com>
Signed-off-by: Jerry.Zhong <15951609026@163.com>
Signed-off-by: Jerry.Zhong <15951609026@163.com>
Any suggestions? @sczyh30 |
I'v re-triggered the CI. I'll take a review for this PR these days. Also cc @cdfive @jasonjoo2010 |
Describe what this PR does / why we need it
Improve degradation mechanism of RT-based circuit breaking, support more timely downgrades in some bad situations.
e.g. If something went wrong which may cause RT down to more than 10s (100ms for normal), the RT-based breaker will cut off the traffic after 10s.
Does this pull request fix one issue?
Fixes #1405
Describe how you did it
SlowRequestLeapArray
, ensurewindowLengthInMs
is fixed tostatIntervalMs
andintervalInMs
is just happen to greater thanmaxAllowedRt
.SlowRequestCounter
inSlowRequestLeapArray
should count inflight entries, increment counts when entries try pass and decrement counts when entries exit. EachSlowRequestCounter
should track the predecessor deprecated bucket when it reset.SlowRequestCounter
s in the deprecated buckets just once.maxAllowedRt
, then check currentSlowRequestCounter
.Status of the
SlowRequestCounter
CHECKED_BY_SLOW:indicate current count has been checked by slowly exit or a no successor entry.Normal status flow : UNCHECKED -> CHECKED_BY_ENTRY -> UNCHECKED -> CHECKED_BY_ENTRY
Slow RT status flow:UNCHECKED -> CHECKED_BY_ENTRY -> CHECKED_BY_SLOWSimplify the status control (2021-04-19)
Describe how to verify it
Run the test cases.
Special notes for reviews
Should consider is it works well at some concurrency scenarios not considered.