Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bilibili调度新增回避策略 #573

Merged
merged 15 commits into from
Aug 4, 2024
Merged

Bilibili调度新增回避策略 #573

merged 15 commits into from
Aug 4, 2024

Conversation

AzideCupric
Copy link
Collaborator

原先的30s对叔叔而言还是太敏感了

Copy link

netlify bot commented Jun 18, 2024

Deploy Preview for nonebot-bison ready!

Name Link
🔨 Latest commit 38dd0d4
🔍 Latest deploy log https://app.netlify.com/sites/nonebot-bison/deploys/66ab051dfcdedf0008c360a1
😎 Deploy Preview https://deploy-preview-573--nonebot-bison.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link

codecov bot commented Jun 18, 2024

Codecov Report

Attention: Patch coverage is 94.53125% with 14 lines in your changes missing coverage. Please review.

Project coverage is 85.39%. Comparing base (af470f2) to head (38dd0d4).
Report is 3 commits behind head on main.

Files Patch % Lines
nonebot_bison/platform/bilibili/retry.py 94.20% 8 Missing ⚠️
nonebot_bison/platform/bilibili/fsm.py 95.41% 5 Missing ⚠️
nonebot_bison/platform/bilibili/scheduler.py 85.71% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #573      +/-   ##
==========================================
+ Coverage   85.01%   85.39%   +0.38%     
==========================================
  Files          88       90       +2     
  Lines        4491     4718     +227     
==========================================
+ Hits         3818     4029     +211     
- Misses        673      689      +16     
Flag Coverage Δ
smoke-test 85.39% <94.53%> (+0.38%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@AzideCupric AzideCupric changed the title 🐛 将Bilibili的调度速度降低到60s 🐛 将Bilibili的调度间隔降低到60s Jun 18, 2024
@AzideCupric AzideCupric changed the title 🐛 将Bilibili的调度间隔降低到60s 🐛 将Bilibili的调度间隔减慢到60s Jun 18, 2024
@AzideCupric AzideCupric changed the title 🐛 将Bilibili的调度间隔减慢到60s 🐛 Bilibili调度新增回避策略 Jun 20, 2024
@AzideCupric AzideCupric force-pushed the fix/bilibili-slower branch 4 times, most recently from f3bfa96 to 822622d Compare June 21, 2024 18:02
@AzideCupric
Copy link
Collaborator Author

AzideCupric commented Jun 23, 2024

设想的状态转换图

stateDiagram-v2
	direction LR
    [*] --> NORMAL
    NORMAL --> NORMAL: 请求成功
    NORMAL --> REFRESH: 请求失败
    REFRESH --> NORMAL: 请求成功
    REFRESH --> REFRESH: 请求失败
    REFRESH --> BACKOFF: 请求失败 若重试满但回避未满
    REFRESH --> RAISE: 请求失败 若重试满且回避满
    BACKOFF --> BACKOFF: 回避中
    BACKOFF --> REFRESH: 回避中 若已超时
    RAISE --> RAISE: 请求失败
    RAISE --> NORMAL: 请求成功
Loading

修改记录:
检测达到最大BACKOFF次数应在REFRESH状态进行
增加条件控制

@AzideCupric AzideCupric force-pushed the fix/bilibili-slower branch 3 times, most recently from bef83cb to ba3df50 Compare June 28, 2024 21:04
@AzideCupric
Copy link
Collaborator Author

用生成器正经搓了个小状态机,感觉这样看起来更清晰一点
cookie pool 这几天测试情况来看没必要加,先删了(留给ospp吧

@AzideCupric AzideCupric requested a review from felinae98 June 28, 2024 21:06
return waited < should_wait


B = TypeVar("B", bound="Bilibili")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个是不是没啥必要

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不这样写的话retry_for_352的类型得全用引号包起来(

@AzideCupric AzideCupric force-pushed the fix/bilibili-slower branch from a879824 to d5bd87f Compare July 16, 2024 16:44
@AzideCupric AzideCupric requested a review from felinae98 July 16, 2024 16:50


class RetryState(StrEnum):
NROMAL = "normal"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NORMAL

self.graph = graph
self.current_state = graph["initial"]
self.machine = self.core()
self.ctx = ctx
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个叫 additional state 吧

self.machine = self.core()
self.ctx = ctx

async def core(self) -> AsyncGenerator[ActionReturn, TEvent]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
async def core(self) -> AsyncGenerator[ActionReturn, TEvent]:
async def _core(self) -> AsyncGenerator[ActionReturn, TEvent]:

await fsm.emit(RetryEvent.REACH_MAX_REFRESH)
case RetryActionReturn.BACKOFF_MAX:
logger.error("指数回避次数达到上限,放弃回避")
await fsm.emit(RetryEvent.REACH_MAX_BACKOFF)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个地方太奇怪了,要根据 fsm 返回的结果 emit 特定事件,你这个自动机改叫手动机好了

@AzideCupric AzideCupric mentioned this pull request Jul 30, 2024
2 tasks
@felinae98 felinae98 added the bug Something isn't working label Aug 4, 2024
@felinae98 felinae98 changed the title 🐛 Bilibili调度新增回避策略 Bilibili调度新增回避策略 Aug 4, 2024
@felinae98 felinae98 merged commit 38f0edc into main Aug 4, 2024
35 checks passed
@felinae98 felinae98 deleted the fix/bilibili-slower branch August 4, 2024 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants