Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[serve] recursive cancellation #47935

Open
zcin opened this issue Oct 8, 2024 · 0 comments
Open

[serve] recursive cancellation #47935

zcin opened this issue Oct 8, 2024 · 0 comments
Assignees
Labels
ray-team-created Ray Team created serve Ray Serve Related Issue

Comments

@zcin
Copy link
Contributor

zcin commented Oct 8, 2024

We've relied on recursive cancellation of tasks in Ray Core, but that doesn't fully cover recursive cancellation of requests in Ray Serve. If a request is queued and hasn't been sent off to a replica yet at the time of the parent request's cancellation, then it will remain in the queue and eventually be sent to a downstream replica, i.e. the recursive cancellation will fail.

We should fix this by implementing recursive cancellation at the Serve layer.

@zcin zcin self-assigned this Oct 8, 2024
@zcin zcin added serve Ray Serve Related Issue ray-team-created Ray Team created labels Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ray-team-created Ray Team created serve Ray Serve Related Issue
Projects
None yet
Development

No branches or pull requests

1 participant