You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We've relied on recursive cancellation of tasks in Ray Core, but that doesn't fully cover recursive cancellation of requests in Ray Serve. If a request is queued and hasn't been sent off to a replica yet at the time of the parent request's cancellation, then it will remain in the queue and eventually be sent to a downstream replica, i.e. the recursive cancellation will fail.
We should fix this by implementing recursive cancellation at the Serve layer.
The text was updated successfully, but these errors were encountered:
We've relied on recursive cancellation of tasks in Ray Core, but that doesn't fully cover recursive cancellation of requests in Ray Serve. If a request is queued and hasn't been sent off to a replica yet at the time of the parent request's cancellation, then it will remain in the queue and eventually be sent to a downstream replica, i.e. the recursive cancellation will fail.
We should fix this by implementing recursive cancellation at the Serve layer.
The text was updated successfully, but these errors were encountered: