Skip to content

Commit

Permalink
[BugFix] Ensure worker model loop is always stopped at the right time (
Browse files Browse the repository at this point in the history
  • Loading branch information
njhill authored Jul 1, 2024
1 parent c4059ea commit c87ebc3
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion vllm/engine/llm_engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -838,7 +838,7 @@ def step(self) -> List[Union[RequestOutput, EmbeddingRequestOutput]]:
# Tracing
self.do_tracing(scheduler_outputs)

if not request_outputs:
if not self.has_unfinished_requests():
# Stop the execute model loop in parallel workers until there are
# more requests to process. This avoids waiting indefinitely in
# torch.distributed ops which may otherwise timeout, and unblocks
Expand Down

0 comments on commit c87ebc3

Please sign in to comment.