Skip to content

Commit

Permalink
[Misc] Fix minor typo in scheduler (vllm-project#8765)
Browse files Browse the repository at this point in the history
  • Loading branch information
wooyeonlee0 authored and MengqingCao committed Sep 30, 2024
1 parent 2bff8c8 commit e492915
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions vllm/core/scheduler.py
Original file line number Diff line number Diff line change
Expand Up @@ -1554,14 +1554,14 @@ def _get_num_new_tokens(self, seq_group: SequenceGroup,
# the number of new tokens that is dividable by the block size
# to avoid partial block matching.
block_size = self.cache_config.block_size
reminder = budget.token_budget % block_size
if reminder != 0:
remainder = budget.token_budget % block_size
if remainder != 0:
raise ValueError("When enabling chunked prefill and "
"prefix caching, max_num_batched_tokens "
"(chunk size) must be dividable by "
"block size, but got chunk_size "
f"({budget.token_budget}) % block_size "
f"({block_size}) = {reminder}")
f"({block_size}) = {remainder}")
if remaining_token_budget < num_new_tokens:
num_new_tokens = (remaining_token_budget //
block_size) * block_size
Expand Down

0 comments on commit e492915

Please sign in to comment.