Skip to content

Commit

Permalink
fix: chunked prefill with v2 block manager (#679)
Browse files Browse the repository at this point in the history
  • Loading branch information
AlpinDale committed Sep 10, 2024
1 parent 3bbb3f2 commit 79d6039
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions aphrodite/processing/block/block_table.py
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,12 @@ def _chunk_token_blocks_for_append(
appended to blocks. The first such "token block" may have less token ids
than the block size, since the last allocated block may be partially
full.
If no token ids are provided, then no chunks are returned.
"""

if not token_ids:
return []

first_chunk_size = self._block_size - (self._num_full_slots %
self._block_size)
token_blocks = [token_ids[:first_chunk_size]]
Expand Down

0 comments on commit 79d6039

Please sign in to comment.