Skip to content

Commit

Permalink
TEMP: no shared tokenizer from PR-3512
Browse files Browse the repository at this point in the history
Gotta get conflicts with the main line figured resolved first. Just
don't use the shared tokenizer for now.

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
  • Loading branch information
tjohnson31415 authored and joerunde committed May 20, 2024
1 parent 79cc739 commit 64b18a7
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion vllm/entrypoints/grpc/grpc_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,8 @@ def __init__(self, engine: AsyncLLMEngine, args: argparse.Namespace):

async def _post_init(self):
self.config = await self.engine.get_model_config()
self.tokenizer_group = await self.engine.get_tokenizer_group()
# self.tokenizer_group = await self.engine.get_tokenizer_group()
self.tokenizer_group = self.engine.engine.tokenizer
self.tokenizer = await self.engine.get_tokenizer()

# Swap in the special TGIS stats logger
Expand Down

0 comments on commit 64b18a7

Please sign in to comment.