Skip to content

Commit

Permalink
docs: Justify non-generic filenames for adapter tensors and json
Browse files Browse the repository at this point in the history
  • Loading branch information
sangstar committed Dec 11, 2024
1 parent d638acc commit 480404e
Showing 1 changed file with 6 additions and 7 deletions.
13 changes: 6 additions & 7 deletions vllm/model_executor/model_loader/tensorizer.py
Original file line number Diff line number Diff line change
Expand Up @@ -477,14 +477,13 @@ def tensorize_vllm_model(engine_args: EngineArgs,

def tensorize_lora_adapter(lora_path: str,
tensorizer_config: TensorizerConfig):

# TODO: Consider if the model tensors are in an initial format
# other than safetensors, such as .pt
# TODO: Is there a guarantee the adapter_model and adapter_config
# prefixes are the ones that will always used? Probably allow
# specifying paths for these files

lora_files = snapshot_download(repo_id=lora_path)

# Current LoRA loading logic in
# vllm.lora.models.LoRAModel.from_local_checkpoint assumes that
# the tensors and config filenames are adapter_model.safetensors and
# adapter_config.json respectively, so this logic makes the same
# assumption
tensor_path = os.path.join(lora_files, "adapter_model.safetensors")
config_path = os.path.join(lora_files, "adapter_config.json")
with open(config_path) as f:
Expand Down

0 comments on commit 480404e

Please sign in to comment.