Skip to content

Commit

Permalink
[Bugfix] Fix phi3v incorrect image_idx when using async engine (vllm-…
Browse files Browse the repository at this point in the history
  • Loading branch information
Isotr0py authored and triple-Mu committed Sep 4, 2024
1 parent a4ec497 commit 97bb17e
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion vllm/model_executor/models/phi3v.py
Original file line number Diff line number Diff line change
Expand Up @@ -422,7 +422,9 @@ def input_processor_for_phi3v(ctx: InputContext, llm_inputs: LLMInputs):

prompt = llm_inputs.get("prompt")
if prompt is None:
image_idx = []
# for async server request, we assume prompt and its token_ids is always
# in correct format. And num_image_tags == len(image_data) always True.
image_idx = range(1, len(image_data) + 1)
new_prompt = None
else:
image_idx = sorted(map(int, re.findall(r"<\|image_(\d+)\|>+", prompt)))
Expand Down

0 comments on commit 97bb17e

Please sign in to comment.