Skip to content

Commit

Permalink
[Bugfix] Fix phi3v incorrect image_idx when using async engine (#7916)
Browse files Browse the repository at this point in the history
  • Loading branch information
Isotr0py authored Aug 27, 2024
1 parent 42e932c commit b09c755
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion vllm/model_executor/models/phi3v.py
Original file line number Diff line number Diff line change
Expand Up @@ -422,7 +422,9 @@ def input_processor_for_phi3v(ctx: InputContext, llm_inputs: LLMInputs):

prompt = llm_inputs.get("prompt")
if prompt is None:
image_idx = []
# for async server request, we assume prompt and its token_ids is always
# in correct format. And num_image_tags == len(image_data) always True.
image_idx = range(1, len(image_data) + 1)
new_prompt = None
else:
image_idx = sorted(map(int, re.findall(r"<\|image_(\d+)\|>+", prompt)))
Expand Down

0 comments on commit b09c755

Please sign in to comment.