Skip to content

Commit

Permalink
[Doc] Add note about context length in Phi-3-Vision example (vllm-pro…
Browse files Browse the repository at this point in the history
  • Loading branch information
DarkLight1337 authored and prashantgupta24 committed Jul 1, 2024
1 parent 1127c34 commit dc97246
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion examples/phi3v_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,16 @@

def run_phi3v():
model_path = "microsoft/Phi-3-vision-128k-instruct"

# Note: The model has 128k context length by default which may cause OOM
# If that's the case, override `max_model_len` with a smaller value via args
llm = LLM(
model=model_path,
trust_remote_code=True,
image_input_type="pixel_values",
image_token_id=32044,
image_input_shape="1,3,1008,1344",
image_feature_size=1921,
disable_image_processor=False,
)

image = Image.open("images/cherry_blossom.jpg")
Expand Down

0 comments on commit dc97246

Please sign in to comment.