[Misc] Update benchmark_prefix_caching.py fixed example usage (vllm-p…

…roject#11920) Signed-off-by: Ren MinMin <renmm6@chinaunicom.cn> Co-authored-by: Ren MinMin <renmm6@chinaunicom.cn> Signed-off-by: Bowen Wang <abmfy@icloud.com>
abmfy · Jan 24, 2025 · 8a16c31 · 8a16c31
1 parent f075824
commit 8a16c31
Showing 1 changed file with 2 additions and 1 deletion.
diff --git a/benchmarks/benchmark_prefix_caching.py b/benchmarks/benchmark_prefix_caching.py
@@ -10,7 +10,8 @@
         --model meta-llama/Llama-2-7b-chat-hf \
         --enable-prefix-caching \
         --num-prompts 1 \
-        --repeat-count 100
+        --repeat-count 100 \
+        --input-length-range 128:256
 
 ShareGPT example usage:
     # This command samples 20 prompts with input lengths