[S-LoRA] Something is wrong with the s-lora arguments in server_api

[sleepwalker2017]

I notice that there is no --lora-modules argument in the vllm.entrypoints.api_server, which means I must add the lora local path when sending request.

That's unrealistic. Because the client doesn't know the lora path.

Any plan to fix it ?

[simon-mo]

Please use the vllm.entrypoints.openai.apI_server instead, which have similar functionality and OpenAI compatible API.

[sleepwalker2017]

Please use the vllm.entrypoints.openai.apI_server instead, which have similar functionality and OpenAI compatible API.

thank you. Is there any script to benchmark the multiple lora serving?

[simon-mo]

I don't believe we have a full benchmark on LoRA. Please check out what we have in benchmarks/ in the repo and contribution welcomed!

[sleepwalker2017]

I don't believe we have a full benchmark on LoRA. Please check out what we have in benchmarks/ in the repo and contribution welcomed!

ok, I find some performance issues and post it here #3219.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

[github-actions]

This issue has been automatically closed due to inactivity. Please feel free to reopen if you feel it is still relevant. Thank you!