FEAT: support scheduling-policy for vllm

[hwzhuhao]

since version 0.6.3, the AsyncLLMEngine in vllm supports priority scheduling (https://github.com/vllm-project/vllm/pull/8850)，so add a new parameter "scheduling-policy" for the vllm model config to support priority scheduling.

[qinxuye]

Oh, I wrongly thought this solved #2669 , besides the scheduling policy, we can support priority as well right?

[hwzhuhao]

yes, but need to set the scheduling policy to priority, the default value is fcfs

[qinxuye]

yes, but need to set the scheduling policy to priority, the default value is fcfs

OK, can you add the priority option in this PR?

[hwzhuhao]

user can set the scheduling policy to priority by passing custom parameters when launching models with the vllm inference engine, whether through the interface or the command line. I didn't quite understand the specific meaning for add the priority option. Could you please elaborate further?

[qinxuye]

user can set the scheduling policy to priority by passing custom parameters when launching models with the vllm inference engine, whether through the interface or the command line. I didn't quite understand the specific meaning for add the priority option. Could you please elaborate further?

Indeed, the priority is an generation option, never mind, let’s check it later.

[qinxuye]

LGTM