[Feature]: Force enable fake streaming per model

[Manouchehri]

The Feature

I would like to disable streaming on a per-model basis in my proxy yaml config, and have LiteLLM do a non-streaming request upstream (while returning a fake-streaming response to the client).

Basically what was mentioned here: #61 (comment)

Motivation, pitch

When streaming on OpenAI, their moderation filter makes the responses insanely slow. Like, I sometimes timeout after >10 minutes.

I don't want to change my client-side code, as I do want other providers (like Azure OpenAI) in the same model group to stream (as the perf impact isn't as bad).

Twitter / LinkedIn details

https://www.linkedin.com/in/davidmanouchehri/

[mikewirth]

This would be useful for me as well, as I'm trying to use https://mockgpt.wiremock.io/ through LiteLLM.

MockGPT doesn't support streaming, but I want to use it when load testing to replace GPT-4o. I can't control if my client application sends stream=True to LiteLLM.

[tan-yong-sheng]

Facing similar problem when I try to use GitHub/o1-mini on openwebui... I hope I could disable the stream per model basis.