Make chat default to streaming=true for output

[acherla]

By default it appears the chat interface does not stream out responses in the chat UI for a team. This results in relatively long wait times to fetch a response. Set it to automatically use streaming=true by default to fetch responses in chunks.

[StreetLamb]

Hi @acherla, responses should already be streaming if the model provider supports it. Could you share what model you are using?

[acherla]

Ollama3.1

[StreetLamb]

Based on the blog post from ollama, streaming is not supported yet when tool calling. However, if you try without tools, the responses should be streamed.

Future improvements
Streaming tool calls: stream tool calls back to begin taking action faster when multiple tools are returned
Tool choice: force a model to use a tool

[acherla]

Thanks good to know. Ill close this until its supported

[acherla]

Seems like the root crux of the issue is streaming the state graph in general. While i dont expect stream tool calls to be supported, it seems like it waits till the entire graph dag executes before outputting the response which seems unintentional.