[Request]: Implement Streaming Responses

[ericrallen]

Is your feature request related to a problem? Please describe.

Responses take too long to deliver to the UI.

Describe the solution you'd like

Now that OpenAI has added streaming support to the openai npm package, switch to streaming responses to the UI.

[yigitkonur]

There hasn't been any improvement in this repo for the past few months - I hope you haven't lost your motivation. I think what you're doing in this field is truly valuable. I hope you create a way for us to boost your motivation here by allowing us to contribute as sponsors <3

[ericrallen]

@yigitkonur I really appreciate the kind words. It’s awesome to know that someone is using and enjoying the stuff I’ve built.

I promise that I haven’t abandoned AI Research Assistant - I still use it almost daily and plan to have some updates ready to release after the next couple of weeks are over.

I still have plans to add streaming responses, but it requires a little more work than just enabling the stream: true option because when I started this project there weren’t SDKs for the API, the chat completions endpoint didn’t exist, and there were very few examples of how to handle steaming responses, so the chat logic needs some refactoring to support properly rendering chunks.

It’s been just over a month since v 1.5.0 - which added the GPT-4 model and hot-swapping models during a conversation - was released.

Since then I’ve taken a much needed vacation, and it’s been a busy time where I’ve had a lot of other work to take care of like:

• building a demo AI app for a talk for Prompt Engineering Conference
• contributing a few features to Open Interpreter
• updating another AI demo and it’s corresponding talk for All Things Open
• learning Python and how to use Jupyter Notebooks via an interactive demo

Contributing to Open Interpreter has helped me consider and reprioritize some features for AI Research Assistant, but I’d love to know what my users want and need.

Please feel encouraged to provide suggestions for features as Issues or start a Discussion so I know what folks are looking for from AI Research Assistant.

[yigitkonur]

Great developments, Eric - I certainly don't underestimate your effort, you're producing excellent work. I can imagine that implementing the stream section wasn't easy - it seems like you've been through a busy period and you're still producing great results. As someone who believes in the future of this project, I wouldn't want you to feel any pressure on yourself, I just want to show my support 🙏

[github-actions]

🎉 This issue has been resolved in version 1.6.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

[ericrallen]

@yigitkonur Streaming responses from OpenAI should be available in the latest version of AI Research Assistant.