Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UI can be overwhelmed when streaming a huge response that's coming in too fast #6722

Open
1 task done
TiagoTiago opened this issue Feb 2, 2025 · 0 comments
Open
1 task done
Labels
bug Something isn't working

Comments

@TiagoTiago
Copy link

TiagoTiago commented Feb 2, 2025

Describe the bug

I just had a spooky moment; I asked something to the LLama 8B distill of Deepseek R1 that made it go into a very long train of thought. After a while I realized it was going in the wrong direction and I needed to reformulate the question to make clear what I was asking about, but the stop button didn't work. I gave it some time, and then when it still didn't stop after additional clicks of the stop button and a little more waiting, I decided to go to the terminal and Ctrl-C the whole thing. But coming back to the browser the damn thing was still "typing" the response!

Only when I reloaded the page it finally showed that I had indeed terminated the process with the Ctrl-C.

Now after I booted up the web UI again, turns out it already had the whole response in the log, so I was actually not waiting the generation to finish, but for the page to finish the "typing animation" of the huge text it already had even on disk.

Is there an existing issue for this?

  • I have searched the existing issues

Reproduction

Not sure the exact steps that will guarantee a repro, but the description of the event given above probably has enough clues.

Screenshot

No response

Logs

.

System Info

.
@TiagoTiago TiagoTiago added the bug Something isn't working label Feb 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant