Support pending/loading message while waiting for response

[michaelchia]

Description

Add support for a pending or loading message to be displayed to let the user know that the request is being processed. Provides updates for the progress on multi-step pipelines.

Fixes #176.

Demo

Screen.Recording.2024-06-07.at.6.46.31.PM.mov

Features

• Is controlled from the respective ChatHandler on the server side.
• Text can be configured
• Supports sequential step-by-step messages before a response
  • E.g. for a RAG chain is may show "Retrieving documents" followed by "Generating answer" once the retrieval step is done.
• Supports multiple concurrent messages
  • E.g. showing a message for each pending concurrent async task

Usage

This first draft contains some example usage in a few default chat handlers. I would expect the team to modify the message to their liking.

To display a pending message, you can use the BaseChatHandler.pending(text: str, ellipsis: bool = True) -> PendingMessage context manager method to start and remove the pending message.

This was used in default to provide a 'Thinking...' message
https://github.com/michaelchia/jupyter-ai/blob/d499edaab60e60192712ab96e4331c96e07deab1/packages/jupyter-ai/jupyter_ai/chat_handlers/default.py#L61-L65

Similarly in /ask
https://github.com/michaelchia/jupyter-ai/blob/d499edaab60e60192712ab96e4331c96e07deab1/packages/jupyter-ai/jupyter_ai/chat_handlers/ask.py#L75-L78

Used in /learn to replace the static reply message
https://github.com/michaelchia/jupyter-ai/blob/d499edaab60e60192712ab96e4331c96e07deab1/packages/jupyter-ai/jupyter_ai/chat_handlers/learn.py#L154-L167

Instead of the context manager, users can manually start the pending message with BaseChatHandler.start_pending(text: str, ellipsis: bool = True) -> PendingMessage and close it by calling BaseChatHandler.close_pending(message: PendingMessage) -> None.

Implemenation summary

• 2 new message types were added PendingMessage and ClosePendingMessage.
• pending start_pending and close_pending where added to BaseChatHandler
• components/chat.tsx modified to handle the new message types
  • array of PendingMessage maintained following how chat messages was done.
  • PendingMessages element added just below ChatMessages
• PendingMessages element implemented in compontents/pending-messages.tsx as a modified version of compontents/chat-messages.tsx

Next steps

If this idea is accepted by the team I would need some help cleaning up so it adheres to the product standards.

• Feel free to rename "pending message" to something else.
  • I chose amongst "pending", "loading" and "temp"
• You would probably want to reimplement the JSX Element stuff.
  • I just hacked it together to get a working demo. I have very limited experience with React or any frontend stuff.
  • I did some hacky stuff like creating a dummy AgentChatMessage so that I could reuse ChatMessageHeader
  • Consider doing some styling on pending text so it looks different from normal chat text
  • Consider changing from animated ellipsis to some kind of spinner or loading widget if it looks better.

Let me know what you guys think and if there is anything you would like me to add.

[JasonWeill]

Thank you so much for opening this! When it's ready for review, please add "Fixes #176" to the description, so that we can close this long-running enhancement request.

[michaelchia]

I've done whatever I want to for now but definitely a bit needs work for the fontend part. I would need help with that. Would you like me a mark it ready for review first before you guys take a look or would you guys want to provide some comments for me to fix stuff and only mark it when it is at a more ready state?

[dlqqq]

@michaelchia Thank you for this contribution! Feel free to mark it as ready for review if you're seeking reviews. I'll have time to read through this on Monday.

[krassowski]

Maybe the frontend part should be contributed to https://github.com/jupyterlab/jupyter-chat and reused here?

[dlqqq]

@michaelchia Thank you for this PR! This would be a wonderful addition to Jupyter AI. I circled this PR around our team, and we are in alignment with the UX you chose. I will review this tomorrow as soon as I get an opportunity and fast-track this PR for merge & release.

Apologies for the delay in my review; our team is mostly out-of-office and I am currently cutting a new Jupyter AI release.

[dlqqq]

@michaelchia Awesome work! I love the context manager implementation you built in the Python backend. I left a few points of feedback concerning the frontend implementation below.

[dlqqq]

@michaelchia Thank you for working on this so quickly! I have a few more minor callouts and this PR should be good-to-go.

[dlqqq]

@michaelchia I left a couple more points of feedback above. No rush at all, just writing this as a reminder in case you missed my update on Friday. 😄

[dlqqq]

@michaelchia Awesome work, thank you very much for contributing and addressing my review feedback! I pushed a couple of commits that 1) simplify the rendered element and 2) match the font size of Generating response... to other Jupyternaut messages. Tested locally and everything is working as expected.

Really excited to include this in the next release! 🎉

Demo screenshot