Add ability add a single message from the bot/user side #3165

dawoodkhan82 · 2023-02-09T23:24:32Z

Description

Add ability add a single message from the bot/user side. Plus add ability to specify if the chat starts with bot or user.
gr.Chatbot([("Hi, I'm DialoGPT. Try asking me a question.", None)], starts_with="bot")

Please include:

relevant motivation
a summary of the change
which issue is fixed.
any additional dependencies that are required for this change.

Closes: #3092

Checklist:

I have performed a self-review of my own code
I have added a short summary of my change to the CHANGELOG.md
My code follows the style guidelines of this project
I have commented my code in hard-to-understand areas
I have made corresponding changes to the documentation
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes

A note about the CHANGELOG

Hello 👋 and thank you for contributing to Gradio!

All pull requests must update the change log located in CHANGELOG.md, unless the pull request is labeled with the "no-changelog-update" label.

Please add a brief summary of the change to the Upcoming Release > Full Changelog section of the CHANGELOG.md file and include
a link to the PR (formatted in markdown) and a link to your github profile (if you like). For example, "* Added a cool new feature by [@myusername](link-to-your-github-profile) in [PR 11111](https://github.com/gradio-app/gradio/pull/11111)".

If you would like to elaborate on your change further, feel free to include a longer explanation in the other sections.
If you would like an image/gif/video showcasing your feature, it may be best to edit the CHANGELOG file using the
GitHub web UI since that lets you upload files directly via drag-and-drop.

gradio-pr-bot · 2023-02-09T23:37:30Z

🎉 The demo notebooks match the run.py files! 🎉

gradio-pr-bot · 2023-02-09T23:38:50Z

All the demos for this PR have been deployed at https://huggingface.co/spaces/gradio-pr-deploys/pr-3165-all-demos

gradio/components.py

abidlabs · 2023-02-10T01:10:14Z

Nice @dawoodkhan82! I noticed a few issues when I was testing with demo/chatbot_multimodal:

If you set starts_with="bot", the top of the first message is cut off by the label:

1b) Also it's interesting the color of the messages has also switched. In other words, user messages are now gray and bot messages are now orange. I don't think the colors should switch?

If you have an initial message (using None as one of the entries of the tuple), then it disappears after the next messages show up

If you return a tuple like ("message", None) from your function, then interestingly you see null:

3b) For some reason, the markdown / image postprocessing isn't working here^ but it's also broken on main -- do you experience this as well with demo/chatbot_multimodal?

abidlabs · 2023-02-10T01:16:54Z

(3) makes me think it might not be possible to use None as the symbol to indicate that a chatbot message doesn't show up. Perhaps "" or any falsey string should be what users use to indicate that a chatbot message shouldn't appear. I can't think of any use case where you actually want to show an empty message like that.

aliabid94 · 2023-02-10T02:00:34Z

Hmm I've never really liked the data structure of chatbot because it was so restrictive - simply assuming one chat between a user and bot back and forth, starting with a user. I think we should be considering making the underlying data structure much more flexible, rather than adding these args. Would love a data structure such as:

[
  ["user", "what color is the sky"],
  ["bot", "the sky is blue"],
]

This would make many things possible - setting which message comes first, allowing multiple messages from one party in a row, or even allowing multiple parties in a chat.

Could be set via gr.Chatbot(type="groupchat") vs gr.Chatbot(type="response_pairs") (can't think of better names for now).

If we don't want to change the data structure for now, I'm not sure we need a "starts_with". Why can't we just have the first message be [None, "I am a bot"]? None seems like the natural way to skip a message, disagree with @abidlabs that we should use an empty string instead.

I think we can just remove the BlockLabel for Chatbot, or make it non-floating, like we do with DataFrame so it doesn't block the chat view (@pngwn might have thoughts on that).

dawoodkhan82 · 2023-02-10T05:01:53Z

@aliabid94 I considered changing the underlying data structure completely. It would be a breaking change to all chatbot demos tho (which there is a lot of now). I like the structure of a list of tuples, where the first element is "user" or "bot", etc. and second element is the message.

When passing [None, "I am a bot"] the frontend received ["I am a bot"] a single element tuple. Since I believe we remove None types.

aliabid94 · 2023-02-10T08:19:04Z

Shouldn’t be a breaking change if we use types to keep the formats separate and support both.
Where are the None’s removed from the output? Just remove that logic.

abidlabs · 2023-02-10T08:46:57Z

@aliabid94 the problem is that we always remove any None's from the config dictionary or any update before we pass it into the frontend -- see the gr.utils.delete_none() method. I believe we do this because None / null behaves weird in JS (@pngwn might remember more details about this). I don't think we should carve out an exception for Chatbot specifically.

However, I have an idea @dawoodkhan82 -- which is that in the postprocessing method of the Chatbot, we can programmatically replace any entry of None with a special enum such as gr.components._Keywords.NO_VALUE and then in the frontend, we treat this special value as null.

That should allow us to handle these kinds of cases without having to change the data structure. We could even handle the multiple messages from one user in a row case that @aliabid94 is talking about by having successive messages in which the "bot entry" is None.

Standard message: [("hi", "hi"), ("how are you?", "I'm good")]
Bot starts message: [(None, "welcome to the bot!"), ("how are you?", "I'm good")]
Multiple messages from the bot: [(None, "welcome to the bot!"), (None, "please say something to get started")]

aliabid94 · 2023-02-10T18:42:19Z

I don't understand this utils.delete_none. None gets converted to null in javascript. We need to preserve the Nones. So right now, if a user returns this dict to a gr.JSON component

{"a": 1, "b": None}

the "b" key will be removed? This is incorrect behaviour, the null is part of the JSON output.

I thought the original intention of utils.delete_none was when an update() was called, a value with None meant no update. But this recursive removal of None on all output doesn't make sense to me.

pngwn · 2023-02-10T22:19:21Z

We can just add some padding to the chatbot to get rid of the label obstruction, that is what we do with other components. Check HighlightedText or JSON for an example.

I don't like the starts_with="bot" kwarg really. I agree with @aliabid94 that the data structure should be enough to control that. I actually don't love the idea of adding kwargs just so we can change the data structure either. I think that the data structure should be:

[
  {speaker: bot, message: "blah"},
  {speaker: user, message: "blah"},
  {speaker: user, message: "blah"},
  {speaker: bot, message: "blah"},
]

But I don't think it is essential we change it now. If None is being removed from inside of a data structure then that is a bug. Only top level 'props' need their None properties removing, that should never happen to the data inside a structure held in a prop, that is potentially very problematic. for example:

{
  "props": {
    "prop1": None -> should be removed, not converted to null
    "prop2": [["hi", None], [None, "hi"]] -> [["hi", null], [null, "hi"]]
  }
}

dawoodkhan82 · 2023-02-11T18:10:12Z

I think for now I'll implement @abidlabs approach. Since it won't require changing the data structure. We can look into make the underlying data structure into a list of dicts if it makes sense in the future. The only limitation I can think of for the current data structure is, it doesn't allow for multiple "users". Which we haven't heard the need for yet. I didn't realize the removing of None in this case was a bug. Will remove the unnecessary starts_with kwarg.

pngwn · 2023-02-12T14:09:38Z

If we just fix the bug (#3183), doesn't mean we can simply pass None and change the frontend?

abidlabs · 2023-02-13T19:08:51Z

I think so @pngwn. Let me get a fix in for #3183

dawoodkhan82 · 2023-02-14T18:00:44Z

If you have an initial message (using None as one of the entries of the tuple), then it disappears after the next messages show up

@abidlabs This is because the initial message is never saved into the history state. Would it be fine to expect the developer to pass in the same default value into the state object? Or do you have a better suggestion.

abidlabs · 2023-02-14T18:22:09Z

@abidlabs This is because the initial message is never saved into the history state. Would it be fine to expect the developer to pass in the same default value into the state object? Or do you have a better suggestion.

Ah you're right, that seems like a perfectly reasonable thing to do. Thanks @dawoodkhan82 and happy to re-review whenever the PR is ready

dawoodkhan82 · 2023-02-14T18:23:15Z

3b) For some reason, the markdown / image postprocessing isn't working here^ but it's also broken on main -- do you experience this as well with demo/chatbot_multimodal?

works for me. When running on the built version of the frontend.

abidlabs · 2023-02-14T18:28:04Z

Let me retest

abidlabs · 2023-02-14T19:05:37Z

Hmm @dawoodkhan82 it's not working for me locally or on the deployed PR Space from this branch:

dawoodkhan82 · 2023-02-15T16:00:34Z

Hmm @dawoodkhan82 it's not working for me locally or on the deployed PR Space from this branch:

Ok I'll take a look at this as a separate issue. Seems like the chatbot just can't locate the tmp file. But also happening on main right?

abidlabs · 2023-02-15T16:25:36Z

But also happening on main right?

Yep

dawoodkhan82 · 2023-02-15T16:48:04Z

ui/packages/chatbot/src/ChatBot.svelte

@@ -94,7 +96,7 @@

 <style>
 	.wrap {
-		height: var(--size-80);


This should fix the height issue @yvrjsharma was facing with his demo

gradio/components.py

abidlabs · 2023-02-16T05:27:35Z

Looks really good @dawoodkhan82! One small issue I noticed was that if a message is None, then no message bubble appears but there is a blank space where the bubble would have appeared.

See this:

Or this:

I don't think any blank space should appear so that the effect is that of getting multiple messages one after the other

dawoodkhan82 · 2023-02-16T05:29:40Z

Oh I see, yeah good catch. Ill fix this

…

On Thu, Feb 16, 2023 at 12:27 AM Abubakar Abid ***@***.***> wrote: Looks really good @dawoodkhan82 <https://github.com/dawoodkhan82>! One small issue I noticed was that if a message is None, then no message bubble appears but there is a blank space where the bubble would have appeared. See this: [image: image] <https://user-images.githubusercontent.com/1778297/219276375-d602ba82-1bf4-4782-83e4-7e9143759893.png> Or this: [image: image] <https://user-images.githubusercontent.com/1778297/219276427-4779715b-0ba5-457d-813e-b701d0ac4c5f.png> I don't think any blank space should appear so that the effect is that of getting multiple messages one after the other — Reply to this email directly, view it on GitHub <#3165 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADBCYLFDP6NUZYJD6MPJIN3WXW3FHANCNFSM6AAAAAAUXEWHA4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

abidlabs · 2023-02-16T05:30:00Z

Side note: the image issue seems to have been fixed!

dawoodkhan82 · 2023-02-16T05:30:40Z

Huh how? Was it fixed by another pr

Co-authored-by: Abubakar Abid <abubakar@huggingface.co>

dawoodkhan82 · 2023-02-17T17:55:57Z

@abidlabs Fixed the display issue

abidlabs

LGTM tested & works very nicely @dawoodkhan82!

dawoodkhan82 added 2 commits February 9, 2023 18:23

chatbot fices

7cff328

chagnelog

0a3ccdd

abidlabs reviewed Feb 10, 2023

View reviewed changes

gradio/components.py Outdated Show resolved Hide resolved

dawoodkhan82 force-pushed the chatbot-fixes branch from 994d937 to 0a3ccdd Compare February 11, 2023 18:34

merge main

e421906

abidlabs and others added 4 commits February 13, 2023 12:08

Merge branch 'main' into chatbot-fixes

6f6db29

merge main

a801baa

remove starts_with

eb129c1

more fixes

12be984

added chatbot multimodal demo

05fd55c

dawoodkhan82 added 2 commits February 15, 2023 11:46

fix height

3c391cf

height

e56ed3c

dawoodkhan82 commented Feb 15, 2023

View reviewed changes

dawoodkhan82 and others added 7 commits February 15, 2023 16:12

update demo

bcce732

Merge branch 'main' into chatbot-fixes

67783b2

changelog

165d0c3

format

b540411

Merge branch 'main' into chatbot-fixes

9256cd7

format

ef5b9f6

Merge branch 'main' into chatbot-fixes

24d4181

abidlabs reviewed Feb 16, 2023

View reviewed changes

gradio/components.py Outdated Show resolved Hide resolved

dawoodkhan82 and others added 4 commits February 16, 2023 11:45

Update gradio/components.py

7775f24

Co-authored-by: Abubakar Abid <abubakar@huggingface.co>

fix

c9141e6

Merge branch 'main' into chatbot-fixes

f2e113d

format

57ac78e

abidlabs approved these changes Feb 17, 2023

View reviewed changes

dawoodkhan82 merged commit 9c08bb9 into main Feb 17, 2023

dawoodkhan82 deleted the chatbot-fixes branch February 17, 2023 18:02

pngwn mentioned this pull request Feb 17, 2023

Make user's message in a gr.Chatbot appear immediately #3092

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ability add a single message from the bot/user side #3165

Add ability add a single message from the bot/user side #3165

dawoodkhan82 commented Feb 9, 2023

gradio-pr-bot commented Feb 9, 2023 •

edited

Loading

gradio-pr-bot commented Feb 9, 2023

abidlabs commented Feb 10, 2023 •

edited

Loading

abidlabs commented Feb 10, 2023 •

edited

Loading

aliabid94 commented Feb 10, 2023 •

edited

Loading

dawoodkhan82 commented Feb 10, 2023

aliabid94 commented Feb 10, 2023

abidlabs commented Feb 10, 2023

aliabid94 commented Feb 10, 2023

pngwn commented Feb 10, 2023 •

edited

Loading

dawoodkhan82 commented Feb 11, 2023 •

edited

Loading

pngwn commented Feb 12, 2023 •

edited

Loading

abidlabs commented Feb 13, 2023

dawoodkhan82 commented Feb 14, 2023

abidlabs commented Feb 14, 2023

dawoodkhan82 commented Feb 14, 2023

abidlabs commented Feb 14, 2023

abidlabs commented Feb 14, 2023

dawoodkhan82 commented Feb 15, 2023

abidlabs commented Feb 15, 2023

dawoodkhan82 Feb 15, 2023

abidlabs commented Feb 16, 2023

dawoodkhan82 commented Feb 16, 2023 via email

abidlabs commented Feb 16, 2023

dawoodkhan82 commented Feb 16, 2023

dawoodkhan82 commented Feb 17, 2023

abidlabs left a comment

Add ability add a single message from the bot/user side #3165

Add ability add a single message from the bot/user side #3165

Conversation

dawoodkhan82 commented Feb 9, 2023

Description

Checklist:

A note about the CHANGELOG

gradio-pr-bot commented Feb 9, 2023 • edited Loading

gradio-pr-bot commented Feb 9, 2023

abidlabs commented Feb 10, 2023 • edited Loading

abidlabs commented Feb 10, 2023 • edited Loading

aliabid94 commented Feb 10, 2023 • edited Loading

dawoodkhan82 commented Feb 10, 2023

aliabid94 commented Feb 10, 2023

abidlabs commented Feb 10, 2023

aliabid94 commented Feb 10, 2023

pngwn commented Feb 10, 2023 • edited Loading

dawoodkhan82 commented Feb 11, 2023 • edited Loading

pngwn commented Feb 12, 2023 • edited Loading

abidlabs commented Feb 13, 2023

dawoodkhan82 commented Feb 14, 2023

abidlabs commented Feb 14, 2023

dawoodkhan82 commented Feb 14, 2023

abidlabs commented Feb 14, 2023

abidlabs commented Feb 14, 2023

dawoodkhan82 commented Feb 15, 2023

abidlabs commented Feb 15, 2023

dawoodkhan82 Feb 15, 2023

Choose a reason for hiding this comment

abidlabs commented Feb 16, 2023

dawoodkhan82 commented Feb 16, 2023 via email

abidlabs commented Feb 16, 2023

dawoodkhan82 commented Feb 16, 2023

dawoodkhan82 commented Feb 17, 2023

abidlabs left a comment

Choose a reason for hiding this comment

gradio-pr-bot commented Feb 9, 2023 •

edited

Loading

abidlabs commented Feb 10, 2023 •

edited

Loading

abidlabs commented Feb 10, 2023 •

edited

Loading

aliabid94 commented Feb 10, 2023 •

edited

Loading

pngwn commented Feb 10, 2023 •

edited

Loading

dawoodkhan82 commented Feb 11, 2023 •

edited

Loading

pngwn commented Feb 12, 2023 •

edited

Loading