SD Api Pics extension, v.1.1

[Brawlence]

Lots of new things, gonna do a quick draft right now:

New features: API detection (press enter in the API box), more robust host address handing, VRAM management (model shuffling), separate image dimensions sliders for H&W, three different trigger modes (manual, interactive, always-on). The button to force/suppress the image on the next reply.
The model input is modified only in the interactive mode; other two are unaffected. The output pic description is presented differently for Picture-book / Adventure mode.

Connection check:

It's gonna strip the http(s):// part and the trailing / if present and also return the status when pressing Enter in that field

#533

[oobabooga]

It's looking great! I'm really excited for this.

[vaidasmaciulis]

Would be nice, if there is a way, to skip generating image when loading character which already has some history (now it generates picture, but it is not visible anywhere)

[Brawlence]

Whoa that's a nice catch! Can you list the exact steps to reproduce it?

Gonna think if I can actually override that behaviour

[vaidasmaciulis]

Steps: Open some character, select Adventure mode, generate some text with images, switch character, switch back to one with images in history. This will issue new SD request and take some time to load as image is being generated. I use gallery extension for switching characters, but I think same happens when using Character tab.

[vaidasmaciulis]

Would be nice to have seed, sampler_name, steps and cfg_scale configurable as well.

[mcmonkey4eva]

I don't know why you'd want to manually specify a seed when an AI is randomly generating the prompt?

[vaidasmaciulis]

some times same seed provides similar images on slightly different promts, it helps to get more consistent characters/visuals (or I am just seeing things)

[laughinggaschambers]

send_pictures always triggers generation in immersive mode. I believe the input prompt from send_pictures has the words send and picture in them. It might be good to filter out any prompts that uses send_pictures.

[Brawlence]

Okay, just a little checklist to keep my thoughts organized:

• Fix bugs:

  • title duplication
  • adventure mode not activating image-gen right away
  • send_pictures triggering image generation (because of the replacement text)

• Write a readme, include

  • Connection checker: how does it work, allowed inputs
  • warning for WSL users: loopback is limited to WSL \ Native, use local IP instead
    ( sd_api_pictures can't connect to stable diffusion on wsl #573 )
  • Difference between Manual, Picturebook Adventure (always-on) and Interactive modes
  • VRAM shuffler explanation
  • How to setup persistent extension settings via settings.json

• Add functionality

  • Manually configure seed, sampler_name, steps and cfg_scale
  • Try to override image viewport, make it display original sized-images
    • If the save original file checkbox is set, display via file link instead (smart!)
    • Keep images unique, introduce subfolders, grouping received pics by date

---

• Planned (later):
  • BUG fix: character swapping triggering image generation (the image is immediately discarded)
  • request available samplers from SD
  • request and set availible MODELS from SD
  • Interactive mode flow change
    • Factor out the decider function
    • Allow the bot to refuse sending pics? w/ option to force anyway
    • In-chat hidden context button for 'illustrate'
  • Zoom-in feature for pics?

[laughinggaschambers]

using regenerate display "is typing" instead of "Is sending a picture"

[Brawlence]

@laughinggaschambers that's unfortunate but I can't detect re-generations; so if you didn't like the image it sent you'll have to 'remove-last' and then 'generate' instead

[ARandomUserFromGithub]

Hi, I have an issue with this plugin whereas the generated picture only quickly flashes and instantly disappears. Also the HTML displays nothing between the textbox divs.
`

<div class="message">
                <div class="circle-bot">
                  <img src="file/cache/AI Helper.png_cache.png">
                </div>
                <div class="text">
                  
                  
                </div>
              </div>

`

[vaidasmaciulis]

Okay, just a little checklist to keep my thoughts organized:

@Brawlence would it make sense to merge current PR and address remaining issues with separate PRs?

[oobabooga]

I agree with @vaidasmaciulis. There is no need for @Brawlence to overwhelm himself and try to fix everything at once. If there are not any new bugs in this PR that need to be addressed, I'd vote for merging it.

[mcmonkey4eva]

(Review submitted because I was tagged for it) there's only one potentially-breaking issue I see, the https part, and a lot of formatting, but that is stuff that can easily be worked on in followups

EDIT: gave it a testrun, worked flawlessly from the text-gen-webui side.
(The Auto WebUI side had errors, but, er, that's an auto webui issue unrelated. Extensions and API calls don't get along)

I'd say either good to pull now, or revert the change that breaks https support and then pull.
Other things to improve aren't blockers.

[Brawlence]

@vaidasmaciulis although nothing seems broken right now, for the last 3 or 4 days I couldn't really test the full extent of my changes because the flow kept breaking cause I want all the new features. I should have fixed the repo state, but oh well.

@oobabooga thanks but don't worry, I am not really overwhelmed

@mcmonkey4eva thanks, your feedback is invaluable!

[Brawlence]

Okay, finished with everything I wanted but readme. It's coming later™

[oobabooga]

In Manual mode, if I click on Force (Suppress) the picture response, the button changes to Suppress the picture response and the next reply is an image.

After that, the button text remains at Suppress the picture response, but the next generation returns text instead of another image.

[Brawlence]

@oobabooga so the state changes correctly but the button is not reset after the generation. Hmmmm

Is there a way to access this button component outside of extension ui()'s scope?

[oobabooga]

You can turn it into a global variable (inside script.py), like with global my_button at the top of the ui() function.

[Brawlence]

Strangely, although declaring it globally does allow me to change its label, the rendered button won't automatically adjust. So, as it is seemingly useless as a feedback for the user, I'm splitting it in two buttons. There won't be any reliable indication for the state of the next reply (picture/regular), but one can always force it with those.

[oobabooga]

I did some reorganizing/reformatting. Everything is working well on my end. I'm impressed at how engineered and detailed this extension has become. In particular, the new Picturebook is really fun to use.

Thanks for the update, @Brawlence :) (and feel free to submit new PRs in the future if you come up with further enhancements)