[Feature Request]: Add-to-prompt in Hires Fix

[dchatel]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

Right now we can specify particular prompts for hires fix that replace the prompts used in the first pass.
It would be very nice to be able to add new tokens to the prompt used in the first pass instead (when checking a checkbox, for example).

Why ?
Basically, tokens like "low quality" and other tokens that mostly serve to increase the quality of the picture also have a limitating effect on the variability of the composition of the image.

A typical example is "fantasy landscape, realistic, photorealistic, etc..." add any more prompts to make the picture as realistic as possible. And you end up with a picture that looks like a real photo from our world, and not a realistic photo of a fantasy landscape: instead of having weird mountain ranges for example, you get regular ones.
If you remove these quality tokens, then your picture will get more creative but will also look more fake.

Proposed workflow

1. Enable Hires Fix as usual.
2. check a checkbox that says "add to prompt"

or

in settings, next to the settings that enable hires fix prompt/checkpoint also add the checkbox here.

Then, when you can use hires fix prompts like you would with styles, for example: "{prompt}, realistic, photorealistic, ..."
{prompt} referring to the prompt used in the first pass.

Additional information

No response

[w-e-w]

I assume that you didn't know about the feature

Settings > User interface > Hires fix: show hires prompt and negative prompt

close this post if this is the case

[dchatel]

I know about this feature. But this is something slightly different that I am suggesting.

Basically, right now, if you set "x" in the normal prompt and "y" in the hires fix prompt, the hires fix pass will use "y".

What I am suggesting is to add a checkbox or some token like "{prompt}" that would be parsed when using the hires fix prompt, such that:
if you set "x" in the normal prompt and "{prompt}, y" in the hires fix prompt, the hires fix pass will use "x, y".

[w-e-w]

I think I get what you mean
essentially you want some sort of checkbox or syntax
that would change the behavior of the hr prompt to append (checkbox) or replace (syntax) the first-pass prompt
as opposed to same as first pass when blank new prompt when non-blank

this should be relatively easy to implement as an extension

[kjerk]

This might be suited to be an extension of the prompt parsing syntax, because it seems like you're immediately going to run into "Wait no I don't want it appended, the attention is stronger at the front. Oh but I do want", and then "Wait but no I wanted this part at the end, because it's a refinement quality and that's okay at the end." and you're basically back to needing to rewrite the prompt in the boxes as pointed out above anyway.

But something like the prompt editing syntax where you specify [realistic:photorealistic:hr], a photo of a [dog:very fluffy dog:hr] running on a beach with a rainbow[, 4k, high resolution:hr] (As a rough example, that syntax is a bit weird). I find myself wanting to do similar operations but usually both adding and removing tokens from specific places in both the positive and negative hr prompt.

But extending the prompting syntax is going to be a real headache, understandable, maybe there's some inspiration to be found in sd-dynamic-prompts

[w-e-w]

I actually do have an extension planned
https://github.com/w-e-w/sd-webui-hires-fix-tweaks (not finished it does not contain this feature yet)
because I the UI to be right I'm waiting on this PR #14119 to continue development

note I do intend to make it backwards compatible for version prior to this PR

[catboxanon]

Doesn't the change implemented in #12457 (and described on the wiki page) already allow for this? Something like this would only use y in the second/hires pass (whitespace for clarity):

x [   : y : 1.0 ]

You can of course do the inverse of this if you switch where y is placed, so it's only used in the first/pre-hires pass.

x [ y :   : 1.0 ]

[w-e-w]

Doesn't the change implemented in #12457 (and described on the wiki page) already allow for this? Something like this would only use y in the second/hires pass (whitespace for clarity):

as I understand it I believe it's more "for convenience"
yes using prompt editing or manually doing everything would certainly work
but it would be nice to streamline the experience

[dchatel]

Totally forgot about the new behavior for prompt editing. It's probably the best way to do it. No need to introduce more complexity into an already complex project for this.