Fix prompt parser default step transformer

[catboxanon]

Describe what this pull request is trying to achieve.

This PR aims to fix the prompt parser's default step transformer as part of processing the prompt AST. In particular, the current implementation causes a case where if emphasis is used in the alternating words syntax, it incorrectly parses the prompt. Ultimately, I believe this is a very simple fix.

This is an output from doctest in the current implementation for this case:

Failed example:
    g("[a|(b:1.1)]")
Expected:
    [[1, 'a'], [2, '(b:1.1)'], [3, 'a'], [4, '(b:1.1)'], [5, 'a'], [6, '(b:1.1)'], [7, 'a'], [8, '(b:1.1)'], [9, 'a'], [10, '(b:1.1)']]
Got:
    [[1, 'a'], [2, '('], [3, 'a'], [4, '('], [5, 'a'], [6, '('], [7, 'a'], [8, '('], [9, 'a'], [10, '(']]

After implementing this PR, the test passes, and all other existing tests pass (besides the one explicitly labeled not handling this right now).

Additional notes and description of your changes

I've included a compatibility option incase users (like myself) had previously used this syntax and still want to replicate old seeds. If you believe this is unnecessary feel free to remove it.

Environment this was tested in

• OS: Windows 11
• Browser: Firefox
• Graphics card: NVIDIA RTX 3090

[AUTOMATIC1111]

This looks like a very uncommon case. Just add the fix, without settings, and we will document this.

[catboxanon]

Removed the compat option from settings now.

[AUTOMATIC1111]

[ 0 ] days without breaking old seeds!

[lbeltrame]

I believe this breaks the [aaaa] emphasis. In fact, if you use [idontwantthistoken] everything else which follows in the prompt is silently ignored.

[catboxanon]

Full sample prompt? If I test for a [b] c it returns [[10, 'a [b] c']], so I don't believe it's ignoring the rest. I ran a sample prompt in the webui as well and nothing seems wrong to me.

[lbeltrame]

At least with Anything v3, I tried (DPM++ 2M Karras, 30 sampling steps, 512x512):

(masterpiece:1.3), (best quality), (highres:1.1), ultra-detailed, incredibly detailed, 4k, 8k, (illustration), (art by kantoku:1.2), a girl tripping and falling on the ground inside a highly futuristic executive office.
from above, 1girl, (solo:1.2), [open mouth], full body, sitting, arm support, spread legs, (on floor:1.1), (hand on own head:1.2), ), (medium breasts:1.2), short hair, ahoge, (two side up:1.4), (dark brown hair:1.1), (mature:1.05), employee card, lanyard, (labcoat:1.1)

If [open mouth] is present, everything past this prompt is ignored.

I'll attach an example in a few minutes once my current task is over.

[catboxanon]

Running a test on that prompt via doctest doesn't cut the rest off, so it parses it correctly. I also get different results in the webui with and without the rest after [open mouth].

[lbeltrame]

Here are the tests:

Prompt 1:

(masterpiece:1.3), (best quality), (highres:1.1), ultra-detailed, incredibly detailed, 4k, 8k, (illustration), (art by kantoku:1.2), a girl tripping and falling on the ground inside a highly futuristic executive office.
from above, 1girl, (solo:1.2), full body, sitting, arm support, spread legs, (on floor:1.1), (hand on own head:1.2), ), (medium breasts:1.2), short hair, ahoge, (two side up:1.4), (dark brown hair:1.1), (mature:1.05), employee card, lanyard, (labcoat:1.1)

Result:

Prompt 2:

(masterpiece:1.3), (best quality), (highres:1.1), ultra-detailed, incredibly detailed, 4k, 8k, (illustration), (art by kantoku:1.2), a girl tripping and falling on the ground inside a highly futuristic executive office.
from above, 1girl, (solo:1.2), [open mouth], full body, sitting, arm support, spread legs, (on floor:1.1), (hand on own head:1.2), ), (medium breasts:1.2), short hair, ahoge, (two side up:1.4), (dark brown hair:1.1), (mature:1.05), employee card, lanyard, (labcoat:1.1)

Result:

Common negative prompt (for both):

(3d, 3dcg, cg, smooth shading, realistic:1.2),lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts,signature, watermark, username, blurry, artist name, (ugly), fat, obese, (((deformed))), blurry, , bad anatomy, disfigured, poorly drawn face, mutation, furry, mutated, (((extra limb))), (poorly drawn hands), messy drawing, (((loli)))), white hair, silver hair, flat chest, (lapis lazuli hair:1.2), hollow eyes, (small breasts:1.2), (large breasts:1.2), realistic face , , , , , , , , , , , , , , , , , , , fantasy, art by [Edward Lear|Jean Arp|Jacob Lawrence] , art by [El Greco|Jacek Yerka|Mark Ryden|Studio Trigger|Tex Avery] , art by [Balthus|Ivan Albright|Rockwell Kent|Sofonisba Anguissola] , art by [Albrecht Durer|Alfred Eisenstaedt|Antonello da Messina|Carlo Crivelli|Ernst Ludwig Kirchner|Tim Burton|Tintoretto] , huge breasts, red hair, water,  on floor, (outdoors:1.3), earth, black labcoat, gloves, child, window

Parameters: 30 sampling steps, 12 CFG scale, model Anything v3, sampler DPM++ 2M Karras, 512x512, seed 1699627363.

[catboxanon]

Well for prompt 1 it's basically identical to your's. For prompt 2 it's actually working fine for me.

Image & PNG Info

(masterpiece:1.3), (best quality), (highres:1.1), ultra-detailed, incredibly detailed, 4k, 8k, (illustration), (art by kantoku:1.2), a girl tripping and falling on the ground inside a highly futuristic executive office.
from above, 1girl, (solo:1.2), [open mouth], full body, sitting, arm support, spread legs, (on floor:1.1), (hand on own head:1.2), ), (medium breasts:1.2), short hair, ahoge, (two side up:1.4), (dark brown hair:1.1), (mature:1.05), employee card, lanyard, (labcoat:1.1)
Negative prompt: (3d, 3dcg, cg, smooth shading, realistic:1.2),lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts,signature, watermark, username, blurry, artist name, (ugly), fat, obese, (((deformed))), blurry, , bad anatomy, disfigured, poorly drawn face, mutation, furry, mutated, (((extra limb))), (poorly drawn hands), messy drawing, (((loli)))), white hair, silver hair, flat chest, (lapis lazuli hair:1.2), hollow eyes, (small breasts:1.2), (large breasts:1.2), realistic face , , , , , , , , , , , , , , , , , , , fantasy, art by [Edward Lear|Jean Arp|Jacob Lawrence] , art by [El Greco|Jacek Yerka|Mark Ryden|Studio Trigger|Tex Avery] , art by [Balthus|Ivan Albright|Rockwell Kent|Sofonisba Anguissola] , art by [Albrecht Durer|Alfred Eisenstaedt|Antonello da Messina|Carlo Crivelli|Ernst Ludwig Kirchner|Tim Burton|Tintoretto] , huge breasts, red hair, water,  on floor, (outdoors:1.3), earth, black labcoat, gloves, child, window
Steps: 30, Sampler: DPM++ 2M Karras, CFG scale: 12, Seed: 1699627363, Size: 512x512, Model hash: 38c1ebe3, Clip skip: 2

[lbeltrame]

Weird indeed. I'll go through the list of the extensions, perhaps there's something interfering there.