Console buffer colors not drawn/restored properly (posterized heavily)

[HamRusTal]

Far Manager version

3.0.6369.0 x86

OS version

10.0.19045.4894

Other software

Chafa: chafa.exe

Steps to reproduce

1. Download the attached image: Kusya.png
2. Put chafa.exe to the same directory where the image is (or add it to PATH).
3. Run FAR (e.g. without plugins), navigate to the directory with Kusya.png and chafa.exe; shrink panels vertically (or just hide them by CtrlO).
4. Run chafa.exe Kusya.png, observe the output, run the same command again, observe the output.
5. Expand panels so that they cover part of the image, e.g. just below the green bow (or re-show them by CtrlO).
6. (optional) Call e.g. the plugins menu (F11), cancel it (Esc), observe the background regions affected by drawing the menu (their width depends on the “ClerType-friendly redraw” option in “Interface settings”).
7. Shrink (or hide) panels again, observe the background regions affected by drawing the panels.

Expected behavior

Colors of the background are preserved as output, regardless of whether any FAR UI elements were ever drawn over them.

Actual behavior

In Step 4, the first image output is all posterized heavily, the second image is fine:

Reference image

After Step 6 (if taken), the regions affected by drawing the menu are posterized:

With ClearType-friendly redraw

After Step 7, the regions affected by drawing the panels are posterized:

Max damage

Note 1: the leftmost column of the affected full rows becomes black, unlike other white background cells.

Note 2: there's no such issue in the Command Prompt e.g. when calling and discarding the History menu (F7):

With the menu still shown

[johnd0e]

@HamRusTal
Make sure that VT checkbox is on (in Interface settings).
Also note that for the correct processing the latest WT or openconsole.exe required.

[alabuzhev]

To draw / restore console output we fist need to read / save console output.

To not repeat myself, I refer you to these issues:
microsoft/terminal#292
microsoft/terminal#10810
There are lengthy explanations of what is going on and why and why we cannot have nice things anymore and what can be done and so on.

Eventually an alternative solution was delivered:
microsoft/terminal#14285
microsoft/terminal#16615

And used on our side:
7374303

TL;DR: It allows saving and restoring arbitrary rectangles opaquely, without bothering with characters and color attributes.

As mentioned above, you need relatively recent WT/OC binaries for this to work.
And a relatively recent Far obviously.

The solution is not perfect:

• The new method is applied on top of the classic method, not instead of it, leading to minor flickering.
  It can be improved in theory, but the renderer complexity will skyrocket, so highly unlikely.
• It does not work in the cases where opaque access is not good enough and we absolutely need to see the colors, e.g. to draw shadows or the grabber.

But it is better than nothing.

P.S.

Note 1: the leftmost column of the affected full rows becomes black, unlike other white background cells.

This is what ReadConsoleOutput returns for that column: CHAR_INFO.Attributes is 0x4000 (black background + black foreground + COMMON_LVB_REVERSE_VIDEO), which does not make much sense. In other columns it is 0x00f0 (white background + black foreground). You can try reporting it, but I doubt that bugs in legacy color approximation will receive much attention.

Note 2: there's no such issue in the Command Prompt e.g. when calling and discarding the History menu (F7)

Popups in Command Prompt are a special black magic (you can tell by the fact that they have their own color config in console settings). They work differently and are not exposed to third party apps like Far.

[HamRusTal]

Also note that for the correct processing the latest WT or openconsole.exe required.

@johnd0e Thank you, within OpenConsole.exe this scenario works as expected (apart from FAR clock being imprinted to the scrollback buffer but that's a distinct issue and not specific to OpenConsole).

@alabuzhev Thank you for the explanation (and for all your hard work on FAR!). I'll close the Issue as Won't Fix due to a Limitation of the platform.

[j4james]

It does not work in the cases where opaque access is not good enough and we absolutely need to see the colors, e.g. to draw shadows or the grabber.

@alabuzhev Just on the subject of shadows, you may be able to achieve something like that with a DECCARA operation. That can be used to change the attributes of a cell without affecting the text. So to get a shadow similar to what I've seen in the Far Manager screenshots, you could set the background color to black, and apply a dim attribute to darken the foreground. I don't know if that's exactly the effect you're looking for, but it might be better than nothing.

[alabuzhev]

@j4james We try to draw 'real' shadows by merging both bg and fg with 50% transparent black:

The 16-color fallback looks worse of course, but it's not the end of the world:

The broken text in these areas is probably more pronounced:

If we can keep the text intact it would definitely be better than nothing.
I'll try DECCARA, thank you.

[j4james]

We try to draw 'real' shadows by merging both bg and fg with 50% transparent black:

OK, I don't think we can achieve anything quite as good as that with VT sequences as this point in time, because the dim attribute only applies to the foreground color. Technically there is a mode you can set (not yet implemented in Windows Terminal) that would make it apply to both foreground and background, but that also affects the bold/bright and the blink attributes, which is probably not a desired side effect.

If we can keep the text intact it would definitely be better than nothing.

Yeah, that part should definitely work. It won't be able to create a shadow through the middle of a DBCS character (I think it'll just apply the attributes to both halves), but it at least won't destroy the text.

Btw, I forgot to mention that you'll likely need to change the DECSACE setting before using DECCARA, to get it to apply to a rectangular area rather than a "stream" selection.