C0 characters in screen buffer are incorrectly interpreted as controls by conpty

[j4james]

Environment

Windows build number: Version 10.0.18362.535
Windows Terminal version (if applicable): 0.8.10091.0

Steps to reproduce

Compile the follow C++ program, and then run the resulting executable from a cmd shell in the Windows Terminal.

#include <windows.h>
#include <string>

void main() {
    HANDLE handle = GetStdHandle(STD_OUTPUT_HANDLE);

    // Enable VT mode.
    DWORD mode;
    GetConsoleMode(handle, &mode);
    mode |= ENABLE_VIRTUAL_TERMINAL_PROCESSING;
    SetConsoleMode(handle, mode);

    // Write directly to the screen buffer with control chars.
    DWORD written = 0;
    const std::wstring text = L"\x1B[31mIs this text red?\nIs this on a new line?\n";
    WriteConsoleOutputCharacterW(handle, text.c_str(), text.length(), COORD{ 0, 0 }, &written);
}

Expected behavior

The WriteConsoleOutputCharacterW api writes directly to the screen buffer, without attempting to process any control characters. I would thus expect the text message to be output entirely on the first line of the screen buffer, with the ESC and LF control characters output as glyphs, and not interpreted. This is what it looks like in a conhost cmd shell:

Note that the ENABLE_VIRTUAL_TERMINAL_PROCESSING mode should have no effect on this API. I've only turned it on to make sure the code is running under the same conditions in conhost as it is in WT (which enables VT mode by default).

Actual behavior

In the Windows Terminal, the ESC and LF characters are interpreted as control characters instead of being output as glyphs. As a result, you get an escape sequence changing the text color to red, and the text will be output over multiple lines.

Depending on timing, you can also end up with other parts of the screen being redrawn incorrectly, as the terminal cursor position is now out of sync with where the conhost thinks it is (notice part of the copyright message being redrawn two lines down in red).

[DHowett-MSFT]

Yep, absolutely. This is pretty bad, but it's something we've kicked down the road a bit.

I was secretly hoping we could add 0x2400 to each one when we render it and get the Control Picture...

I had written "this is the actual root cause of #1965", but then I read the history and I think we just never filed the followup bug for this, and to boot I realized that you had filed it. 😁

[DHowett-MSFT]

#1965 (comment) and #1965 (comment) for context

[j4james]

Ah, that's the issue I was looking for. I knew we'd discussed this somewhere before, but all I could find was PR #1964 (which I thought at the time would have solved the problem, but clearly not in all situations).

As for how we fix it, if we want WT to be "compatible" with the legacy console, then I would have thought conpty could simply translate these characters to the Unicode replacement character (U+FFFD)? If that's the glyph they produce in conhost, then surely that's what you'd want to see in WT too.

[adipose]

Piling on. I have used WriteConsoleOutputCharacter in the past to bypass printing the control characters, but now they are broken (note that in latest non-insider windows builds, the glyphs themselves are also broken on cmd.exe, but at least the formatting is incorrect).

vs.