Windows Console doesn't support Unicode out of the box

[barrytang]

Description of the new feature/enhancement

This suggestion is originated from https://developercommunity.visualstudio.com.

Right now, Windows Console doesn't support Unicode by default for English OS. To make it work, the app needs to either set output encoding to UTF8 or configure Windows Console properties to UTF8. Even after doing so, some Unicode still won't get rendered correctly because Windows Console doesn't support font-fallback as explained here.

Proposed technical implementation details (optional)

It would be great to enable an out-of-the-box console experience that supports Unicode for customers.

[237dmitry]

I think this is rather OS wide settings

[zadjii-msft]

Yea, I'm not sure that's something we'll be able to change in any short order. Using the console APIs in a client application to specifically request CP_UTF8 is the best way to do this.

There's a collection of issues here to follow:

• Terminal should force pseudoconsole host into UTF-8 codepage by default #1802 might make this better for the Terminal in the future. We can use the Terminal to opt applications in to UTF8 as the default codepage. We'd have a setting within Terminal to opt out of this support. That would alieviate the need to call SetConsole*CP for any apps that are being run in the Terminal.
• [Epic] Text Buffer rewrite  #8000 and Add emoji support to Windows Console #190 are generally tracking any other work to be done to better support the breadth of unicode in the buffer, in input and output.
• Replace PolyTextOutW with ExtTextOutW #10478 should have largely mitigated any unicode rendering issues in the vintage console, as far as I'm aware. Conhost will still be subject to issues that would arise from the buffer not supporting full unicode (as tracked above)
  • This PR should be available in SV2 builds of Windows 11 (e.g. >22000.000), but I don't know the first build number it appeared in.

I'm gonna collectively resolve this as a dupe of those conversations. Thanks!
/dup #1802 #8000 #190 #10478

[ghost]

Hi! We've identified this issue as a duplicate of another one that already exists on this Issue Tracker. This specific instance is being closed in favor of tracking the concern over on the referenced thread. Thanks for your report!

[asmichi]

Sorry for posting in a closed issue.

Using the console APIs in a client application to specifically request CP_UTF8 is the best way to do this.

I agree and that's the very reason Visual Studio should do so. Therefore the original suggestiion should be reopened -- Visual Studio should have the ability to emulate a chcped console for debug execution.
(It's not normal for the debugee to set the code page since (IMO) console programs should respect the current code page. We normally do chcp.com <codepage> and then run a program when we want the program to obey a specific code page.)

Well, when the world is like "There is only one true console code page -- UTF-8", the problem itself disappears. If that is our direction, yeah, #1802 is it.