Problem with printing UTF8 with NO BOM

[BobFrankston]

Environment

Windows build number: [run `[Environment]::OSVersion` for PowerShell, or `ver` for cmd]
Microsoft Windows [Version 10.0.18362.295]
Windows Terminal version (if applicable): 0.11.1191.0

Any other software?

Steps to reproduce

create a file with utf-8 content as in

 “test’

type test.txt

Expected behavior

It should simply show

 “test’

Actual behavior

In PowerShell and with BOM see the proper output

Without BOM
In Command Shell  - ΓÇ£testΓÇÖ
PowerShell - “test’
Ubuntu:  “test’

with BOM
In command Shell - “test’
PowerShell:  “test’
Ubuntu:  “test’

Very simply Windows still seems stuck in the old days of UTF7 After 25 years of Unicode, it needs to grow up. At very least there should be a flag in the settings to enable UTF8 as a default.

[j4james]

You could try setting your code page to UTF-8 with chcp 65001.

[BobFrankston]

Thanks. interesting that it works in Command but not PowerShell. Is there a way to set it as the default in the settings. At least for the Command shell?

Separately I'll submit feedback that CHCP would allow names, not numbers. It should be easy to update it to circa 1999.

[BobFrankston]

Checking there seem to be ways to set it in the register but given Windows Terminal has a settings file that would seem the appropriate place.

[MohammatMajken]

I think I have the same error. hmm :/

[DHowett-MSFT]

So, at its core, this is a request for /dup #1802. I'm not averse to breaking compatibility here, we just haven't done the work necessary here 😄

[ghost]

Hi! We've identified this issue as a duplicate of another one that already exists on this Issue Tracker. This specific instance is being closed in favor of tracking the concern over on the referenced thread. Thanks for your report!

[DHowett-MSFT]

(Thanks for the report!)

[jonasnordlund]

As for why this curiously worked in CMD but not Powershell (and you're actually talking about Powershell here and not Powershell Core), I think it's because Powershell has a requirement of a BOM and your text was without one.

There's a solution though -- one may additionally need to first set:

$OutputEncoding = System.Text.UTF8Encoding

Note that this is no longer necessary in Powershell Core, which thankfully no longer expects a BOM for UTF-8.

[LuanVSO]

in PowerShell 7 it is printed correctly, on windows PowerShell you need to pass -encoding utf8 to the command