Nested .dim() affects surrounding styles too

[sindresorhus]

It seems dim doesn't handle being nested inside another style:

The following:

const chalk = require('chalk');

console.log(chalk.bold(chalk.dim('bold-dim'), 'bold'));

Produces:

While it should produce:

I reproduced this in both Terminal.app and iTerm.app on macOS 10.13.

// @Qix- Any ideas?

[sindresorhus]

I can reproduce this in Chalk v1 too, so not a regression.

[Qix-]

Yep, another "quirk" of an archaic scheme (ANSI escapes).

Bold-off/dim-off are presently represented as the same code, 22m. Historically, bold-off was represented as 21m but for some reason most emulators nowadays don't honor this code and lump both into 22m.

It's wrong and doesn't make any sense IMO, but that's the reality of it.

In order for Chalk to fix this, it would have to keep track of dim and bold statuses throughout nested calls would have to do 22;1m or 22;2m intelligently.

This also means that, unless a global was introduced, using two separate instances of chalk would break this.

const chalk = require('chalk');
const chalk2 = new chalk.constructor();

// below would result in same as OP unless
// a global was introduced to keep track of bold/dim state
// between chalk instances.
console.log(chalk.bold(chalk2.dim('bold-dim'), 'bold'));

And even then, it wouldn't work with two chalk instances from separate packages unless a real global (e.g. Node.js global) was used, which also breaks for all old versions.

It's not a straightforward issue to solve the edge cases - I don't really care about them personally, however.

---

EDIT: and before anybody asks, terminfo wouldn't help here unfortunately.

[sindresorhus]

Bold-off/dim-off are presently represented as the same code, 22m. Historically, bold-off was represented as 21m but for some reason most emulators nowadays don't honor this code and lump both into 22m.

We should at least try to fix the terminals. Do you know if Hyper also has this issue?

[Qix-]

Yes it does.

The unfortunately problem goes a bit deeper than merely fixing the terminals - on windows systems, bold is bright and dim doesn't exist.

Further, many emulators treated bold as a '+1' and dim as a '-1' which is why (I think) the 22m code was unified to bring it back down to '0', if that makes any sense.

It'd be interesting to get a sense of which terminal emulators were being used.

[sindresorhus]

@Qix- But you still agree we should try to get the terminals fixed, right?

[sindresorhus]

Todo report to their issue tracker:

• Hyper
• iTerm
• ?

[Qix-]

And terminal.app.

And MacTerm (also, not sure why it's red).

And Therm (fork of iTerm 2).

---

Was going to try Kitty but the icon bounces a few times and then it crashes on my machine (cc @kovidgoyal).

---

And those are just a few on Mac. I can't imagine the lack of support on Windows. That would be a major undertaking, IMO.

Not that it shouldn't happen, but I feel as though ANSI is too old to 'fix' anyway and should be replaced entirely. Escape codes were to drive serial devices back when networks and display servers (XOrg/Quartz/etc.) weren't a thing. They're among the oldest living and thriving mechanisms that have yet to be re-imagined.

So it really depends on which one should have more effort put forth - fix a potentially long list of terminals for one code, or work to improve the ecosystem as a whole.

Further, we could work to phase out supports-color, which is (in my opinion) bloated and a complete bandaid fix.

[kovidgoyal]

@qix If you want me to look at it it, please open a bug report on kitty's bug tracker and attach the crash report.

And regarding terminal escape codes, the situation with 22 is damn silly, but to get a sense of the problems in this space, see for example kovidgoyal/kitty#226

I agree there should be escape codes to independently turn off bold and dim since in modern terminal emulators they are independent properties. Historically both were implemented by simply adjusting the strength of the beam in a CRT, which I suspect is why they have no independent way to control them.

kitty already implements many extra escape codes to fill the holes in this space, see https://sw.kovidgoyal.net/kitty/protocol-extensions.html

I'll be happy to add a new one for this. However, to make it widely useful you will need to get buy in from other terminal developers. Your best bet at doing that is ccing (egmontkob, gnachman, vapier, mintty and myself on an new issue proposing an escape code). These are the people I know of that can get the proposal into at least some terminal emulators, assuming they agree it is a good idea.

[Qix-]

21 for double underline was a printer code, not a screen display code. This was the case for many escape codes, as early systems treated serial lines as ambiguous (could be screens or could be printers). This is also why old printer paper used to be on perforated reams instead of individual sheets and had the hole-punched tracks on the sides.

However, I understand the frustration.

Double underline isn't pertinent anymore and not having a standard on this stuff is what got us into this mess - hence why I personally think investing time in a new standard that could be agreed upon by emulator and TTY developers alike would be beneficial. Escape codes have been a mess for a long, long time.

[kovidgoyal]

On Tue, Sep 18, 2018 at 08:20:42PM -0700, Qix wrote: Double underline isn't pertinent anymore and not having a standard on this stuff is what got us into this mess - hence why I personally think investing time in a new standard that could be agreed upon by emulator and TTY developers alike would be beneficial. Escape codes have been a mess for a long, long time.

Yeah, but that is not actually going to happen. If you want to solve the individual issue highlighted here, namely a way to turn off bold/dim independently, that can probably be solved by getting buy in from enough terminal developers, for an escape code for it, in the way I outlined.

[Qix-]

Yeah, but that is not actually going to happen.

Why not?

If you want to solve the individual issue highlighted here. . .

Sure, I understand all of that. But this is only a small issue among a sea of problems associated with escape codes.

For example, check out these lines from ansi-regex - that should feel like a hack to most people (because it is).

Further, the fact the community is still fragmented and while most people use VT escape codes (those incorrectly listed as the definitive codes on the Wikipedia page), proper terminals should be adhering to terminfo - which is a pain to parse and is outdated (doesn't support 256 colors, Truecolor, images, etc). Further, the terminfo database implements a fully-fledged, turing complete scripting language. It's a pain to implement (I should know - I wrote a feature-complete implementation and I wanted to cry afterward).

The database existed in order for disparate devices to save bytes-on-the-line when communicating with other proprietary devices. It was very much in-line with Unix's "everything is a file" philosophy - every TTY device handled escapes both similarly and differently, hence why the database was necessary. There was no protocol negotiation and standards weren't as ubiquitous as they are now.

This is no longer the case - we don't use TTY and escapes as often as we used to (especially since we have the internet) and OEMs that use serial lines for hardware and such have their own protocols and definitely don't use terminfo anymore.

Wikipedia shouldn't be used as a method for standards compliance, either - and the 'proper' ways to achieve terminal emulation are outdated, cumbersome, and slow.

Everything about escapes are terrible for modern software. I find it surprising that the community hasn't moved towards something better, honestly.

[kovidgoyal]

On Tue, Sep 18, 2018 at 08:34:01PM -0700, Qix wrote: > Yeah, but that is not actually going to happen. Why not?

Because there simply isn't enough interest/momentum in this space to redo everything from scratch. But hey, I'll be delighted to be proved wrong.

> If you want to solve the individual issue highlighted here. . . Sure, I understand all of that. But this is only a small issue among a sea of problems associated with escape codes. For example, check out [these lines from `ansi-regex`](https://github.com/chalk/ansi-regex/blob/master/index.js#L9-L10) - that should feel like a hack to most people.

regexes are the wrong way to parse ANSI escaped text. You need to write a proper parser for them.

Further, the fact the community is still fragmented and while most people use VT escape codes (those **incorrectly** listed as the definitive codes on the Wikipedia page), _proper_ terminals should be adhering to terminfo - which is a pain to parse and is outdated (doesn't support 256 colors, Truecolor, images, etc). Further, the terminfo database implements a fully-fledged, turing complete scripting language. It's a pain to implemented (I should know - I wrote a [feature-complete implementation](https://github.com/anubis/prtty) and I wanted to cry afterward).

terminfo and ncurses more generally are a tire fire, I agree. IMO the entire concept of using a database for something as basic as interpreting the meaning of escape codes is entirely wrong headed and should be abandoned in favor of a well maintained, centrally distributed standard. That does not however mean escape codes themselves should be abandoned.

Wikipedia shouldn't be used as a method for standards compliance, and the 'proper' ways to achieve terminal emulation are outdated, cumbersome, and slow. Everything about escapes are terrible. I find it surprising that the community hasn't moved towards something better, honestly.

See above, this is simply never going to happen. Huge parts of the terminal ecosystem are inhabited by moribund, barely maintained software that is nonetheless still widely used enough that getting rid of it is not practical.

[Qix-]

Because there simply isn't enough interest/momentum in this space to redo everything from scratch.

But which parts would be "from scratch"? Terminal emulators respond to escapes - why not completely standardize them?

regexes are the wrong way to parse ANSI escaped text. You need to write a proper parser for them.

I agree. Hence why I said it's a hack.

That does not however mean escape codes themselves should be abandoned.

I agree - escape codes shouldn't be abandoned, merely standardized or re-imagined a bit.

See above, this is simply never going to happen.

I'm not entirely sure what it appears I'm suggesting but I'm not saying to completely replace escape codes. Given how file descriptors and streams work, you can't do much to the problem space outside of escapes. Escapes wouldn't go away - they would just be standardized.

[Qix-]

The way I see it, terminal emulators need to be able to tell applications exactly what they support. Do they support color? 256 color? Truecolor? Manual cursor movement? Inline images? Color re-configuration? Can I change the window's title? Mouse support? etc.

A central database to accommodate the matrix of emulators and features - both past, present, and future - is ludicrous. But that functionality needs to exist.

That's all I'm saying.

[kovidgoyal]

:) Have you ever tried to actually get a bunch of terminal developers to agree to a standard on anything? The level of bikeshedding is insane. But as I said, feel free to try, I for one am happy to support your efforts.

As for a centralised standard, the way it works is simple, You define a versioned baseline spec for a bunch of features with a test suite, that any terminal emulator that claims to support the standard must pass. Then there can be optional extensions to the standard that are merged into the baseline when broadly adopted (this is how standardisation works on the web, for example).

There simply needs to be some agreed on method for advertising the baseline version and set of extensions a terminal emulator supports. Ideally by adding it to termios (but that will require buy in from the kernel developers for various OSes), or a an environment variable such as TERM, but that will require buy in from openssh developers to forward that variable by default, and so on.

The problem with this, and indeed any proposals in this space is that there simply aren't enoughpeople available to do the work required.

[Qix-]

or a an environment variable such as TERM, but that will require buy in from openssh developers to forward that variable by default, and so on.

This was my initial approach with a standard I was working on back at ZEIT with Hyper. Whereas termios functionality is very much unix-centric, environment variables are quite universal (including on Windows) and support binary values, which made it ideal.

Since emulators already provide environment variables, adding support would have been quite minor with complete backwards compatibility to boot.

[kovidgoyal]

On Tue, Sep 18, 2018 at 09:18:39PM -0700, Qix wrote: > or a an environment variable such as TERM, but that will require buy in from openssh developers to forward that variable by default, and so on. This was my initial approach with a standard I was working on back at ZEIT with [Hyper](https://github.com/zeit/hyper). Whereas termios functionality is very much unix-centric, environment variables are quite universal (including on Windows) and support binary values, which made it ideal. Since emulators already provide environment variables, adding support would have been quite minor with complete backwards compatibility to boot.

The problem with env vars is they dont survive across ssh, su, sudo and friends, but otherwise, yes, I agree they are ideal.

[Qix-]

Submitting patches for those projects wouldn't be impossible, though.

For what it's worth, I fully understand and appreciate the extent of the changes I'm proposing. It's no small feat but it's entirely worth it in my opinion. It's something @sindresorhus and I have discussed for quite some time.

[kovidgoyal]

Sure, more power to you. I can promise to at least implement them in kitty if you are able to get the ball rolling.

[Qix-]

Years later, I'm going to go ahead and close this. Similar discussions in the emulator community have stalled a few times in recent years and I simply don't see this problem getting fixed in any meaningful way.

Unless a whole new CLI technology rolls around to completely replace the archaic, broken, underspecified, slow, unportable, incompatibility-laden, opinionated, arbitrary, poorly implemented, spottily supported, Wikipedia-based, and holy-war-causing PTY/TTY "standards", this isn't something that is chalk's burden to bear, nor is it something I feel could be resolved with a few stray bug reports.

There are too many emulators nowadays all doing their own thing based on their own set of philosophies and it's not worth anyone's time to bikeshed on how serial codes meant for transmission control over parallel/RS-232 ports should be interpreted in order to produce rainbow text on a $4000 gaming machine in 2021.

There's nothing actionable here - I thank everyone's discussion and hope it serves as a historic reference for anyone interested in endeavoring down this treacherous and unending path.