Consider allowing users to disable E3 scrollback clear behavior of CSI J

[naikel]

Environment

Windows build number: Microsoft Windows [Version 10.0.19042.685]
Windows Terminal version (if applicable): 1.4.3243.0

Any other software?

Steps to reproduce

Open any kind of terminal like WSL and type clear (bash) or a cmd.exe and type cls.

Expected behavior

The screen should be cleared but the scroll back should not disappear. This is the behavior I would expect if I type clear/cls or send \033c to the terminal.

Actual behavior

The screen is cleared and the scrollback as well, and can't go back. In some environments the scrollback is cleared completely, in some others only the exact number of lines of the terminal are kept. If your terminal has 24 lines, then the last 24 onlines are kept (example ssh to a Linux box has this last lines are kept behavior).

I have used several terminal emulators and this is the first I see that does that. The scrollback is supposed to last forever, until a "Clear Scrollback" menu is selected or something. Clearing the screen should not clear the scrollback.

If this is indeed intended behavior for some obscure Microsoft reason can you at least implement an option to keep the scrollback when the screen is cleared?

Also if you can point me to some commits where this behavior was implemented I would really appreciate it so I can revert it in my dev version.

Workaround

A workaround at least for WSL is pressing Ctrl-L if you use a bash with no modified readline options and it will clear the screen and keep the scrollback, like it is supposed to.

[naikel]

Well this is embarrassing and I'm sorry but after reading the OutputStateMachineEngine code and some other classes it seems that you fully support the extended E3 capability of the terminal, which nobody else does.

So this is not really a obscure Microsoft reason after all.

Since as far as I know all other emulators block the scrollback clear from the E3 extension I guess the fair solution here is a setting where a user can just block it. Implementation is fairly easy in TerminalApi.cpp just treating eraseType as DispatchTypes::EraseType::All always and ignoring DispatchTypes::EraseType::Scrollback if this setting is enabled.

A sad workaround then is something like export TERM=vt100 which doesn't support the scrollback clear but then I'm missing a lot of features since I'd be using a terminal that's literally 42 years old. I'd still prefer an option to keep my scrollback forever.

Also a historySize: 0 or -1 to disable the limit would be nice. I want my whole 32 gigs of RAM of scrollback!

EDIT: Another workaround for WSL: alias clear='clear -x'

[naikel]

Unfortunately I also realized that the historySize is a SHORT and a buffer of only 32,767 lines is not that very useful 😢. Infinite scrollback is very urgent.

[DHowett]

If this is indeed intended behavior for some obscure Microsoft reason

So this is not really a obscure Microsoft reason after all.

This might be a good lesson about making assumptions. We're generally good people with positive intent, and we try to document all of our decisions here in issues or in our docs folder. 😄

We definitely want infinite scrollback (#1410).

I'm genuinely curious, though -- why would clear emit 3J to so many terminals if they didn't support it? Its default behavior is to request the scrollback buffer be deleted... so I can't imagine why it would continue to do so if terminals in common use ignored it.

We're all trying to converge on a standard!

[naikel]

I'm genuinely curious, though -- why would clear emit 3J to so many terminals if they didn't support it? Its default behavior is to request the scrollback buffer be deleted... so I can't imagine why it would continue to do so if terminals in common use ignored it.

I agree it should be backwards, the default behaviour should be just the screen and with an option to delete the screen + scrollback.

... but cls in cmd.exe does the same thing with no option to disable it. Same for clear/clear-host in PowerShell. There should be a way to stop this.

[DHowett]

Fair point. That one was a choice we made (pull request: #5627) due to overwhelming community desire. PowerShell and cmd outside of Terminal do destroy the user's scrollback, and folks found it jarring that it didn't do so in Terminal. #1305 and #3126, the issues tracking those complaints, had a total of 9 duplicates (disincluding the two main issues.)

[zadjii-msft]

it seems that you fully support the extended E3 capability of the terminal, which nobody else does.

That's not really true though, right? gnome-terminal and xterm definitely support ^[[3J. Those are the terminals I used to help implement that in the first place.

I'd be willing to bet that this is something that varies from distro to distro and TERM to TERM.

• What distro are you using in WSL?
• What's the value of TERM for you?
• And what does infocmp output?

I'm just gonna cc @j4james on this thread, cause he's usually interested in these sorts of matters.

[naikel]

That's not really true though, right? gnome-terminal and xterm definitely support ^[[3J. Those are the terminals I used to help implement that in the first place.

yeah... but some popular terminals like xfce4-terminal or even SecureCRT for Windows would never clear the scrollback.

Again I'm not against the proper implementation of E3 standards, but since popular terminals would never kill the scrollback to you, I think it's fair for an option to avoid that, even if it's something never seen before in for example cmd.exe.

• What distro are you using in WSL?

Arch

• What's the value of TERM for you?

xterm-256color

• And what does infocmp output?

xterm-256color|xterm with 256 colors,
        am, bce, ccc, km, mc5i, mir, msgr, npc, xenl,
        colors#0x100, cols#80, it#8, lines#24, pairs#0x10000,
        acsc=``aaffggiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,
        bel=^G, blink=\E[5m, bold=\E[1m, cbt=\E[Z, civis=\E[?25l,
        clear=\E[H\E[2J, cnorm=\E[?12l\E[?25h, cr=\r,
        csr=\E[%i%p1%d;%p2%dr, cub=\E[%p1%dD, cub1=^H,
        cud=\E[%p1%dB, cud1=\n, cuf=\E[%p1%dC, cuf1=\E[C,
        cup=\E[%i%p1%d;%p2%dH, cuu=\E[%p1%dA, cuu1=\E[A,
        cvvis=\E[?12;25h, dch=\E[%p1%dP, dch1=\E[P, dim=\E[2m,
        dl=\E[%p1%dM, dl1=\E[M, ech=\E[%p1%dX, ed=\E[J, el=\E[K,
        el1=\E[1K, flash=\E[?5h$<100/>\E[?5l, home=\E[H,
        hpa=\E[%i%p1%dG, ht=^I, hts=\EH, ich=\E[%p1%d@,
        il=\E[%p1%dL, il1=\E[L, ind=\n, indn=\E[%p1%dS,
        initc=\E]4;%p1%d;rgb\:%p2%{255}%*%{1000}%/%2.2X/%p3%{255}%*%{1000}%/%2.2X/%p4%{255}%*%{1000}%/%2.2X\E\\,
        invis=\E[8m, is2=\E[!p\E[?3;4l\E[4l\E>, kDC=\E[3;2~,
        kEND=\E[1;2F, kHOM=\E[1;2H, kIC=\E[2;2~, kLFT=\E[1;2D,
        kNXT=\E[6;2~, kPRV=\E[5;2~, kRIT=\E[1;2C, ka1=\EOw,
        ka3=\EOy, kb2=\EOu, kbs=^H, kc1=\EOq, kc3=\EOs, kcbt=\E[Z,
        kcub1=\EOD, kcud1=\EOB, kcuf1=\EOC, kcuu1=\EOA,
        kdch1=\E[3~, kend=\EOF, kent=\EOM, kf1=\EOP, kf10=\E[21~,
        kf11=\E[23~, kf12=\E[24~, kf13=\E[1;2P, kf14=\E[1;2Q,
        kf15=\E[1;2R, kf16=\E[1;2S, kf17=\E[15;2~, kf18=\E[17;2~,
        kf19=\E[18;2~, kf2=\EOQ, kf20=\E[19;2~, kf21=\E[20;2~,
        kf22=\E[21;2~, kf23=\E[23;2~, kf24=\E[24;2~,
        kf25=\E[1;5P, kf26=\E[1;5Q, kf27=\E[1;5R, kf28=\E[1;5S,
        kf29=\E[15;5~, kf3=\EOR, kf30=\E[17;5~, kf31=\E[18;5~,
        kf32=\E[19;5~, kf33=\E[20;5~, kf34=\E[21;5~,
        kf35=\E[23;5~, kf36=\E[24;5~, kf37=\E[1;6P, kf38=\E[1;6Q,
        kf39=\E[1;6R, kf4=\EOS, kf40=\E[1;6S, kf41=\E[15;6~,
        kf42=\E[17;6~, kf43=\E[18;6~, kf44=\E[19;6~,
        kf45=\E[20;6~, kf46=\E[21;6~, kf47=\E[23;6~,
        kf48=\E[24;6~, kf49=\E[1;3P, kf5=\E[15~, kf50=\E[1;3Q,
        kf51=\E[1;3R, kf52=\E[1;3S, kf53=\E[15;3~, kf54=\E[17;3~,
        kf55=\E[18;3~, kf56=\E[19;3~, kf57=\E[20;3~,
        kf58=\E[21;3~, kf59=\E[23;3~, kf6=\E[17~, kf60=\E[24;3~,
        kf61=\E[1;4P, kf62=\E[1;4Q, kf63=\E[1;4R, kf7=\E[18~,
        kf8=\E[19~, kf9=\E[20~, khome=\EOH, kich1=\E[2~,
        kind=\E[1;2B, kmous=\E[<, knp=\E[6~, kpp=\E[5~,
        kri=\E[1;2A, mc0=\E[i, mc4=\E[4i, mc5=\E[5i, meml=\El,
        memu=\Em, mgc=\E[?69l, oc=\E]104\007, op=\E[39;49m, rc=\E8,
        rep=%p1%c\E[%p2%{1}%-%db, rev=\E[7m, ri=\EM,
        rin=\E[%p1%dT, ritm=\E[23m, rmacs=\E(B, rmam=\E[?7l,
        rmcup=\E[?1049l\E[23;0;0t, rmir=\E[4l, rmkx=\E[?1l\E>,
        rmm=\E[?1034l, rmso=\E[27m, rmul=\E[24m,
        rs1=\Ec\E]104\007, rs2=\E[!p\E[?3;4l\E[4l\E>, sc=\E7,
        setab=\E[%?%p1%{8}%<%t4%p1%d%e%p1%{16}%<%t10%p1%{8}%-%d%e48;5;%p1%d%;m,
        setaf=\E[%?%p1%{8}%<%t3%p1%d%e%p1%{16}%<%t9%p1%{8}%-%d%e38;5;%p1%d%;m,
        sgr=%?%p9%t\E(0%e\E(B%;\E[0%?%p6%t;1%;%?%p5%t;2%;%?%p2%t;4%;%?%p1%p3%|%t;7%;%?%p4%t;5%;%?%p7%t;8%;m,
        sgr0=\E(B\E[m, sitm=\E[3m, smacs=\E(0, smam=\E[?7h,
        smcup=\E[?1049h\E[22;0;0t,
        smglr=\E[?69h\E[%i%p1%d;%p2%ds, smir=\E[4h,
        smkx=\E[?1h\E=, smm=\E[?1034h, smso=\E[7m, smul=\E[4m,
        tbc=\E[3g, u6=\E[%i%d;%dR, u7=\E[6n,
        u8=\E[?%[;0123456789]c, u9=\E[c, vpa=\E[%i%p1%dd,

[DHowett]

I'll mark this up for backlog consideration to disable E3. Thanks!

[12Me21]

I would recommend either creating an alias for clear -x, as you mentioned, or defining a custom terminfo entry
this creates a copy of xterm-256color with the E3 capability disabled:

xterm-256color-scrollback|xterm-256color without scrollback clearing,
  E3@,
  use=xterm-256color,

install it with tic -x <filename>, and set TERM to "xterm-256color-scrollback"

[naikel]

I would recommend either creating an alias for clear -x, as you mentioned, or defining a custom terminfo entry
this creates a copy of xterm-256color with the E3 capability disabled

It does work on bash.