[Conpty] Add support for mouse input

[wez]

On: Microsoft Windows [Version 10.0.17763.134]

I have implemented conpty in my terminal emulator:
https://github.com/wez/wezterm

I can successfully run target\debug\wezterm.exe to spawn console applications such as cmd.exe powershell.exe and bash.

The issue I'm seeing is that when I launch bash, either indirectly via cmd.exe or directly via the bash launcher, conpty seems to swallow the mouse reporting escape sequences; I don't see them being received by my terminal parser, and thus vim has no effective mouse support despite being configured with set mouse=a.

Running the same WSL install via wsl-terminal does have working mouse support, and wezterm has been my daily driver on Linux for about a year with working mouse support, so we can rule out an obvious misconfiguration with vim and the parser in wezterm.

I've also tried echo -e "\e[?1000h" to manually enable mouse reporting from the shell; normally (on linux and via wsl-terminal) this causes clicks in the terminal to send data to the shell (which appears as garbage input), but when running my terminal with conpty, these are also swallowed up somewhere.

Is there something special needed for the apps that I spawn into my pty to be able to work with the mouse?

In case you want to double check the key portion of the code, the relevant file is:
https://github.com/wez/wezterm/blob/master/src/winpty.rs
The flow is to CreatePipe a pair of pipes, CreatePseudoConsole, and then pass that to a spawned child via the threadproc attributes, as the samples in the MSDN docs and this repo also do.

[DHowett-MSFT]

Hey @wez,
Unfortunately, ConPTY won't transit mouse reports (or, from a hosted application, requests for mouse reporting). We've got a backlog item tracking this that we're hoping to get to soon.

Unfortunately, we can't just pass encoded mouse events through: since ConPTY can host standard windows console applications, the likes of which would be expecting MOUSE_EVENTs to come in through ReadConsoleInput, we'll need to do some translation.

In all other regards, though, it sounds like you're setting up the pseudoconsole correctly.

---

Tracking: MSFT:20469462

[wez]

@DHowett-MSFT thanks for the response!
It's a bit of a bummer that the mouse reporting isn't there yet, but still good that the rest of the pty stuff is possible now!

[orlp]

I have nothing to add other than really wanting mouse support with ConPTY. With alacritty now having ConPTY support using alacritty + ssh + tmux is an incredible linux terminal on Windows, only mouse support is lacking now.

[nurbles]

I am a heavy user of midnight commander in my Ubuntu bash shell where it works GREAT. Unfortunately, the Ubuntu shell tab in Windows Terminal 0.3.2171.0 does not appear to send ANY mouse events through to the mc application, making it extremely difficult for me to use. I was going to post a bug, but I this it would duplicate this.

[zadjii-msft]

Wait uh, did #9970 close this? @DHowett

[billybraga]

With Microsoft Terminal Preview 1.9.14450.0 and using set mouse=a in vim, scrolling works for me!!!

[DHowett]

Hey, it sure should have!

[DHowett]

For everyone following along in the audience: This shipped with ConPTY as of Terminal 1.9; eventually the change will flow back out to Windows as well.

[tig]

Woot! Woot! Woot.

I've just closed gui-cs/Terminal.Gui#332

I thank you. And all Terminal.Gui users thank you.

[ozgb]

Thank you devs! 🎉❤️ Been following this one for a while!

[silverqx]

With Microsoft Terminal Preview 1.9.14450.0 and using set mouse=a in vim, scrolling works for me!!!

Does it really work for you? Because it does not for me, I'm talking about pure vim on windows in windows terminal.

WinTerm - 1.11.2731
Vim - 8.2.3487

It only works if I ssh to my Gentoo virtual machine and run vim there (from the WindowsTerminal or from the cmd terminal) and it also works when I start vim from MSYS2 mingw64.
Is good to tell that scrolling works in the Midnight Commander ( mc.exe executed in WT ).

Left-click works for me in all the above-described environments.

I have tried different settings, --clean param, and also tried to disable QuickEdit Mode.
Scrolling does not work also in the cmd and PowerShell terminals.

[rbreaves]

@DHowett @DHowett-MSFT I have noticed that the current release build of Windows 10 does not appear to have mouse support when going through VSCode's integrated console - however Windows 11 and VSCode does have mouse support in the integrated Powershell console (to ssh sessions).

Is this is an oversight and will it be remedied in a future update? Also the VSCode team says the issue is with ConPTY, not VSCode.

[DHowett]

@rbreaves Unfortunately, they're right -- the issue is in ConPTY. Also unfortunate is that we don't get any say in what features get backported to Windows 10 ☹️.

Effectively, Windows development has all moved on to Windows 11. Only specific features, with leadership approval, get cherry-picked into the stable Windows 10 branches. The successive updates to Windows 10 20H1 (20H2, 21H1, 21H2) are all based on the same original code plus some "leadership approved" features backported.

That's why we're working so hard to get conhost (and therefore conpty) decoupled from the operating system: so that we can give folks like you updates in a more timely fashion.

[rbreaves]

Oddly enough I’ve mostly stopped using the built in Terminal w/ vscode. I now toggle Windows Terminal as though it’s a quake terminal which works fine either way. I didn’t do this because of the lack of proper conpty support in vscode though w/ win10. I moved on because I do a fake blur w/ glassit & just wanted more space for my code editor.

I still think it’s important for them to decouple conpty - clearly leadership doesn’t understand its importance to developers. Guessing its fix looks no better than a half a dozen others they’d reject & unless you’re a dev then a good terminal experience is just “nice to have” & not a must.

https://gist.github.com/rbreaves/eb7c989ecf71440e31d1f159d2a4619d

[cbartondock]

:set mouse=a still not working for me in Powershell + VIM. No scrolling.

[rbreaves]

Why would it? Vim was written to work w/ posix compliant shells.. aka sh, bash, zsh. Etc.

[zadjii-msft]

@rbreaves That's just... not accurate at all. The shell should have nothing to do with how vim handles mouse mode. Mouse mode is a terminal feature, not a shell one.

@cbartondock would you mind filing a separate issue so we can follow up with you and get to the root cause of your issue? Mouse mode should generally work for both the console and the Terminal now. There are a few bugs on the backlog IIRC, but filing a separate issue would be the easiest way for us to get to the root cause here. (without pinging the other 50 people on this thread 😝)

[silverqx]

The problem is in the vim itself, I have debugged it and mouse events from the scroll wheel are correctly propagated to the vim, but vim doesn't correctly handle them.

It is logical because when the vim WIN32 implementation was written mouse wheel events did not work, that was years before.