Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Mouse/Touch/Pointer Bindings (like middle-click paste, right-click context menu, etc.) #1553

Open
1 of 14 tasks
carlos-zamora opened this issue Jun 24, 2019 · 49 comments
Assignees
Labels
Area-Input Related to input processing (key presses, mouse, etc.) Area-Settings Issues related to settings and customizability, for console or terminal Area-TerminalControl Issues pertaining to the terminal control (input, selection, keybindings, mouse interaction, etc.) Help Wanted We encourage anyone to jump in on these. Issue-Scenario Product-Terminal The new Windows Terminal.

Comments

@carlos-zamora
Copy link
Member

carlos-zamora commented Jun 24, 2019

Summary of the new feature/enhancement

Expand settings to be able to define remappable mouse bindings. Arguably, different touch events should fall under this too. So let's just call this "pointer bindings" for now.

Proposed technical implementation details (optional)

Mouse bindings are a bit trickier than keybindings in that the mouse has a location where the event occurs. For example, right-clicking a tab should have a different effect on the terminal than right-clicking the terminal.

As a super early idea, consider this format:

"pointerbindings": [
{
  "device": "mouse",
  "event": "rightClick",
  "where": "tab",
  "command": "flyoutMenu"
},
{
  "device": "mouse",
  "event": "doubleClick",
  "where": "terminal",
  "command": "wordSelection"  
},
{
  "device": "touch",
  "event": "swipe",
  "where": "terminal",
  "command": "scroll"
}
]

We definitely need a spec for this because it'll be a bit hefty. We'll also need to update a decent amount of documentation (particularly settings schema) to be able to describe what combinations are acceptable (again, the JSON mentioned up here is just me rambling about a potential implementation.)

There may be overlap between some commands in keybindings. Be sure to think that through.

Mike notes:

we should go back and collect up all the threads we've said "this would be a good mouse bindings feature" below


spec draft

Mouse bindings

Abstract

We've had numerous requests to configure how the mouse behaves in the Terminal.
The original behavior was a simple duplication of how conhost behaved: a right
click will copy the a selection if there is one, or paste the clipboard if there
isn't. Over time, we've accumulated a number of scenarios that we believe can
all be addressed by allowing more fine-grained mouse binding support. However,
upon further review, ultimately, we don't really need deep mouse binding
support.

Scenarios

The following is a list of all the feature requests we've linked to mouse
bindings in the past, grouped into categories of related requests:

Change how mouse multi-click selects

Change the action that L/M/R-mouse performs

  • #7646 xterm-style selection, paste on middle click, copy on mouse release
  • #10802 - VK_XBUTTON1/2, etc.
  • #6250 - separate "Paste Clipboard" vs "Paste Selection" actions
  • #3337 - Add a right-click context menu

Other

These are smaller, independent features that could all have an individual setting (if needed)

  • #11710 Request: Setting to disable zooming with ctrl+mousewheel
  • #13598 Add option to configure URL clicking behavior
  • #11906 Option to disable "Pinch to Zoom"
  • #6182 Fast scroll by holding down Alt while scrolling with mouse wheel
  • Block selection by default (without alt) (see mail thread "RE: How to disable line wrap selection in Terminal")
  • #17610 Configure block select to use ctrl, not alt

Solution design

Following the above scenarios, solutions are proposed below:

Change how mouse multi-click selects

Across the requests here, we've got the following requests:

  • double-click: selects a "word" between 2 same delimiters
  • triple-click: selects an entire string separated by spaces
  • 4-click: entire line

Currently, Ctrl+A will select the entire command/text input, however, triple
clicking (mouse version of Select All selects the entire line (including the
prompt). GIF shows selecting with mouse vs with keyboard:
...
I would like the triple click action to align to the Ctrl+A selection method.

Could we maybe add shift double click to select using alternate word
delimiters?

I was really thinking more of regex though, because it can be a good starting
point for implementing more advanced features like type-specific smart
highlighting and hyperlinking of terminal text, not just smart selection.

To boil this down, users want to be able to configure the behavior of double,
triple, and quadruple clicks. The most common request is to change the
delimiters for double-click selection. But users also want to be able to
configure the delimiters to change on
Shift/Alt/Ctrl clicks.

"mouse": {
    "clicks": {
        { "click": "double", "command": { "action": "expandSelection", "delimeters": " /\\()\"'-.,:;<>~!@#$%^&*|+=[]{}~?\u2502" } }
        { "click": "shift+double", "command": { "action": "expandSelection", "delimeters": " " } }
        { "click": "triple", "command": { "action": "expandSelection", "regex": "^.*$" } }
    }
}

Alternatively,

"mouse": {
    "doubleClick": " /\\()\"'-.,:;<>~!@#$%^&*|+=[]{}~?\u2502",
    "tripleClick": { "regex": "^.*$" }
}
@carlos-zamora carlos-zamora added the Issue-Feature Complex enough to require an in depth planning process and actual budgeted, scheduled work. label Jun 24, 2019
@ghost ghost added Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting Needs-Tag-Fix Doesn't match tag requirements labels Jun 24, 2019
@carlos-zamora carlos-zamora added Area-Interaction Interacting with the vintage console window (as opposed to driving via API or hooks) Area-Settings Issues related to settings and customizability, for console or terminal Help Wanted We encourage anyone to jump in on these. Issue-Docs It's a documentation issue that really should be on MicrosoftDocs/Console-Docs Product-Terminal The new Windows Terminal. and removed Needs-Tag-Fix Doesn't match tag requirements Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting labels Jun 24, 2019
@carlos-zamora carlos-zamora self-assigned this Jun 24, 2019
@carlos-zamora
Copy link
Member Author

A few things:

  • Treat this as a megathread so feel free to share your ideas here 😁
  • I'm a bit busy with Accessibility (TOP PRIORITY) at the moment so if anybody wants to start working on this, just let me know.

@philippeller
Copy link

Probably related, make for example scrolling work when using vim etc....used to work fine for example in WSL terminal

@DHowett-MSFT
Copy link
Contributor

DHowett-MSFT commented Jun 25, 2019

make .. scrolling work .. vim

That is not related. That is #376 then #545.

@schorrm
Copy link
Contributor

schorrm commented Jun 27, 2019

Can I pile on here -- can we get two finger scrolling on the mousepad?

@cerebrate
Copy link

A small point that's probably obvious, but just to make sure it's said, can we make sure the default is and the defaults are to pass through mouse events within the terminal to the underlying VT, per #545?

Customizability I like, but not as much as not breaking anything that would otherwise work. If someone's using, for example, tmux mouse support in conhost, they shouldn't have to grovel through the options to figure out why it doesn't work in Terminal.

@zadjii-msft
Copy link
Member

Though it's also being discussed in #524, I want to make sure it's tracked here that some people might not want copy-on-select. So that needs to somehow be included in these bindings. Though, that might make sense as a different setting, since I'm not sure if there are any other "onSelect" actions that would possibly make sense.

@vyevdokimov
Copy link

Add something like "lines to scroll" option - in some cases scroll is too fast in terminal using touchpad.

@getify
Copy link

getify commented Feb 22, 2022

@carlos-zamora thanks for that update. Further question: what about #3990, which is proposed as a shorter path to "fixing" a somewhat breaking behavior of mouse-scroll-ctrl-zooming?

It's a small subset of this larger feature request, but presumably if it's prioritized in that same queue it might be able to be tackled much more quickly (and fix our blocker problem) while this larger feature is worked on.

@zadjii-msft
Copy link
Member

Part of me thinks I'd almost rather figure out the regression than add the escape hatch. This totally seems like the kind of thing that would have regressed in #10051. I've marked #3990 as Help Wanted and Easy Starter. I don't think we're gonna have time to get on those with the other bugs on our plate in 1.14.

That being said, we should get to the bottom of the regression. @getify mind filing a separate issue for the touchpad scrolling regression? I don't want to lose track of that for 1.14

(idly: that might be related to #9955.b)

@getify
Copy link

getify commented Feb 22, 2022

@zadjii-msft The "regression" is not, from what I can tell, in Terminal. I think the regression is Windows and/or the built-in touchpad drivers.

I thought initially this issue was only affecting Terminal; I hadn't experienced the problem in any other apps. That's why I came here. But another app I use does experience this same increase in the issue/sensitivity: Sublime. As far as I can tell, Terminal and Sublime are the only two apps I use which ctrl+scroll causes a zoom behavior at all, so it makes sense they're the only two where the regression/change in sensitivity has become obvious.

I've chased similar rabbit trails with Sublime, looking for some way to disable the ctrl+scroll zooming. They do have keybinding settings, and in a limited way I was able to shut off the annoying zooming behavior. However, their problem is that you can't just tell it to "ignore" the ctrl... you literally disable all actions when ctrl is held down while zooming. That's not what is desired. The outcome currently in Sublime is annoying: if I swipe to scroll, and the inertia is moving the scroll pane, and then I hit the ctrl key, all scrolling abruptly stops. There's no zoom, but I didn't want the scroll to stop like that, so it's not good.

What is desired is a setting that tells the app (Terminal or Sublime), "hey, don't treat ctrl as a special modifier while scrolling, just let scrolling continue to happen as it would if ctrl wasn't pressed".

I bring this detail up to say:

  1. I don't think we need to fix a regression in Terminal
  2. I hope when you add a keybinding or disabling feature into Terminal, you realize the difference between "shut all action down" and "don't pay any attention to this modifier key". The former is what Sublime did, and it sucks. The latter is what I think us users actually want.

@zadjii-msft
Copy link
Member

Weird. Well, looking at the code for a second, I wonder if there's an easier stop gap that could be implemented: when ctrl is pressed, cancel any current momentum in the touch scrolling. I don't even know if that's something that the input APIs let us do. But like, if you did a swipe with the trackpad, then hit Ctrl, then you probably don't want the momentum from that swipe to start zooming...

I did look at the code and I'm pretty sure the regression in #9955.b would also apply to zooming here. This is admittedly psychic debugging at work, but: Precision trackpads send many scroll events with smaller incremental scroll measurements, but I think we just adjust the font size by 1 whole size each event, not based on the actual scroll delta. Unfortunately, the zooming seemed fine on my slaptop 2 so I couldn't repro this specifically.


FWIW, the plan I have in my head for this is a lot like how keybindings work:

[
  { "button": "mouseWheelDown", "command": { "action": "scrollDown", "rollsToScroll": "${scrollDelta}" } }, // or something like that
  { "button": "ctrl+mouseWheelDown", "command": { "action": "scrollDown", "rollsToScroll": "${scrollDelta}" } } // hey look, rebind ctrl+mouse to scroll instead
]

@getify
Copy link

getify commented Feb 22, 2022

when ctrl is pressed, cancel any current momentum in the touch scrolling.

I'm not sure if that's what we want or not... what I prefer is for ctrl to have no effect on scrolling behavior (either directly applied or through momentum) or zooming.

"cancel" implies it interferes in some way, and what I'd rather express is "ignore".

@zadjii-msft
Copy link
Member

Sorry I don't mean "cancel" more, "interrupt"? Like, when the user presses ctrl, maybe we should stop any current momentum that the scrolling might have. Then the user would need to start a new scroll if they wanted to zoom. That might help mitigate the issue a bit.

A similar discussion in another (admittedly totally unrelated) thread:

#3793 (comment)

@getify
Copy link

getify commented Feb 23, 2022

Like, when the user presses ctrl, maybe we should stop any current momentum that the scrolling might have.

That's the behavior I have in my current hack-around in Sublime, and I dislike it. I don't want hitting ctrl to have any effect on my scrolling behavior (momentum or otherwise). I also don't want holding down ctrl before scrolling to prevent the scroll from starting (which is also how my current Sublime behavior is).

@Konstantin-Glukhov
Copy link

How about something like this?

"pointerbindings": [
{
"device": "mouse",
"event": "rightClickOnSelection",
"where": "terminal",
"command": "noAction"
},

@zadjii-msft zadjii-msft removed the Issue-Docs It's a documentation issue that really should be on MicrosoftDocs/Console-Docs label Aug 18, 2022
@samheuck
Copy link

Is this issue dead? It's been... going on nearly 4 years since it was opened now?

I like using "Windows Terminal", but there's still no way I can find to disable "right-click to paste" / "QuickEdit" like you can in PowerShell.

@cloud-66
Copy link

i used to middle click paste, and it's hard for me to use windows terminal without this function. i love wt , but it still doesn't have this.

@korpa
Copy link

korpa commented Feb 11, 2023

i used to middle click paste, and it's hard for me to use windows terminal without this function. i love wt , but it still doesn't have this.

Same here. This is the reason why I switched Windows Terminal to Alacritty a couple of weeks ago.

@amkartashov
Copy link

simple autohotkey script to paste on middle button:

; Paste with middle mouse in Windows Terminal
#IfWinActive ahk_class CASCADIA_HOSTING_WINDOW_CLASS
MButton::Send ^{vk56} ; Send Ctrl-V with middle button
#IfWinActive

@zadjii-msft
Copy link
Member

Is this issue dead? It's been... going on nearly 4 years since it was opened now?

Nope, there's just plenty of other higher-priority work unfortunately. I've got a draft PR for an experimental setting just for the right-click context menu over in #14775. We however don't have a full spec for how we'd actually like to express the fullness of mouse bindings. That's the biggest blocker here - someone taking the time to enumerate what people have asked for and what's possible, and putting them together into a spec.

  • Do folks want to bind arbitrary actions to mouse buttons, or really only a subset of actions?
  • Does this become an extension of bindings, with new "keys" values to indicate mouse buttons?
  • Should configuring the number of lines to scroll be a part of this? Like, rebinding mouseWheelDown to scrollDown(5)? Or is that a separate (but related) setting?
  • How do we represent multi-clicks and multi-click actions? (|double|triple)(Left|Middle|Right)Click seems... verbose...

Those are the thoughts from the top of the dome. If someone wants to put together a proposal, I'd love to review it. Writing the code shouldn't be that hard, so long as we have the right design in place ☺️

@hmb
Copy link

hmb commented Apr 6, 2023

Well, the most pressing issue is probably to just paste the clipboard with the middle mouse button. As many linux terminals, e.g. in VMs, X-Servers like Xming, even in the windows version of git-bash, "paste" is always the middle mouse click. So using the right click in ms-terminal makes it kind of random to which click has to be used:

  • git-bash native -> middle click
  • git-bash embedded in ms-terminal -> right click
  • linux VM -> middle click

Guess what: you always use the wrong one. So in terms of a hamonized user exp would suffice for starters to "paste" with middle mouse button. The "generic covering all eventualities mouse configurator" is probably overkill for all those working in a hybrid environment.

@lucky-wolf
Copy link

Wow, 5y and not solved?

A trivial way to disable "quickedit" would suffice.

@zadjii-msft
Copy link
Member

I mean, there is: experimental.rightClickContextMenu

@TheZetner
Copy link

simple autohotkey script to paste on middle button:

; Paste with middle mouse in Windows Terminal
#IfWinActive ahk_class CASCADIA_HOSTING_WINDOW_CLASS
MButton::Send ^{vk56} ; Send Ctrl-V with middle button
#IfWinActive

Just updated this to AHK 2.0:

; Paste with middle mouse in Windows Terminal
#HotIf WinActive("ahk_class CASCADIA_HOSTING_WINDOW_CLASS")
MButton::Send "^{vk56}" ; Send Ctrl-V with middle button

@psergiu
Copy link

psergiu commented May 23, 2024

It's 2024 and it's still not added :( Some companies don't allow the use of AutoHotKey on work machines. Please just add a simple "X11-style middle mouse button paste" option and it's perfect. If there's time to add more options in the future, even better.
Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Input Related to input processing (key presses, mouse, etc.) Area-Settings Issues related to settings and customizability, for console or terminal Area-TerminalControl Issues pertaining to the terminal control (input, selection, keybindings, mouse interaction, etc.) Help Wanted We encourage anyone to jump in on these. Issue-Scenario Product-Terminal The new Windows Terminal.
Projects
Status: Spec In Progress ✏
Development

No branches or pull requests