Improve wrap around/edge behavior when inputing text in IME #3657
Assignees
Labels
Area-Input
Related to input processing (key presses, mouse, etc.)
Area-TerminalControl
Issues pertaining to the terminal control (input, selection, keybindings, mouse interaction, etc.)
Area-UserInterface
Issues pertaining to the user interface of the Console or Terminal
Help Wanted
We encourage anyone to jump in on these.
Issue-Bug
It either shouldn't be doing this or needs an investigation.
Priority-1
A description (P1)
Product-Terminal
The new Windows Terminal.
Resolution-Fix-Committed
Fix is checked in, but it might be 3-4 weeks until a release.
Milestone
Comments
ghost
added
Needs-Triage
philnach
added
Area-Input
5 tasks
DHowett-MSFT
removed
the
Needs-Triage
DHowett-MSFT
pushed a commit
that referenced
this issue
Nov 22, 2019
TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
zadjii-msft
added
the
Area-TerminalControl
cinnamon-msft
added
Help Wanted
ghost
pushed a commit
that referenced
this issue
Mar 19, 2020
## Summary of the Pull Request This PR turns on TextWrapping on `TSFInputControl::TextBlock`. Once the TextBlock hits the end of the Terminal window, it will wrap downwards, but the TextBlock will have as much width as it had when composition started. Unfortunately, this means if composition starts right at the end of the Terminal with enough width for just one character, there will be a vertical line of characters down the right side of the Terminal 😅. It's definitely not ideal, and I imagine this won't be the last time we visit this issue, but for now users can see what they're typing. ## PR Checklist * [x] Closes #3657 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Played around with IME towards the edge of the Terminal window.
ghost
added
the
Resolution-Fix-Committed
|
🎉This issue was addressed in #5005, which has now been successfully released as Handy links: |
This issue was closed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When the cursor is near the right side or bottom of the window, does the canvas appear hanging off the side of the window, or is it clipped by the bounds of the window?
If it's clipped, should we file a follow up to try and position the canvas heuristically based on the cursor position relative to the bounds of the window, to keep it in bounds?
Originally posted by @zadjii-msft in https://github.com/_render_node/MDExOlB1bGxSZXF1ZXN0Mjk2Mzk0Mjcw/pull_requests/unread_timeline
Terminal should do better than the original and wrap around and edge behavior when inputting text in the IME. Current behavior isn't any better than the original command-line (inputted text is off screen). But input should wrap around like in Notepad.
The text was updated successfully, but these errors were encountered: