[FHL] Make VTApiRoutines, which does VT translation for output #11264

miniksa · 2021-09-17T16:43:48Z

Make a VTApiRoutines servicer that does minimal translations instead of
environmental simulation for some output methods. Remaining methods are
backed on the existing console host infrastructure (primarily input
related methods).

PR Checklist

I work here
It's Fix-Hack-Learn quality so it's behind a feature gate so we
can keep refining it. But it's a start!

To turn this on, you will have to be in the Dev or Preview rings
(feature staged). Then add experimental.connection.passthroughMode: true to a profile and on the next launch, the flags will propagate down
through the ConptyConnection into the underlying Openconsole.exe
startup and tell it to use the passthrough mode instead of the full
simulation mode.

Validation Steps Performed

Played with it manually in CMD.exe, it seems to work mostly.
Played with it manually in Ubuntu WSL, it seems to work.
Played with it manually in Powershell and it's mostly sad. It'll get
there.

Starts #1173

miniksa · 2022-01-12T20:57:27Z

Ugh I forgot to add a UI toggle.

miniksa · 2022-01-13T00:57:02Z

Ugh I ruined all the tests...

zadjii-msft

Wow okay I got through it all - honestly wasn't that hard to read for 65 files. Pleasantly refreshing to do a conhost code review.

This is a soft block. There's a LOT of TODOs left in here. Most seem like they're fine for follow-up work. I personally wouldn't bother with separate threads for each of them. I'd take a reserved thread (maybe #10001?), and add each of these lines as a check box in there. Things like the Title work for example - doesn't need an issue, just a note of future work to improve this.

It's all experimental anyways for now, so I'm 🤏 close to signing off.

src/host/lib/hostlib.vcxproj

src/host/ut_lib/host.unittest.vcxproj

src/host/srvinit.cpp

src/host/VtIo.cpp

src/host/VtApiRoutines.cpp

src/renderer/vt/state.cpp

src/renderer/vt/VtSequences.cpp

miniksa · 2022-01-27T22:58:34Z

@zadjii-msft can I get you to pass this one more time?

zadjii-msft · 2022-01-27T23:00:08Z

sure, once I fix the honks

zadjii-msft

You know what, let's just stealth yolo this into 1.13 and see what happens

src/host/VtApiRoutines.cpp

lhecker · 2022-01-31T17:25:44Z

src/cascadia/TerminalControl/ControlCore.cpp

+        auto pfnWriteInput = std::bind(&ControlCore::_sendInputToConnection, this, std::placeholders::_1);
+        _terminal->SetWriteInputCallback(pfnWriteInput);


std::bind is basically deprecated. I'd suggest using lambdas if you can.

I hear you, but the whole remainder of this file is already doing it this way, so I'd rather stick to the pattern if that's OK.

I just don't totally get why this had to change so drastically ;)

I roll my eyes at both of you. Fine. I'll convert it back.

…of environmental simulation for some output methods.

miniksa · 2022-02-23T20:58:44Z

Fixed this too.

EDIT: Proof:

lhecker

yeet

miniksa · 2022-02-23T21:28:03Z

@msftbot merge when @DHowett signs off.

DHowett · 2022-02-24T00:27:18Z

src/cascadia/ut_app/TerminalApp.Unit.Tests.manifest

+        <!-- You apparently are expected to include all the maxversiontested
+        entries separately. They are treated like an array. This one turns on Segoe
+        UI Variable, GH #12452 -->
+        <maxversiontested Id="10.0.22000.0"/>


you may conflict on this file

DHowett

58/67

DHowett · 2022-02-24T00:28:27Z

src/cascadia/TerminalSettingsEditor/Resources/en-US/Resources.resw

@@ -1051,6 +1051,10 @@
    <value>Enable experimental text rendering engine</value>
    <comment>An option to enable an experimental text rendering engine</comment>
  </data>
+  <data name="Profile_VtPassthrough.Header" xml:space="preserve">
+    <value>Enable experimental virtual terminal passthrough</value>
+    <comment>An option to enable experimental virtual terminal passthrough connectivity option with the underlying ConPTY</comment>


This comment could use some work -- it is supposed to be read by a human translator to help them understand what they are translating

DHowett · 2022-02-24T00:31:59Z

src/cascadia/TerminalControl/ControlCore.cpp

+        auto pfnWriteInput = std::bind(&ControlCore::_sendInputToConnection, this, std::placeholders::_1);
+        _terminal->SetWriteInputCallback(pfnWriteInput);


I just don't totally get why this had to change so drastically ;)

DHowett · 2022-02-24T00:33:25Z

src/cascadia/TerminalConnection/ConptyConnection.cpp

+            {
+                if (_passthroughMode)
+                {
+                    WI_SetFlag(flags, PSEUDOCONSOLE_PASSTHROUGH_MODE);


This may not compile in Release. You are using a preprocessor define to control whether the constant is visible, but a constexpr if (which must compile successfully) wherein you are using the constant.

Actually, you're gating the presence of the member using the preprocessor as well.

Try a build from the commandline with /p:WindowsTerminalBranding=Release

I'd say, keep the member and the constant defined and only gate whether the code actually touches them based on if constexpr

lhecker · 2022-02-24T03:39:15Z

I've benchmarked this change and found no regressions.
However I also found that setting the new option has only a small positive impact on throughput of 7.2MB/s -> 7.9MB/s (+9%).
OpenConsole's in-proc architecture still has a significant advantage here with 21MB/s.

miniksa · 2022-02-28T17:09:30Z

OK that's fine. I'm not expecting this change alone to make a massive difference but to rather remove a massive part of the system for further optimization purposes.

lhecker · 2022-02-28T19:09:13Z

OK that's fine. I'm not expecting this change alone to make a massive difference but to rather remove a massive part of the system for further optimization purposes.

The impact is surely greater for VT-heavy applications. But I couldn't test that yet, since the output is broken for my benchmark applications.

# Conflicts: # src/host/lib/hostlib.vcxproj # src/host/ut_lib/host.unittest.vcxproj

src/host/VtIo.cpp

Let the compiler do dead code elimination or whatever

ghost · 2022-03-30T22:52:50Z

Hello @DHowett!

Because this pull request has the AutoMerge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (`@msftbot`) and give me an instruction to get started! Learn more here.

## Summary of the Pull Request When the conpty passthrough mode is enabled, it often needs to send `DSR-CPR` queries (cursor position reports) to the client terminal to obtain the current cursor position. However, the code that originally handled the responses to these queries got broken by the refactoring of the `ConGetSet` API. This PR is an attempt to correct that regression. ## References The conpty passthrough mode was introduced in PR #11264. The refactoring that broke the cursor position handling was in PR #12703. ## PR Checklist * [x] Closes #13106 * [x] CLA signed. * [ ] Tests added/passed * [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx * [ ] Schema updated. * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments Prior to the `ConGetSet` refactoring, the code that handled `DSR-CPR` responses (`InteractDispatch::MoveCursor`) would pass the cursor position to `ConGetSet::SetCursorPosition`, which in turn would forward it to the `SetConsoleCursorPositionImpl` API, and from there to the `VtIo` class. After the refactor, all of those intermediate steps were removed - the cursor was simply updated directly in `InteractDispatch::MoveCursor`, and the `VtIo` call was moved from `SetConsoleCursorPositionImpl` to `InteractDispatch` (since that was the only place it was actually required). However, when the conpty passthrough mode was introduced - which happened in parallel - it relied on the `SetConsoleCursorPositionImpl` API being called from `InteractDispatch` in order to handle its own `DSR-CPR` responses, and that's why things stopped working when the two PRs merged. So what I've done now is made `InteractDispatch::MoveCursor` method call `SetConsoleCursorPositionImpl` again (although without the intermediate `ConGetSet` overhead), and moved the `VtIo::SetCursorPosition` call back into `SetConsoleCursorPositionImpl`. This is not ideal, and there are still a bunch of problems with the `DSR-CPR` handling in passthrough mode, but it's at least as good as it was before. ## Validation Steps Performed I've just manually tested various shells with passthrough mode enabled, and confirmed that they're working better now. There are still issues, but nothing that wasn't already a problem in the initial implementation, at least as far as I can tell.

## Summary of the Pull Request When the conpty passthrough mode is enabled, it often needs to send `DSR-CPR` queries (cursor position reports) to the client terminal to obtain the current cursor position. However, the code that originally handled the responses to these queries got broken by the refactoring of the `ConGetSet` API. This PR is an attempt to correct that regression. ## References The conpty passthrough mode was introduced in PR #11264. The refactoring that broke the cursor position handling was in PR #12703. ## PR Checklist * [x] Closes #13106 * [x] CLA signed. * [ ] Tests added/passed * [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx * [ ] Schema updated. * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx ## Detailed Description of the Pull Request / Additional comments Prior to the `ConGetSet` refactoring, the code that handled `DSR-CPR` responses (`InteractDispatch::MoveCursor`) would pass the cursor position to `ConGetSet::SetCursorPosition`, which in turn would forward it to the `SetConsoleCursorPositionImpl` API, and from there to the `VtIo` class. After the refactor, all of those intermediate steps were removed - the cursor was simply updated directly in `InteractDispatch::MoveCursor`, and the `VtIo` call was moved from `SetConsoleCursorPositionImpl` to `InteractDispatch` (since that was the only place it was actually required). However, when the conpty passthrough mode was introduced - which happened in parallel - it relied on the `SetConsoleCursorPositionImpl` API being called from `InteractDispatch` in order to handle its own `DSR-CPR` responses, and that's why things stopped working when the two PRs merged. So what I've done now is made `InteractDispatch::MoveCursor` method call `SetConsoleCursorPositionImpl` again (although without the intermediate `ConGetSet` overhead), and moved the `VtIo::SetCursorPosition` call back into `SetConsoleCursorPositionImpl`. This is not ideal, and there are still a bunch of problems with the `DSR-CPR` handling in passthrough mode, but it's at least as good as it was before. ## Validation Steps Performed I've just manually tested various shells with passthrough mode enabled, and confirmed that they're working better now. There are still issues, but nothing that wasn't already a problem in the initial implementation, at least as far as I can tell. (cherry picked from commit e1086de) Service-Card-Id: 82005205 Service-Version: 1.14

ghost · 2022-05-24T17:06:58Z

🎉Windows Terminal Preview v1.14.143 has been released which incorporates this pull request.:tada:

Handy links:

miniksa force-pushed the dev/miniksa/vtapi branch from ba6895a to a19198a Compare September 17, 2021 21:51

This comment has been minimized.

Sign in to view

miniksa changed the title ~~[DRAFT] Make a VTApiRoutines servicer that does minimal translations instead of environmental simulation for some output methods~~ [FHL] Make a VTApiRoutines servicer that does minimal translations instead of environmental simulation for some output methods Sep 17, 2021

WSLUser mentioned this pull request Oct 14, 2021

Add keyboard protocol support (the kitty one) #11509

Open

zadjii-msft mentioned this pull request Oct 28, 2021

Introduce AtlasEngine - A new text rendering prototype #11623

Merged

miniksa marked this pull request as ready for review January 12, 2022 00:41

zadjii-msft requested changes Jan 13, 2022

View reviewed changes

ghost added the Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something label Jan 13, 2022

ghost removed the Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something label Jan 13, 2022

miniksa mentioned this pull request Jan 13, 2022

VT Passthrough enhancements beyond the FHL #10001

Open

16 tasks

zadjii-msft approved these changes Jan 28, 2022

View reviewed changes

src/host/VtApiRoutines.cpp Outdated Show resolved Hide resolved

zadjii-msft added the Needs-Second It's a PR that needs another sign-off label Jan 28, 2022

ghost requested review from carlos-zamora, DHowett, leonMSFT, PankajBhojwani and lhecker January 28, 2022 12:13

lhecker reviewed Jan 31, 2022

View reviewed changes

miniksa added 4 commits February 11, 2022 15:45

Make a VTApiRoutines servicer that does minimal translations instead …

eebed6a

…of environmental simulation for some output methods.

the rest of the stuff I did on Tuesday.

759a919

cooked read in cmd works omg.

9325839

remove one bit of badness

3314e4a

lhecker approved these changes Feb 23, 2022

View reviewed changes

DHowett reviewed Feb 24, 2022

View reviewed changes

DHowett requested changes Feb 24, 2022

View reviewed changes

ghost added the Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something label Feb 24, 2022

ghost removed the Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something label Feb 25, 2022

address feedback

df4614e

DHowett added 2 commits March 21, 2022 15:12

Merge remote-tracking branch 'origin/main' into dev/miniksa/vtapi

d6d8521

Merge remote-tracking branch 'origin/main' into dev/miniksa/vtapi

b348b29

# Conflicts: # src/host/lib/hostlib.vcxproj # src/host/ut_lib/host.unittest.vcxproj

DHowett reviewed Mar 30, 2022

View reviewed changes

src/host/VtIo.cpp Outdated Show resolved Hide resolved

Only instantiate VtApiEtc if the feature is on

61e8822

Let the compiler do dead code elimination or whatever

DHowett approved these changes Mar 30, 2022

View reviewed changes

DHowett changed the title ~~[FHL] Make a VTApiRoutines servicer that does minimal translations instead of environmental simulation for some output methods~~ [FHL] Make a VTApiRoutines servicer that does VT translation for output Mar 30, 2022

DHowett changed the title ~~[FHL] Make a VTApiRoutines servicer that does VT translation for output~~ [FHL] Make, VTApiRoutines, which does VT translation for output Mar 30, 2022

DHowett changed the title ~~[FHL] Make, VTApiRoutines, which does VT translation for output~~ [FHL] Make VTApiRoutines, which does VT translation for output Mar 30, 2022

DHowett added the AutoMerge Marked for automatic merge by the bot when requirements are met label Mar 30, 2022

ghost merged commit 2c2f4f9 into main Mar 30, 2022

ghost deleted the dev/miniksa/vtapi branch March 30, 2022 23:22

zadjii-msft mentioned this pull request Apr 4, 2022

How about removing text buffer from conhost #1739

Closed

j4james mentioned this pull request Apr 10, 2022

Further refactor and simplify the ConGetSet API #12703

Merged

j4james mentioned this pull request May 15, 2022

Fix handling of cursor position reports in passthrough mode #13109

Merged

6 tasks

carlos-zamora mentioned this pull request May 16, 2022

Document experimental.connection.passthroughMode MicrosoftDocs/terminal#541

Merged

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FHL] Make VTApiRoutines, which does VT translation for output #11264

[FHL] Make VTApiRoutines, which does VT translation for output #11264

miniksa commented Sep 17, 2021 •

edited by DHowett

Loading

This comment has been minimized.

miniksa commented Jan 12, 2022

miniksa commented Jan 13, 2022

zadjii-msft left a comment

miniksa commented Jan 27, 2022

zadjii-msft commented Jan 27, 2022

zadjii-msft left a comment

lhecker Jan 31, 2022

miniksa Feb 11, 2022

DHowett Feb 24, 2022

miniksa Feb 25, 2022

miniksa commented Feb 23, 2022 •

edited

Loading

lhecker left a comment

miniksa commented Feb 23, 2022 •

edited

Loading

DHowett Feb 24, 2022

DHowett left a comment

DHowett Feb 24, 2022

DHowett Feb 24, 2022

DHowett Feb 24, 2022

DHowett Feb 24, 2022

DHowett Feb 24, 2022

lhecker commented Feb 24, 2022

miniksa commented Feb 28, 2022

lhecker commented Feb 28, 2022 •

edited

Loading

ghost commented Mar 30, 2022

ghost commented May 24, 2022

		auto pfnWriteInput = std::bind(&ControlCore::_sendInputToConnection, this, std::placeholders::_1);
		_terminal->SetWriteInputCallback(pfnWriteInput);

[FHL] Make VTApiRoutines, which does VT translation for output #11264

[FHL] Make VTApiRoutines, which does VT translation for output #11264

Conversation

miniksa commented Sep 17, 2021 • edited by DHowett Loading

PR Checklist

Validation Steps Performed

This comment has been minimized.

miniksa commented Jan 12, 2022

miniksa commented Jan 13, 2022

zadjii-msft left a comment

Choose a reason for hiding this comment

miniksa commented Jan 27, 2022

zadjii-msft commented Jan 27, 2022

zadjii-msft left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

miniksa commented Feb 23, 2022 • edited Loading

lhecker left a comment

Choose a reason for hiding this comment

miniksa commented Feb 23, 2022 • edited Loading

Choose a reason for hiding this comment

DHowett left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lhecker commented Feb 24, 2022

miniksa commented Feb 28, 2022

lhecker commented Feb 28, 2022 • edited Loading

ghost commented Mar 30, 2022

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

ghost commented May 24, 2022

miniksa commented Sep 17, 2021 •

edited by DHowett

Loading

miniksa commented Feb 23, 2022 •

edited

Loading

miniksa commented Feb 23, 2022 •

edited

Loading

lhecker commented Feb 28, 2022 •

edited

Loading

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (`@msftbot`) and give me an instruction to get started! Learn more here.