UI Automation in Windows Console: Make UIA the default for Windows 10 1803 and later #9771

codeofdusk · 2019-06-18T10:25:14Z

Link to issue number:

Builds on #9614. Closes #673 and #6291.

Summary of the issue:

NVDA's UIA console support significantly improves performance and stability, so we should enable it by default.

Description of how this pull request fixes the issue:

While UIA in consoles was added in Windows 10 1709, the initial implementation left much to be desired, particularly concerning caret movement and expansion to character.

A new UIAUtils.shouldUseUIAConsole function has been added which determines whether to use UIA in consoles:

If the user has explicitly chosen to use UIA or legacy console support, the function returns True or False respectively.
Otherwise, if the user is running at least Windows 10 1803, the function returns True, or False if not.

The "use UI Automation in the Windows Console when available" GUI setting has been changed to expose all options in the config spec (i.e. to automatically detect UIA/legacy or override this autodetection).

Testing performed:

Tested the new GUI setting on Windows 10 1903.

Known issues with pull request:

None.

Change log entry:

== New Features ==

Vastly improved performance and stability in Command Prompt, PowerShell, and the Windows Subsystem for Linux on Windows 10 version 1803 and later. (UI Automation in Windows Console: Make UIA the default for Windows 10 1803 and later #9771)
In Command Prompt, PowerShell, and the Windows Subsystem for Linux on Windows 10 version 1803 and later, the reporting of typed text that does not appear onscreen (such as passwords) can now be toggled in NVDA's advanced preferences. (UI Automation in Windows Console: make speaking of passwords configurable #9649)

codeofdusk · 2019-06-18T10:32:46Z

CC @feerrenrut @LeonarddeR @michaelDCurran

josephsl · 2019-06-18T10:32:51Z

Hi, if you can, can you test this in 20H1 builds too? I myself will do some testing in 20H1 via Windows Sandbox. Thanks.

LeonarddeR

Good to see that this is getting in to shape.

source/UIAUtils.py

source/_UIAHandler.py

source/gui/settingsDialogs.py

user_docs/en/userGuide.t2t

source/gui/settingsDialogs.py

codeofdusk · 2019-06-19T04:42:36Z

Once #9773 is merged, I think this PR is ready for master (pending code review, of course).

lukaszgo1 · 2019-06-21T18:11:12Z

It might be worth to have some discussion about when issue is considered closed. In my view one of the following should happen:

The issue is fixed - doesn't apply to any of the bugs listed above.
The issue is impossible to fix - also not applicable because whine proper fixes aren't easy in these cases it is doable.
The product which is affected by the issue is either EOL or NVDA developers decided to no longer support it. It is important to point out that both Windows 7 and Windows 8.1 would be supported by Microsoft until ad least 2023, and LTSB version of Windows 10 1607 on which Server 2016 is based until 2026.

In short I am against closing issues with the previous console implementations until they are properly fixed. The same has been done when UIA for Word was introduced - no one closed any bug report for COM implementation.

feerrenrut · 2019-07-02T14:25:49Z

@lukaszgo1 Could you be specific about which issue you don't think should be closed to make sure we are on the same page?

source/gui/settingsDialogs.py

user_docs/en/userGuide.t2t

lukaszgo1 · 2019-07-02T14:44:37Z

@feerrenrut well I meant all tickets listed in the PR description by @codeofdusk namely: #513, #673, #1682 and #6291.

source/UIAUtils.py

All actions addressed.

AppVeyorBot · 2019-08-01T20:17:42Z

PR introduces Flake8 errors 😲

See test results for Failed build of commit 88a335629f

AppVeyorBot · 2019-08-01T20:53:16Z

PR introduces Flake8 errors 😲

See test results for Failed build of commit 0e77fe664a

feerrenrut

Thanks @codeofdusk

* Fully revert #9771. * Remove import * Fix accelerator, typedCharacter in UIA console * Style. * More style, except spacing of binary operators (to preserve consistency, as fixing this seems out of scope for this PR). * Also fix palcement of binary operators. * Actually, keep winConsoleUIA module from #9771. * Remove extra and. * Add an and. * Keep auto as the config default. * Style.

…(Windows 11 Sun Valley 2) (#10964) Supersedes #9771 and #10716. Closes #1682. Closes #8653. Closes #9867. Closes #11172. Closes #11554. Summary of the issue: Microsoft has significantly improved performance and reliability of UIA console: * microsoft/terminal#4018 is an almost complete rewrite of the UIA code which makes the console's UIA implementation more closely align with the API specification. * microsoft/terminal#10886, microsoft/terminal#10925, and microsoft/terminal#11253 form a robust testing methodology for the UIA implementation, including bug fixes in response to newly added tests based on Word's UIA implementation. * microsoft/terminal#11122 removes the thousands of empty lines at the end of the console buffer, significantly improving performance and stability. Since all console text ranges are now within the text buffer's bounds, it is no longer possible for console to crash due to the manipulation by UIA clients of an out-of-bounds text range. * Countless other accessibility-related PRs too numerous to list here. We should enable UIA support on new Windows Console builds by default for performance improvement and controllable password suppression. Description of how this pull request fixes the issue: This PR: * Exposes all three options for the UIA console feature flag in the UI (replaces the UIA check box with a combo box). * Adds a runtime check to test if `apiLevel >= FORMATTED`, and use UIA in this case when the user preference is auto. This is the case on Windows 11 Sun Valley 2 (SV2) available now in beta and set for release in the second half of 2022.

LeonarddeR previously requested changes Jun 18, 2019

View reviewed changes

codeofdusk changed the title ~~UI Automation in Windows Console: Make UIA the default for Windows 10 1809 and later (except 1903)~~ UI Automation in Windows Console: Make UIA the default for Windows 10 1809 and later Jun 19, 2019

codeofdusk changed the base branch from threshold to master June 19, 2019 04:34

codeofdusk added 5 commits June 19, 2019 00:35

Implement UIA console auto-detection.

9e2f8ad

Fully enable console support configuration.

3adfe36

Review actions.

1d7c86a

Update user guide.

4877242

Update user guide.

322cd03

codeofdusk force-pushed the cmduia5-makedefault branch from 8042351 to 322cd03 Compare June 19, 2019 04:35

Enable UIA console on Windows 10 1903 and later.

9e2e9da

Merge branch 'master' into cmduia5-makedefault

b9e73c5

codeofdusk added 2 commits June 24, 2019 21:28

Merge branch 'master' into cmduia5-makedefault

9075e2d

Always use custom collapse logic and set minimum version to 1803.

ea43baa

codeofdusk changed the title ~~UI Automation in Windows Console: Make UIA the default for Windows 10 1809 and later~~ UI Automation in Windows Console: Make UIA the default for Windows 10 1803 and later Jun 25, 2019

Update user guide.

79e2aed

feerrenrut reviewed Jul 2, 2019

View reviewed changes

source/gui/settingsDialogs.py Outdated Show resolved Hide resolved

feerrenrut reviewed Jul 2, 2019

View reviewed changes

user_docs/en/userGuide.t2t Show resolved Hide resolved

feerrenrut reviewed Jul 2, 2019

View reviewed changes

user_docs/en/userGuide.t2t Outdated Show resolved Hide resolved

Review actions.

21614ec

feerrenrut reviewed Jul 3, 2019

View reviewed changes

source/UIAUtils.py Show resolved Hide resolved

codeofdusk added 2 commits July 17, 2019 02:53

Merge branch 'master' into cmduia5-makedefault

c8a4fb3

Update user guide.

3ec9b3f

codeofdusk mentioned this pull request Jul 31, 2019

Improve keyboard support for some terminal programs #9915

Merged

codeofdusk added 4 commits August 1, 2019 11:13

Merge branch 'master' into cmduia5-makedefault

9debd58

Merge branch 'master' into cmduia5-makedefault

82c2529

Move winConsoleSpeakPasswords option to terminal programs group.

6ce6921

Update user guide.

fb4961b

Style.

fc80111

Lint.

34faf5f

feerrenrut approved these changes Aug 2, 2019

View reviewed changes

feerrenrut merged commit 216e476 into nvaccess:master Aug 2, 2019

nvaccessAuto added this to the 2019.3 milestone Aug 2, 2019

feerrenrut added a commit that referenced this pull request Aug 2, 2019

Update changes file for PR #9771

2989b57

codeofdusk mentioned this pull request Aug 20, 2019

UI Automation in Windows Console: Don't automatically enable UIA in consoles on Windows 10 version 1803 #10129

Merged

JulienCochuyt mentioned this pull request Sep 20, 2019

Fix up of: remove pyd file in installation folder (#10070) #10224

Merged

codeofdusk mentioned this pull request Sep 25, 2019

UI Automation in Windows Console: backport some UIA console fixes to Python 2 (2019.2.1) #10274

Closed

codeofdusk mentioned this pull request Oct 8, 2019

When NVDA is started from source with python.exe it is not possible to interact with any console window. #8653

Closed

lukaszgo1 mentioned this pull request Nov 9, 2019

Reconsider making UIA default for terminals on Windows 10 #10477

Closed

codeofdusk mentioned this pull request Jan 14, 2020

Use legacy winConsole support by default for 2019.3 #10682

Merged

codeofdusk added a commit to codeofdusk/nvda that referenced this pull request Jan 14, 2020

Fully revert nvaccess#9771.

2e500a8

codeofdusk mentioned this pull request Jan 14, 2020

UI Automation in Windows Console: revert for 2019.3 #10684

Merged

codeofdusk added a commit to codeofdusk/nvda that referenced this pull request Jan 14, 2020

Fully revert nvaccess#9771.

ac1ca1e

codeofdusk added a commit to codeofdusk/nvda that referenced this pull request Jan 14, 2020

Fully revert nvaccess#9771.

3196ed4

codeofdusk added a commit to codeofdusk/nvda that referenced this pull request Jan 14, 2020

Actually, keep winConsoleUIA module from nvaccess#9771.

5e1ca8f

codeofdusk mentioned this pull request Apr 8, 2020

UI Automation in Windows Console: make default in FORMATTED consoles (Windows 11 Sun Valley 2) #10964

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

UI Automation in Windows Console: Make UIA the default for Windows 10 1803 and later #9771

UI Automation in Windows Console: Make UIA the default for Windows 10 1803 and later #9771

codeofdusk commented Jun 18, 2019 •

edited

Loading

codeofdusk commented Jun 18, 2019

josephsl commented Jun 18, 2019 via email •

edited by feerrenrut

Loading

LeonarddeR left a comment

codeofdusk commented Jun 19, 2019

lukaszgo1 commented Jun 21, 2019

feerrenrut commented Jul 2, 2019

lukaszgo1 commented Jul 2, 2019

AppVeyorBot commented Aug 1, 2019

AppVeyorBot commented Aug 1, 2019

feerrenrut left a comment

UI Automation in Windows Console: Make UIA the default for Windows 10 1803 and later #9771

UI Automation in Windows Console: Make UIA the default for Windows 10 1803 and later #9771

Conversation

codeofdusk commented Jun 18, 2019 • edited Loading

Link to issue number:

Summary of the issue:

Description of how this pull request fixes the issue:

Testing performed:

Known issues with pull request:

Change log entry:

codeofdusk commented Jun 18, 2019

josephsl commented Jun 18, 2019 via email • edited by feerrenrut Loading

LeonarddeR left a comment

Choose a reason for hiding this comment

codeofdusk commented Jun 19, 2019

lukaszgo1 commented Jun 21, 2019

feerrenrut commented Jul 2, 2019

lukaszgo1 commented Jul 2, 2019

AppVeyorBot commented Aug 1, 2019

AppVeyorBot commented Aug 1, 2019

feerrenrut left a comment

Choose a reason for hiding this comment

codeofdusk commented Jun 18, 2019 •

edited

Loading

josephsl commented Jun 18, 2019 via email •

edited by feerrenrut

Loading