Make numbered browser tabs clickable by voice

[tigersof]

[feature request]

Problem Identified:

I am reliant on voice recognition and struggle to access browser tabs easily by voice. Dragon Professional Individual has not implemented a way to achieve this in its add-on extension for Firefox and there is no Firefox ActiveX component available for Dragon’s advanced scripting.

With Dragon (see numbers on image for reference):

1. I can say “Ctrl 1” to click tab “1”, “Ctrl 2” to click tab “2”, “Ctrl 3” to click tab “3” and so on, up to “Ctrl 9” for tab “9”.
2. Due to obvious keyboard limitations, I cannot voice-click any tabs beyond tab 9. I usually have a lot more tabs open, so I need the ability to voice-click those too.

My attempt at a workaround within the Dragon environment is to implement a command that performs “Ctrl 1” and then iterates down the tabs with “Ctrl Tab” until the tab I want is reached. It works, but not always and it is inefficient and resource intensive, and often gets stuck due to pages reloading on the way down. As another example, I had also tried commands such as saying “Ctrl 123” but this merely resulted in the “Ctrl 1” outcome, and similarly for all other numbers over 9.

Feature Request:

I have been using Tree Style Tab extensively, it’s a great extension and I find it really useful. As you can see, I have flattened the hierarchy as this better meets my needs with voice recognition, and I have inserted some of your code snippets for tab counting and numbering.

With Tree Style Tab (again, refer to numbers in image):

2. Is there a way to make these tab numbers clickable by voice? As one possible idea I have, can they be hyperlinked within TST, with additional perhaps hidden text (e.g. "t123" so that I can tell Dragon to click marker labelled "Tab 123" to access it) to prevent voice-clicking conflicts with other similarly named links on the page or elsewhere within the Firefox window? Or is there a better way to voice-access the tabs within TST via their number?
3. How can I remove the column of “X” (for closing tabs, which I can do by voice)?
4. How can I move my tab numbers to the right hand side (to replace the “X” column)?

I should add that I have next to no programming ability at all; most of what I do is tweaking what's already there to make it work for me. It’s really basic and usually more “miss” than “hit” but I try!

I’d really appreciate your help to resolve this issue. Thanks for making such a great extension!

[piroor]

2. Is there a way to make these tab numbers clickable by voice?

For now, it looks need to create a helper addon inserting a clickable link via TST's API https://github.com/piroor/treestyletab/wiki/Extra-Tab-Contents-API and listen events on them.

3. How can I remove the column of “X” (for closing tabs, which I can do by voice)?

You can hide it via the custom user styles like:

tab-closebox {
  display: none !important;
}

4. How can I move my tab numbers to the right hand side (to replace the “X” column)?

If you inserted it via custom user styles, you need to change some style declarations. Which is the one do you use?

---

Currently TST builds tab UI with no HTML link, because there was no reason to make them detectable from someone outside of TST. If the software Dragon Professional Individual allows you to click any link in TST's sidebar, I think I need to render TST's tabs as HTML link for better accessibility.

[piroor]

I've experimentally introduced a change to make tab labels clickable link. You can try a development build as an artifact of https://github.com/piroor/treestyletab/actions/runs/762405117#artifacts , could you confirm those tab labels are detectable and clickable from Dragon Professional Individual?

[piroor]

Here is another development build using ARIA role "button" for each tab.
https://github.com/piroor/treestyletab/actions/runs/762444852#artifacts
Could you try this also?

[tigersof]

Piroor, thank you so much for doing this – for answering my call, and acting so quickly. I am really happy that I raised this request. You have just about solved a long-standing problem for me and, I hope many others too. There are still a couple things I’d like you to look at further, so please see the details below.

P.S. I hope this is not too long – I just wanted to make sure I covered as much as possible to help you understand in going forward with future development. I hope it is useful.

Artifact 1

I was not able to voice-activate these tabs. Is there something that you implemented and that I need to be saying to do this?

Manually clicking and holding shift to select multiple tabs in TST tab pane (I have 20 in the original window which you cannot see here, like the image I uploaded yesterday) causes a new window to open, as per the above image.

Left hand side zoomed in:

Right hand side zoomed in:

Artifacts 2 & 3

Generally, these work well, and I activated both by simply saying the numbers as I could see them. I am not sure if by “label” or “ARIA” this meant you had included a voice-clickable label that is different to the tab titles shown. I could not see this if this was the case. Please explain further if I have misunderstood this.

Unfortunately, I am unable to illustrate the issues that I had with these artifacts with images because, as soon as I tried to create a screenshot, the flags created by Dragon would disappear. I will try to describe some of the effects that I observed with both artifacts:

1. If I say “3”, not only is tab 3 flagged up, but also several other tabs with the number “3” in their title. Similarly for numbers “8”, “9” and “10” in my current testing example.
2. If I say “4”, the word “for” is flagged up in the TST list of tab page titles, but number “four” or “4” is not (neither in counter number nor title). Yet, both “4” and “for” are present in the TST tab list. “To” and “2” is another example. Dragon understands numbers in both their word and digit format (i.e. that 4 = four, 34 = thirty-four etc), and should otherwise normally seek out both at the same time and resolve as appropriate.
3. Numbers 38, 40, 30, and 80, refused to be called by Dragon! I do not understand this, because Dragon normally dictates them accurately, and these numbers are visibly present as tab counter numbers. Dragon seems to indicate that these “words” were not available/present to click. Is it possible that some invisible code within TST is affecting this?
4. If it only finds one instance of the “number” spoken, it will click it directly, even if that was not intended (e.g. it found one instance of the word “to” when I was targeting “52”. Sometimes this happens when a user has clearly enunciated, say, “52”, but Dragon only picked up the last digit due to resource lag and misrecognised it as the word “to” instead). Not only can this be disorientating for the user but may also very occasionally cause serious problems (e.g. like deleting files etc) and it can be incredibly challenging due to Dragon’s speed to work out what action Dragon took. Other common examples include Fife (5) and line (9) etc.

I expect that there will be other Dragon related quirks that I have not yet identified, but I think this underlies the need for targeting the tabs more closely if possible. (update: following my CSS modification – see further below - I feel this is much less likely to be a problem now).

I have tested over 125 different tabs opened for this test run in number order from 1 onwards and, for the most part, with the exceptions noted above, it runs very smoothly and elegantly.

Of the two, I think the ARIA version (Artifact 3) seemed to have the edge over Artifact 2 although, other than it seems more reliable and a bit quicker, I cannot really explain why. They were both subject to similar issues to one degree or another.

That said, I did have to restart the computer at one point and when I retested Artifact 2, its performance dropped significantly. So far, the ARIA version seems to be performing well between restarts.

Neither of these versions reproduce the issue of the first artifact where manually selecting multiple tabs induces the new window issue seen in the image above (which is good!).

Hierarchical Tabs

Artifacts 2 & 3 do not work so well with hierarchical tabs. This is because the counter enumerates visible tabs only and collapsing and expanding parent tabs causes the counter to be reset each time. Consequently, when a user tries to voice-click a parent/independent tab further down from the current child or parent tab selected, the user will end up further (i.e. equal to total # of children of the previous parent) down the tab list than intended and will have to issue an additional command to correct this.

However, hierarchical tabs will probably work very similarly to the flattened alternative (as I am using) if the tab groups are all expanded and made non-collapsible.

Removing “X” Close Tab Symbols

tab-closebox {
display: none !important;
}

Thank you, I have done this and it works great.

Moving Tab Numbering to RHS

”If you inserted it via custom user styles, you need to change some style declarations. Which is the one do you use?”

• This is currently what I am using (as I do not know which bit is relevant here, I’m posting it all!):

tab-item {
--tab-size: 25px !important;
height: var(--tab-size);
}
tab-item {
--tab-margin-left: 3px;
--tab-margin-right: 3px;
}
/* As little space before the tab name as possible.
The fold/unfold icon is not affected. */
tab-item:not(.pinned) {
padding-left: 15px !important; /* !important is required when there are enough tabs to cause a scrollbar */
}

/* colour tab based on its container colour */
:root.sidebar .contextual-identity-marker {
margin: 0 !important;
position: absolute !important;
pointer-events: none;Full Auto-show/hide Theme
z-index: 0;

bottom: 0 !important;
left: 0 !important;
right: 0 !important;
top: 0 !important;

width: unset !important;
height: unset !important;

opacity: 0.5;
}
:root.sidebar tab-label {
z-index: 1000;
}
/*Tab Counter (for Numbering) */
#tabbar {
counter-reset: vtabs atabs tabs;
/* vtabs tracks visible tabs, atabs tracks active tabs, tabs tracks all tabs */
}
tab-item:not(.collapsed):not(.discarded) {
counter-increment: vtabs atabs tabs;
}
tab-item:not(.collapsed) {
counter-increment: vtabs tabs;
}
tab-item:not(.discarded) {
counter-increment: atabs tabs;
}
tab-item {
counter-increment: tabs;
}
/* Tab Numbering (requires Tab Counter above) */
tab-item .extra-items-container.behind {
z-index: unset !important;
}
tab-item .extra-items-container.behind::after {
background: white; /* Highlight */
opacity: 50%;
color: #1a1a1a; /* HighlightText */
content: counter(vtabs);
font-size: x-small;
font-weight: bold;
left: 0.1em;
padding: 0.2em;
pointer-events: none;
position: absolute;
bottom: 0.2em;
margin-bottom:3px;
z-index: 1000;
}

Additional CSS Modification by Myself

Due to the problems that I was having with certain number recognition in Artifacts 2 and 3, I further tweaked the CSS via the advanced section of TST’s options in Firefox. I followed the example of your tab numbering code to add in the word “tab”, change the word “::after” to “::before”, and then set the opacity to 0% to hide it. It is probably not as “clean” a solution as you would like it to be from a programming point of view, but it works well to resolve this issue between “for” and “4”, etc and in effect, ensures tabs are targeted much more accurately while forcing number recognition. I will leave it to you to decide if it needs tidying up for future public release!

So, for now, I have added the following to the end of my CSS customisation:

tab-item .extra-items-container.behind::before {
content:"tab";
font-size:10px;
color: red; /* just shows me it’s there and hidden when opacity is varied */
opacity:0%;
}

This part of voice enabled TST is now working almost perfectly, apart from a few numbers that Dragon still fails to recognise. I have taken to training it in the hope that this will iron out those issues, but for me it is not a particularly big issue as I have a Dragon command that can handle moving up or down one tab without exccessive resource burdens.

Either way, TST tabs are now reliably voice-clickable whether using the tab counter numbering, or the tabs’ page titles as displayed to the user. It is also very light on system resources.

I should also add here that to use this function with this method, users will need to say, “Click Tab #”. They cannot simply say “Tab #” as this conflicts with the key of the same name. For example, saying “Tab #” would instruct Dragon to hit the tab key # times, which iterates down through the links on the webpage. It would not click any of TST’s tabs.

There is the possibility also, of course, of swapping the word “tab” in the custom CSS for a unique word that is unlikely to clash with anything else important, such that the user could say “X #” (e.g. “Unicorn 7” etc), without needing the word “Click” at all. (update: I have just tested this and it works very well).

And another lovely thing about this is that neither I, nor any other users, need create a separate Dragon command to action these voice commands, so Dragon Home users would still benefit.

It is Dragon's Firefox web extension that enables the clicking of links btw. Voice-clickable TST won't work without this extension and obviously, the Dragon speech engine behind it.

Voice-Right-Click Context Menu

Moreover, an excellent unanticipated bonus and one that I am delighted about, is that I can now voice-right-click tabs as well. Unlike before, I am now able to perform all the other functions of multiple extensions sharing the same tab context menu. This is because your new code focuses the mouse pointer over the tab when it is voice-clicked.

Voice-Scrollable TST Tab Pane/Voice-Clickable Scrollbar

As expected, only the tabs visible to the user are voice-clickable, which means the user needs to scroll up/down separately to see other tabs to voice-click them. In this respect, I have another related request, which I hope will not be too much trouble…

With Dragon, I can issue a command to drag the mouse down or up, dragging an object with it. If I manually hover the mouse over the TST scrollbar and issue this command, the TST scrollbar will move up and down accordingly. This is the only method I can use right now, which can be difficult for some users, including myself.

Alternatively, when the main browser page has focus, I can simply issue a command to “start scrolling up” or “start scrolling down”, or “Page Up”, “Page Down”, and “# Line(s) Up/Down” and the page will immediately respond accordingly, without the need for dragging and completely hands-free. I currently cannot do any of this with the TST pane.

• Can the TST scrollbar be made voice-clickable, and/or, the TST tab pane be made voice-scrollable?

This would really complete it very nicely and be the icing on the cake, please!

[tigersof]

Hi Piroor,

I'd like to just add another thought here. I'm able to fit 30+ tabs vertically in clear view down the right-hand side of the screen. With so many active tab page titles and lots of links in the main browser page, it is possible for Dragon to click a page link rather than the intended tab page title instead, and vice versa, especially if the tabs are of a subject related to the page you are currently reading. This is why the numbering is useful, and it helps to mitigate against unspeakable tab titles (i.e. user-unfriendly code-like strings such as document/file paths/names, which can be difficult to manage with Dragon for the average user).

In this respect, it would be very practical to optionally be able to deactivate TST Tab title links, and rely on the tab counters to access tabs, leaving one free to speak any web page links without fear of Dragon clicking unintended tab title links to elsewhere. When this happens, there is always a sense of "what just happened, where am I, where was I, how do I get back?" type thinking and sometimes it can take a while to sort it out. Dragon is very fast, and this happens in the blink of an eye, so is difficult to track.

• Is there a way to optionally deactivate TST tab title links in favour of voice-clickable tab counters?

Many thanks again.

[piroor]

Thank you for testing! Finally I've decided to revert changes to embed links. Thus finally ARIA-role is applied on the next version. WAI-ARIA is a specification about Web application accessibility, which gives hints for screen reader and similar other applications to control UIs built on HTML. I'm very sorry I didn't write ARIA role on my addons because I thought that TST is strongly depends on its visual UI and it is not suitable to be used with any accessibility software.

Artifact 1

That was caused because the tab label link is built as an HTML link element like <a href="#">tab label text</a>. When you middle-click or ctrl-click on that, Firefox opens the link URL in a new tab - then Firefox uses the URL of the sidebar panel itself. While I'm dogfooding the HTML link version development build, I also saw such problems and finally I had to give up preventing such behaviors of Firefox.

Artifacts 2 & 3

I think that those problem may be caused by Dragon side. TST just declare each tab as a controllable button, and there is no code applying special things for tabs with some specific numbers.

Hierarchical Tabs

TST's twisty (the small chevron to toggle tree expanded state) and close button in each tab didn't have any accessibility information. With recent commits now those buttons have a role "button" and labeled for accessibility applications. I hope that these changes make them voice-clickable. Here is a development build: https://github.com/piroor/treestyletab/actions/runs/770179253#artifacts

If this change breaks accessibility of tabs actually, I need to plan something more changes to make both in-tab UI and tabs voice-clickable. For example: giving "button" role for tab labels, not tabs themselves.

Moving Tab Numbering to RHS

I've tried the CSS and I just had to change one line in the last declaration:

tab-item .extra-items-container.behind::after {
(snip)
/* left: 0.1em; *//* removed */
right: 0.1em; /* added */
(snip)
}

Can the TST scrollbar be made voice-clickable, and/or, the TST tab pane be made voice-scrollable?

Hmm, sorry I don't know how to do that...
I experimentally declare the list of tabs as a listbox. Could you try a development build with this change?: https://github.com/piroor/treestyletab/actions/runs/770355247#artifacts

[piroor]

I've applied more changes to TST to provide accessibility information. Current definitions:

• tab bar: role=listbox, aria-multiselectable=true, aria-activedescendant=(tab ID)
  • tab: role=option, aria-selected=true/false (synchronized to highlighted state of tabs), tabindex=0
    • twisty: role=button, tabindex=0
    • mute/unmute: role=button, tabindex=0
    • closebox: role=button, tabindex=0

I've tried Windows 10's "narrator" feature to test accessibility. With these attributes tabs and in-tab buttons became focusable. But I'm not sure that Dragon recognizes these information. Here is a development build: https://github.com/piroor/treestyletab/actions/runs/770820371#artifacts

[piroor]

After more trials I've understood that tabindex=0 of UI elements in the sidebar steals focus while you press the Tab key to move focus between links and buttons. It looks very annoying and breaks user experience, so I've removed all tabindex attribute from UI elements. Currently applied accessibility information are:

• tab bar: role=listbox, aria-multiselectable=true, aria-activedescendant=(tab ID)
  • tab: role=option, aria-selected=true/false (synchronized to highlighted state of tabs)
    • twisty: role=button
    • label: role=button
    • mute/unmute: role=button
    • closebox: role=button
• Add Tab button: tabindex=-1

I'm afraid that both label with role=button and tab with role=option are recognized by Dragon and it possibly break user experience more, so I need more testing. The development build with roles described above is: https://github.com/piroor/treestyletab/actions/runs/771015894#artifacts

[piroor]

I've tried another screen reader NVDA, and confirmed that it can detect tabs and in-tab buttons as operatable objects in its "object navigation" mode. From this result I think that Dragon also lists tabs and in-tab buttons clickable.

[piroor]

Is there a way to optionally deactivate TST tab title links in favour of voice-clickable tab counters?

Before implementing this kind option, could you try a newer (latest) development build? With recent changes, now TST's tab bar may be detected as a selectable list (like Firefox's bookmarks sidebar), by other applications. Indeed when I tried Windows 10 narrator, it said like "tab, 40 of 50" for 40th tab in a window containing 50 tabs. If Dragon also provides any way to choose specific index item in a listbox, it can help you to move focus to any tab quickly.

[tigersof]

Hi Piroor, I have tested all the artefacts now. I had some issues with my computer/software and then ran into problems with the artefacts behaving differently. So, I will go over it again to be sure before coming back to you. Before I do, I just need to ask whether the existing TST extension should always be removed, and firefox restarted, before installing the next one? Would this make any difference?

I'll try to do all this over the weekend. Thank you for your patience.

[piroor]

I think you don't need to uninstall old experimental builds. Steps to try are:

1. TST Options => Advanced => uncheck "Optimize tree restoration with cache"
2. Load a new experimental build with about:debugging => "My Firefox" => "Temporary Extensions" => "load Temporary Add-on..."
3. Try the operation to operate tabs via Dragon.
4. Load another build with "load Temporary Add-on..."
5. Try the operation to operate tabs via Dragon.
6. Repeat same steps.

[piroor]

You can download each experimental build for every commits: https://github.com/piroor/treestyletab/actions

[tigersof]

Thank you.

I tried the uncheck "Optimise tree restoration with cache" setting, only this didn't work; the tree disappeared and I couldn't get it back at all until I re-checked the box and reloaded the tree again.

One more thing I want to ask you before continuing, are there any specific keywords that I need to say (to Dragon) to activate these controls? (I have tried words like "button", "twisty", "list item", "chevron", "toggle", "expand tree", "collapse tree", etc - similarly for the tab titles, and numbers).

I have been trying a range of different words, and mostly it doesn't work. I'm going to retest everything again, because the tree still isn't working with Dragon currently, whereas I had previously got some response in one or two of the artefacts, and voice-clicking tabs had worked well.

[piroor]

Hmm, deactivating "Optimize tree restoration with cache" just means that TST always build its UI from scratch. Such a full build is slow but safe on most cases, opposite building UI with a cache is fast and unstable...

OK, anyway I've added a new option to change accessibility role of tab elements. You can change it via TST options => Development => All Configs => tabAccessibilityRoleMode (sorry poor UI - currently this is just an experimental.) 1 means button, it looked to work better on your environment. 2 means listbox and option, I think it is meaningful (but it looks to have less accessibility on your environment.) It is 1 by default.

One more thing I want to ask you before continuing, are there any specific keywords that I need to say (to Dragon) to activate these controls? (I have tried words like "button", "twisty", "list item", "chevron", "toggle", "expand tree", "collapse tree", etc - similarly for the tab titles, and numbers).

Currently the tab label is the only one readable text in each tab. Twisty has its aria-label same to its tooltip text ("Collapse this tree" or "Expand this tree") and closebox also Close this tree or Close this tab. If Dragon simply ignores those accessibility labels, we looks to need to embed such a text as a text node instead of an attribute.

[tigersof]

Hi Piroor,

Is there a way we can make this easier, please? I've tried multiple times now to test and report on the testing but, actually, I'm finding it too challenging. There is a lot of inconsistency between repeat test runs on the same artefacts, which is confusing me. Also, I'm trying to assess Twisty buttons and tabs, and both seem to have a number different and overlapping behaviours associated with them, which I'm not all that conversant with in trying to pick apart. I'm also struggling with being concise, and not overload with too many words (which is a problem for me anyway, even if it isn't for you!). So I also tried devising some data/spreadsheet to get round this, but my lack of inside knowledge on how the Tab Pane runs really makes that harder, plus, as I said before, the betwen test inconsistency throws me. I have a limited amount of energy and ability to concentrate due to disability and, apart from day-to-day life generally, have also run into software battles, micophones etc not behaving properly, which has absorbed a lot of my time. It's working OK now, although I don't think Word is going to play so nicely still, resulting in me getting keyboard heavy, which I desperatly need to avoid!

I really want to help, not just for myself, but for anyone else whom this would benefit. If we can prioritise the artefact and the aspect you'd like feedback on first, that might be more doable for me. And then, gradually work through it that way as needed. Does that sound OK to you?

[piroor]

Finally I've given up to debug remotely and bought the license of Dragon Home... Now I'm testing interactions on my PC.

[tigersof]

Oh, wow! Thank you! I wish you didn't have to do that, but I really appreciate your effort to help.

I was just writing up some general observations on attempting to voice-click tabs and how the webpage can interplay with it etc. I'll post this anyway, later tonight/tomorrow.

As for specific tab and twisty behaviours, I'm still happy to test but in a more step-by-step process. Just let me know.

Thank you Piroor, you're a star!

[piroor]

After some trials I've understood following things:

• <tab-item> with role="button" is actually effective. When I say "click (tab title)" exactly, Dragon clicks the center of the tab as expected.
• However, even if I add a visible tooltip with a CSS hack based on ::before (or::after) and content, it can't be clicked, because they are detected as unclickable elements basically.
• Moreover, role="button" element inside another role="button" element is impossible to be clicked. Clicking by Dragon on a nested buttons always treated as a click on the topmost parent button.

So, I've applied some changes:

• Removed --moz-user-focus: ignore !important from tab elements.
• Set role of tab elements as:
  • tab-item: option in a listbox
  • tab-label, tab-twisty, tab-closebox, tab-sound-button: button

The experimental build with these changes is available at:

https://github.com/piroor/treestyletab/actions/runs/806660404#artifacts

On this revision, I've successfully click in-tab buttons via Dragon with a user style:

#tabbar {
  counter-reset: vtabs atabs tabs;
  /* vtabs tracks visible tabs, atabs tracks active tabs, tabs tracks all tabs */
}
tab-item:not(.collapsed):not(.discarded) {
  counter-increment: vtabs atabs tabs;
}
tab-item:not(.collapsed) {
  counter-increment: vtabs tabs;
}
tab-item:not(.discarded) {
  counter-increment: atabs tabs;
}
tab-item {
  counter-increment: tabs;
}

tab-item,
tab-label,
tab-twisty,
tab-closebox,
tab-sound-button {
  -moz-user-focus: normal !important;
}

tab-item:not(.faviconized) .extra-items-container.front::after,
tab-item.faviconized tab-favicon::after {
  content: counter(vtabs);
  background: Highlight;
  bottom: 0.2em;
  color: HighlightText;
  font-size: x-small;
  right: 0;
  padding: 0.2em;
  pointer-events: auto;
  position: absolute;
  user-select:text;
  -moz-user-focus: normal;
  z-index: 1000;
}

tab-label::after,
tab-twisty::after,
tab-closebox::before,
tab-sound-button::before {
  max-height: 0;
  max-width: 0;
  opacity: 0;
  pointer-events: auto;
  position: absolute;
  user-select:text;
  -moz-user-focus: normal;
}

tab-label::after {
  content: "Tab " counter(vtabs);
}

tab-twisty::after {
  content: "V" counter(vtabs);
}

tab-closebox::after {
  content: "X" counter(vtabs);
}

tab-sound-button::after {
  content: "M" counter(vtabs);
}

The last four declarations define "invisible clickable label" for each button. When I say "click tab 4", the tab label of the 4th tab is clicked and I successfully select the tab. When I say "click V4", the twisty of the 4th tab is clicked and I successfully toggle the tree under the 4th tab collapsed/expanded.

My pronunciation is strongly accented and Dragon isn't trained well for Japanese accented English so it was very hard for me to control tabs via voice commands, but hidden labels added with the custom user style are detected as expected when Dragon correctly detected my voice.

[piroor]

On the other hand, scrolling problem is not resolved yet. I still don't know how to scroll the sidebar via Dragon. Maybe we need to implement something buttons to scroll the sidebar contents, if Dragon doesn't support operations on scrollable HTML elements. Leastwise I couldn't scroll it via voice commands like "scroll up/down".

[tigersof]

Hi Piroor,

This is great, I am still testing most of this. I just wanted to step in quickly and let you know that you have already partially solved the scrolling problem!

I am still learning about Firefox and its hidden areas. I only discovered the extension shortcuts panel a week back and was quite amazed to see that you already have commands for Page Up and Page Down, as well as Line Up and Line Down, which all act on the tab pane. I have tested them, of course, and they work. Perfect!

I can easily convert "Alt Shift PageUp" and "Alt Shift PageDown" into accessible voice-commands such as "Tabs PageUp/PageDown". I am not sure that Home version users will be able to do the same, but you can check this. They may still need to say, "Press Alt Shift Page Up/Down", which is bit of a mouthful, so if you can reduce its verbal complexity to a more concise shortcut, this will help those users greatly.

In Firefox, if you watch how the command "Start Scrolling Up/Down" functions for a webpage (using Dragon's Firefox extension), I think you will probably agree that this is likely a "Move Up/Down # Lines" at-a-time type command set into a loop until the user says, "Stop Scrolling". There is a setting in the options somewhere, where the user can decide how many lines up/down it should occur at its base rate (this may be a mouse device setting). There are also secondary commands (i.e. "Scroll Faster/Slower") that users can say while it is still scrolling which increase or decrease the scrolling speed. At its slowest, you can see it going up/down # lines at a time. At its fastest, it appears to scroll up and down smoothly.

So my guess is that if you can create a script that repeats in a loop until instructed to stop, perhaps aided by a hidden "Stop Scrolling" button (?!), you could possibly emulate this in the tab pane. This probably is not the best method, but it is the only example that Nuance has provided in their own web extension for scrolling webpages!

I still think being able to focus the tab bar and drag it up and down would be better but, although basic testing indicates it is draggable on receiving (hovering) focus, that is well beyond me to explain further!

That said, I think I will find the "Page Up/Down" option useful anyway, and I can modify it further to say "Tabs Page Up/Down # Times" as needed. It is quicker than the current webpage scrolling function, where you must sit and wait for it to eventually get there.

NB Shortcuts are extremely useful in Dragon - if you can set a shortcut for something, it is quite likely users can access it directly by voice or, better, create a more suitable voice-command for it! (The one exception is the FN key, which Dragon users can never access by voice).

Hope this helps, I will get back to you about the rest of it later,

Thank you again!

PS I do not know if you are using the Dragon web extension. This may make a difference to testing performance and most users, if not all, will be using it because there is still little voice-activated webpage functionality within Firefox otherwise (despite initial appearances without it; just try it!). It looks like Firefox is improving over time (the GUI is much improved), but it has a way to go yet.

Objects in the webpage can easily conflict with tab functions in the tab pane when using Dragon and vice versa, depending on what the user is doing and the range of focusable/clickable objects in the webpage and tab panes at the time. This is part of what I was going to send over today, so I will do that shortly.

The link for Dragon’s Firefox web extension is here:
https://dnsriacontent.nuance.com/15/setup/ffinstall.html?lang=enx

PPS Here in the UK, Nuance sells the software under a 30-day trial period (I think both Home and Professional versions). If you decide it is not for you, they will refund you, no questions asked. They were very relaxed about it, as was my personal experience previously, before I finally committed to it after several separate trials. I hope they have a similar sales policy wherever you are, should you need it.

[tigersof]

t depends on the background color of tab-twisty::before. So, you can show the twisty with custom color like as:

tab-twisty::before {
  background-color: rgba(255 /* red: 0-255 */, 0 /* green: 0-255 */, 0 /*blue: 0-255 */, 0.5 /* alpha: 0-1*/) !important;
}

Thank you, that's exactly what I was looking for!

[tigersof]

I just hope the table of contents in the page is synchronized with updated contents, and there are no other special rules about editing of the contents. You'll be able to edit the contents via the "Edit" button shown at the right side of the page title. I'm very sorry but I cannot explain more because I have no experience about how to edit wiki contents via speech recognition system...

I'll figure this out, thank you.

[tigersof]

Just an idea, what about a subpanel extension providing buttons to control TST via its API for voice recognition systems like Dragon? There is a helper extension TST Tab Search providing a search box as a sub panel. Like this extension, a helper extension can provide its custom UI as a sub panel.

Actually, thinking this idea through a bit more, I think accessbility should remain inherent and, ideally, a mandatory requirement within all extensions created. In common with many people, I have numerous extensions, and installing add-on accessibility modules per core extension would likely become problematic.

[tigersof]

I don't know if it may help, but this UK site's product details pages (unlike much of the rest of the same site) is fully accessible to me as a Dragon speech user.

      https://amzn.to/2VWNUye

• I can use commands such as "Button", "List" (and then "Drop List"), "Link" or "Show Links", "Menu", and "Text Field" with the Dragon webpage extension installed to target elements indirectly with "Choose #".
• Most of the page however can be accessed directly by saying the element's label, without the need for the above indirect commands.

My feeling is that what makes this page accessible is what would make TST accessible if implemented similarly. That some webpage links are accessible without the Dragon extension installed (I can navigate links this way but not always), "Toggle", "Close", and "Item" are also already available within TST, and TST is (I believe) treated as a webpage, suggests this to me. Here, I'm assuming it is possible to view the site's underlying code to understand how its accessibility works, so my apologies if that is incorrect.

[tigersof]

Hi Piroor, hope you're well - a quick update...

Currently only:

1. "Toggle" then "Choose #" (this is consistently reliable, unlike the others below but now only actions twisties, not the sound button),
2. "Item #",
3. "Mute Tab" or "Unmute Tab", then "Choose #",

are working reliably (although "Item #" can still fail due to hovering effects).

I've since lost all access to:

• "Close" then "Choose #"

whereby TST panel just closes now, instead of all "Close tab" hover elements being flagged by Dragon.

I don't know what further to add, as I think I said everythning before! I hope it's of some use though....

Thank you

[piroor]

Hmm... I still cannot understand why "Close" is inaccessible. Moreover, very sadly Dragon no longer detect any labels around TST's sidebar on Nightly 93 on my environment even if I use CSS hacks copied from the wiki page. I've gotten stuck completely...

[tigersof]

Oh!

Your latest push 3.8.9 updated last night while I was busy online. Took me a while to work out what suddenly happened in that split second. Nice update.

It's useful in other ways though!

• When everything auto-flipped with the update (so twisties now sit RHS of sidebar pane), I tried the commands to see which might work. "Close" became available again for all tabs (but not as a "Close #" command).
• When I later flipped it back via the options panel so that twisties sit LHS, "Close" would not work again, and instead the entire TST pane closed.
• Flipping back again to RHS, "Close" again becomes available.
• Everything else seems to function/not function as previously described

This is checked in Firefox 90.0.2

[tigersof]

Can also now confirm that in Firefox (developer edition) 92.0b5, none of the TST user commands are working.

I see that Firefox is making changes although, as a non-coder, I don't understand what implications this may have. Is this likely to be the reason that TST is no longer voice accessible? (Even the tab URL titles cannot be accessed by voice)

I had thought that they were moving towards an accessible framework!

[tigersof]

OK, now things are different...

I think there may be an issue with the updating process itself, either Firefox or extensions, or both!

1. With my current setup under Firefox 90.0.2, TST 3.8.11 updated itself. All seemed fine at first, but then it stopped doing anything at all. It failed to load my custom styling, and instead of respecting "Left Side" option for twisty placement relative to other elements, it switched them back to "Right Side". The only function available in the TST sidebar was the "Open a new tab" button. However, any new tab or any other webpage opened was not mirrored in the sidebar, only in the horizontal tab bar where everything worked as normal. It was impossible to use the sidebar to any useful effect.
2. I tried updating Firefox to 91.0.1 to see if that would cure it. It didn't, same thing.
3. I then tried reverting back to older TST version 3.8.8, but that didn't help much either.
4. I then created a new firefox profile under Firefox 91.0.1 and installed TST 3.8.8. Now "Toggle", "Item #", "Mute Tab", and "Close" work, as do speaking the tab titles!
5. I then created another new profile under Firefox 91.0.1 and installed TST 3.8.11. I have the same result as in point 4 above, which on the one hand, is great - on the other, it presents a headache as I still haven't caught up with transferring browsing tabs from previous profiles Firefox forced me to abandon on previous updates and this is starting to look too frequent. (is there an easier way?!)

In truth, I am not really sure what's going on here.

[github-actions]

This issue has been labeled as "stale" due to no response by the reporter within 1 month (and 7 days after last commented by someone). And it will be closed automatically 14 days later if not responded.

[tigersof]

Hi Piroor, I'd like to keep this open, should anyone be able to contribute a possible solution, please? Accessibility is still so important.

[piroor]

Sorry for my silence, I' ll be back at next month...

[tigersof]

thank you

[piroor]

I then created another new profile under Firefox 91.0.1 and installed TST 3.8.11. I have the same result as in point 4 above, which on the one hand, is great - on the other, it presents a headache as I still haven't caught up with transferring browsing tabs from previous profiles Firefox forced me to abandon on previous updates and this is starting to look too frequent. (is there an easier way?!)

Tree structure information stored in Firefox's session data looks transferable with copying sessionstore.jsonlz4 from your ordinal profile. (Please remind that you need to exit both Firefox while copying session data.)

On my case, I usually clear cache data from the profile, when I encounter odd problems. For example:

• xulstore
• shader-cache
• addonStartup.json.lz4
• %LocalAppData%\Mozilla\Firefox\Profiles\(random strin).(profile name)

[tigersof]

Thanks Piroor,

With each FF update, previous profiles other than the current one become out of date. This then requires me to create new profiles for each of them, before copying the contents of their respective old profile into the new replacement profile. As I struggled to keep up, I now have an excess of profiles, and management has become difficult.

Sorry, I don't understand how to action this:

On my case, I usually clear cache data from the profile, when I encounter odd problems. For example:

* `xulstore`
* `shader-cache`
* `addonStartup.json.lz4`
* `%LocalAppData%\Mozilla\Firefox\Profiles\(random strin).(profile name)`

Would you mind explaining in a little more detail, please?

Thank you again

[piroor]

Sorry, I don't understand how to action this:

I just meant that I remove those 4 files/folders to "refresh" my profile when I saw problems. On Windows:

1. Go to about:support.
2. Click the button Open Folder in the "Profile Folder" row.
3. Exit Firefox.
4. Remove two folders xulstore, shader-cache, and one file addonStartup.json.lz4.
5. Click the address bar of the folder window and type %LocalAppData%\Mozilla\Firefox\Profiles to there.
6. Hit the Enter key to open the location.
7. You'll see some profile folders, so remove one of them which has a folder name same to toe folder opened at the step 2.
8. Start Firefox again.

In other words, I ordinarily don't refresh my profile truly, because problems I saw almost gone away with this partial refreshing.

---

If something serious problem happens and it forces me to recreate my profile, I'll use Firefox Sync simply. I'll just create a new profile and log in to Firefox Sync, and wait until all preferences, bookmarks, histories and extensions are restored. And I may combine saving existing tabs to a bookmark folder instead of migrating session data. Steps:

1. Select all tabs via Shift-click or Ctrl-click on the TST sidebar.
2. Right-click on a tab in the TST sidebar.
3. Choose "Tree of tabs" => "Bookmark Selected Trees".
4. Save tabs to a bookmark folder with structure information.
5. Wait until all new saved bookmarks are uploaded via Firefox Sync.
6. Exit Firefox.
7. Create a new Firefox profile.
8. Start Firefox with the new clean profile.
9. Log in to Firefox Sync.
10. Wait until all bookmarks, histories, saved logins, and other data are completely downloaded.
11. Find a synchronized bookmark folder corresponding to the folder you created at the step 4.
12. Right-click on the bookmark folder you found at the step 11, and choose "Open ALl as a Tree".

This method will lose some information but I agree to give up to migrate all of profile data, for safety and stability.

[github-actions]

This issue has been labeled as "stale" due to no response by the reporter within 1 month (and 7 days after last commented by someone). And it will be closed automatically 14 days later if not responded.

[tigersof]

Please keep open!

Thank you for you latest replies. I hadn't spotted these, so busy with other things. I'll follow that up soon.

[github-actions]

This issue has been closed due to no response within 14 days after labeled as "stale", 7 days after last reopened, and 7 days after last commented.

[github-actions]

This issue has been labeled as "stale" due to no response by the reporter within 1 month (and 7 days after last commented by someone). And it will be closed automatically 14 days later if not responded.

[github-actions]

This issue has been closed due to no response within 14 days after labeled as "stale", 7 days after last reopened, and 7 days after last commented.