-
-
Notifications
You must be signed in to change notification settings - Fork 97
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve accessibility features for keyboard navigation, UI scaling, TTS, voice recognition, colorblindness and subtitles #983
Comments
Personal thoughts and viewpoints on the matter. This should probably be split into separate proposals as well. Remapping: Focus: Fonts: UIScape: TtS StT: Color blind filters: Subtitles: |
I've worked on input mapping for my project as it is an unusual control scheme that would benefit from users being able to configure input in a way that's comfortable for them. Remapping binds could benefit from a few convenience functions, such as ones to map devices in order of first input (a Microsoft requirement, dunno if it applies to others), a more unified way to detect input axes (usually you have to think of all the different ways this can happen and poll for them manually, see further below), Supporting multiple mice as well as multiple joysticks would also be helpful as some devices map to these. Allow more coder access to Godot's controller layout guesser (if one exists), seems to use a database like SDL. Being able to save and load multiple input map layouts. Layouts could be specified in a list or a dictionary (a collection of layouts), allowing some keys to remain universal (UI) while allowing quick rebinding presets for multiple actions to account for different users sharing a game or different classes of users to select from coder-specified presets. (In my game specifically, this feature could've helped a lot making it easier to account for handedness.) Unifying the reads of joy axis, mouse deltas, and other linear sensors (accelerometers, gyro, and other ones currently exposed to Godot) under some function or in the current Action system (eg:
Easier font scaling can be achieved in theory by setting the dynamic font's resource as local to scene, at the expense of losing some theming ability. Being able to specify default font size separate from instance size would allow bitmap fonts to scale just like a dynamic font, but also step around the "local to scene" issue. Having kerning, word spacing, and tracking offsets available for all font types would go a long way to helping improve readability and make it easier to iterate over variations on a font.
Screen space shaders require backbuffer copies if they're to work with existing games. Multipass shaders currently don't work for canvas items, making it frustrating to implement a simple, custom solution without having to get involved with screen space or multiple viewports. |
This point in particular I don't think it's relevant, especially on the topic of accessibility (is there any person who needs to use two mice?). AFAIK, the API from the OSes will only provide one mouse pointer. You can technically do it by polling the devices directly but I think it's a bit too much for a niche usage, as I don't think it fits the accessibility topic. |
Honestly, I don't know how you do that with only 6 lines of code. You need to parse a SRT file or similar to follow voice-over, you have to trigger the text in audio events (like an explosion), and I might be missing something else. I'm not saying that it's the most difficult thing to do and I'm saying that this (or any other topic for that matter) should be added to the engine. But they should be available in some fashion. This proposal is to make sure that all accessibility topics are either covered by the engine or facilitated in form of a plugin (that is, such plugin should be possible to make). Ideally we should even maintain those officially. |
@vnen: The 6 lines of code comment probably referred to the most naive solution where there's a text file for every audio clip, no SRT, no voice-over, just showing the text (which is fairly easy to do given Godot's Control nodes and signals) EDIT: And mad props for tackling the issue at all, too many games completely forget about people who need subtitles. Being hearing impaired, I tend to prefer old-school games like Baldur's Gate series that just showed all the text on screen, no subtitles needed :P to new-fangled 3D RPGs that don't have subtitles at all! |
Maybe. I built my own device to map to mouse but i don't work in the accessibility field so I can't say how many devices are out there which map to mouse specifically. Usually you can get away with one cursor to do whatever you want if you capture it, so it's probably fine either way. |
May I also suggest W3C's guidleines in the matter of accessibility :
ETSI: |
Hi here, I'm concerned with accessibility and I've some issues too. I speak about myself (with low vision) here, but I can help you to understand some problems if needed. StraToN posts three links of the W3C, and it's the three guidelines for web. I think all the stuff can be adpated for Godot, and there are three standards for three differents axes.
Even if I'm legally blind, I use Godot Engine and its IDE. But, accessibility is not here so I do the maximum with a simple text editor. There are a lot of problems with zoom, I don't have the time to help you, because I have other projects. But I can test when you need, and I'm an user of Godot. I'll keep an eye on this issue, so if you have any question about what is accessibility, I'll try to answer. |
Note that this is mostly about providing accessibility for games. But I do believe that it's a big step into making the Godot editor itself more accessible itself. |
The 2D stretch mode should handle this well enough out of the box. Still, godotengine/godot#21446 will help make this even better by letting the user increase or decrease the UI scale relative to the automatically determined scale. That PR also provides scaling abilities when using the As for custom HUD margins, this is already possible using MarginContainers (once you expose a way for players to change those margins). |
Hey folks, FOr a while now, I've been working on a screen reader written in GDScript and a text-to-speech addon. Known to support Linux, Windows, Android, and the web. May also work on UWP as well, though I'd have to jump through a few more hoops to test that. Could probably be ported to MacOS/iOS easily enough if someone could port tts-rs to those platforms. It took a day or two to knock out a WinRT port, so I don't think it'd be hard. I just don't have the hardware. I'm maybe a month or two out from releasing a game using it, so I'd say it's ready for others to bang on. Note that I myself am blind, so I'm a bit limited on what I can do. In particular, I'm happy to accommodate low-vision users somehow, I just have no way of, well, doing anything meaningful. :) But this at least helps with screen reader and keyboard access. I'd very much appreciate help with this. I'm trying to figure out how to use the editor by writing the screen reader, which works to a point, but I've definitely hit some barriers, and it'd be useful to pop onto a Zoom call and screenshare an editor session with someone who knows Godot well so I can pepper you with questions. :) Also, if someone wants to see an actual engine bug that might be easy to fix and would be a massive QOL improvement for me and others, check out godotengine/godot#36291. Essentially, keyboard focus doesn't seem to work in some editor trees. I have no idea whether this is specific to the editor, happens in all trees, etc. but it means I'm hand-editing TSCN files in cases my plugins could handle, if only focus worked. I'm happy to try fixing it myself if someone with engine experience could help, but I've tried some things on my own and can't seem to make a difference. Also would be nice to see my plugins used in other games, both so they're improved, and so I myself have more games to play. :) If you have a game that might be made screen-reader-accessible, reach out and let's collaborate. |
@ndarilek: You are doing awesome work, man! |
Here's macOS and iOS code (it's almost identical) from my old TTS plugin, it's in Objective-C, and I'm not sure how to interact with Obj-C stuff from rust, but it shouldn't be too hard to port. https://github.com/bruvzg/godot_tts/blob/godot_cpp/src/tts_nsspeech.mm |
Thanks. I'll probably leave the porting effort to someone else for now, though if I do eventually get a Mac then I'll circle back here and check that out. Does that work on iOS too? There's a Unity accessibility plugin that supports iOS, which I definitely want to support and would probably prioritize over macOS at this point. |
AVSpeech is for iOS and macOS 10.14+, NSSpeech for older macOS versions. |
I'm sorry, you *did* say iOS right there in the comment. :/ I blame lack
of coffee. Fixing that now...
|
I found this Godot Speech-To-Text module which may fit this topic and may be useful for you. |
i'm a developer who is working to provide accessibility to the game i'm working on for the blind, so all of this is very welcome. It is sad to see such features get put off. it's harder to add them in later than earlier, but unfortunately people don't think about it intuitively until it starts becoming an issue. i just want to contribute to the record that yes, there are people who need and want these features using Godot. |
I'm still watching this issue, but for me, this is the reason I stopped developing with Godot in favor of Kontra + React. I need to be able to support screen readers and accessibility features more broadly. |
Additionally it will be useful to have something like the accessibility inspector of modern web browsers, either as a core feature or external plugin: https://firefox-source-docs.mozilla.org/devtools-user/accessibility_inspector/index.html |
Describe the project you are working on:
The Godot Engine.
Describe the problem or limitation you are having in your project:
Accessibility features might be required by law in some game projects. It's also nice to allow more people to play the games. Some accessibility features are not obvious or not possible to do only with the base engine and this needs to be improved.
For instance, the CVAA legislation covers anything with communication. So if your game features text or voice chat, you have to comply with the rules (assuming you are selling the game in the US, which most people are). Here's a blog post that covers how it is applicable to games: https://www.gamasutra.com/blogs/IanHamilton/20190123/334910/Demystifying_CVAA.php
Describe the feature / enhancement and how it helps to overcome the problem or limitation:
Improving documentation for existing features and implement missing features needed for providing greater accessibility in game projects. More details on what's needed are presented in the next section.
Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:
There are a few important topics related to improving accessibility in general. I might even be missing something, as I'm not an expert in the area. If that's the case, let me know and I'll edit it in.
Input mapping
Games should provide a way for the user to change the buttons needed to perform actions. Godot's InputMap feature can be used for this, but I'm not sure how difficult it is to make the menu in-game. At least we need a documentation page to describe the process with examples.
Keyboard navigation
Godot can use keyboard to navigate between controls but there are still some issues. For instance, not all controls can receive focus which means they aren't target for keyboard navigation. Also, if you don't set the neighbors properly, navigation can reach a control.
What could be done is improve default control focus and use either the tree or control position to determine neighbors for keyboard navigation.
This is important because some people can't operate a mouse and relies completely on keyboard for getting around.
Touch screen navigation
Mobile OSes provide gestures for blind users to navigate around the UI. Native apps follow this by default without effort when using the native SDKs. But when making a game, devs are using Godot's UI system which don't have the same capabilities AFAIK.
There should be away, preferably baked into the input system, to use the OS gestures to navigate around the game UI.
Font scaling
There must be a way to easily increase the size of all the text displayed. I do believe that if you are properly using themes you can do this by changing font only in a handful of places, but I haven't tested this.
At the very least, we need to improve documentation to show how you can use the available systems to make a global font size slider.
UI scaling
In a similar fashion to the previous point, the whole UI should be scalable. Using the
scale
property of controls is meant mostly for animations, and using that makes it difficult to correctly position everything.This also includes making controls bigger so they can be an easy mouse target and also visible for people with low sight.
We should have a way to easily make everything bigger while maintaining defined positioning margins. Maybe there's no easy way to do this, but we need at least to try and make a documentation page about control scaling with correct positioning.
Here's an example of this applied to a game: https://twitter.com/NateElCabro/status/1235290044866473984
Text-to-speech and speech-to-text
This is one of the most problematic topics. Adding a library to manage this in the base engine might be too much, especially since many games don't really need this.
The better possibility is to allow OS-level applications to properly read the screen. People who need a screen reader already have one installed and configured, so letting them use that is a good thing.
Some systems might not have such OS-level applications, and need a custom solution (e.g. consoles) but in this case I believe it's okay to leave the burden to a third-party library. At least we need to make sure that GDNative is capable of binding such libraries, so plugins can be provided without the need of recompiling the engine.
We probably also need some description field for controls that don't have associated text, which would be used by screen readers. There's a long discussion (initiated by a blind developer) about this here: godotengine/godot#14011
Color blind filters
This is more a "nice-to-have" than anything. The idea is to provide filters that simulate common types of color blindness, so developers can adapt their games to make elements distinct in all cases.
This is covered by #473 but I felt the need to include here. There's also a nice plugin that allows you to this already: https://github.com/paulloz/godot-colorblindness
Maybe we should integrate or endorse such plugin.
Subtitling
This is also a "nice-to-have" which is meant to facilitate the addition of subtitles to games. Ideally anything that is said should also have a subtitle, including audio-description (especially when relevant to gameplay).
This one might be more suitable for a plugin, but I added here for completeness. For reference, see this subtitling plugin for the Unreal Engine: https://www.unrealengine.com/marketplace/en-US/product/yellow-subs-machine
If this enhancement will not be used often, can it be worked around with a few lines of script?:
As it stands now it cannot be easily added with a few scripts (and should be used often).
Is there a reason why this should be core and not an add-on in the asset library?:
Many points here are related on how the engine behave which should be standard, not modified by a plugin. Some of the points might be okay as plugins (we do need to make sure such plugins can be made).
The text was updated successfully, but these errors were encountered: