Accessibility #121
Comments
|
Excellent questions. Accessibility functionality is very important, and we will most certainly work towards support in dvui. To be clear, because dvui is a small, new, open-source project, the features and functionality that are added come from people showing up, asking for them, and being willing to help. Here's what I currently understand about accessibility-related functionality:
Most of these are going to require separate work per backend, and I expect support will vary by backend. My plan here (like most things), is to wait for someone to show up who understands one of these well and can help us figure out the requirements and help test solutions. To anyone reading this - if I forgot to list something please reply so I can add it. |
|
As a concrete example, making a high contrast theme is something we could implement quickly. But how would we know if we did the right thing? So we need someone who knows what a high contrast theme (themes?) should be, and can help us test and tweak it. We don't need perfect requirements, but personally I don't know enough to do this satisfactorily. |
Initially, we could work on copying an existing high contrast color theme, like
Yes, this is one of the main things I was thinking of. It will probably take some work figuring out how to integrate this properly with operating systems. I am the most familiar with this type of accessibility because one of my younger brothers is dyslexic and screen readers have been a big help to him. We could also look into adding fonts like OpenDyslexic. Actually this should be fairly quick to add, I'll try to work on that once my current pr is done. |
|
Great! Copying an existing theme sounds good (especially because you are now the theme expert). For text-to-speech it's a relief to have someone familiar with it. Can you open an issue dedicated to that and put whatever you currently understand about it there? Thanks! |
|
As a side note. I really miss Opera version 9 which implemented VXML. I wrote my cms web site for it. Each web page was a 3D colletion of information. |
|
OS Specific TTS integration
|
|
I'm watching https://www.youtube.com/watch?v=Jf0cjocP8Wk and just found out about https://webaim.org/resources/contrastchecker/ which might help with deciding how much contrast is enough for normal and high contrast themes. |
About IME -- fcitx (ibus compat)
Our SDL works (missing system font): Peek.2024-09-19.21-01.mp4Our raylib doesn't. I tested it. IME input events are treated as normal key input events. glfw is stuck in time. raysan5/raylib#1945 |
|
Talking about fonts, we need a set of "system UI fonts" to fallback to. Firefox has their own list of fonts. On Linux, |
|
Thanks very much for trying this! I can see from your video that the IME temporary window is nowhere near the dvui textEntry. I've filed #132 for this. I'm less sure how to deal with fonts, but I think you are right that we'll have to connect to some system fonts somehow, while keeping the ability to have only compiled-in fonts present. I've filed #133 |
|
There's also AccessKit with generated C bindings, although it's written in Rust and sadly I don't know how to integrate cbindgen in build.zig. |
|
That looks promising! |
VoiceXML was literally better. This need us to build a document tree too. |
|
I think we will just use AccessKit. I looked into IME support as well. dbus -> AT-SPI dbus is at every point. The SDL implementation is here: https://github.com/libsdl-org/SDL/tree/main/src/core/linux Maybe we can copy SDL's code into dvui. About dbus libraries, there are currently two.
The complexity of DBus is really ..., and we are stuck with it. See AccessKit and SDL deals with DBus for us. Further readings:
|
|
Hi, AccessKit dev here. Happy to see this project considering accessibility seriously. Since dvui appears to mostly use SDL as its windowing layer, here is an example app showing how to integrate AccessKit C bindings with SDL. I am not aware of any project written in Zig currently integrating AccessKit and I don't know if our C bindings would be enough for you, but I can help if dedicated Zig bindings are needed. You might also want to look at egui, an immediate mode UI toolkit which uses AccessKit. |
|
Thank you very much! That's very helpful, I'll take a look at those. I'm pretty new to this stuff. Can you recommend what a first-step integration goal would be? Would it be to read a description of a single UI element? |
|
@david-vanderson If you want to be able to inspect the accessibility tree, you will first want to hook AccessKit into your windowing provider. The example program I linked above should contain all you need for this. After that, you will need to come up with a way to identify your UI elements: these IDs must be stable across frames. You can then try building accessibility nodes with We are still figuring out how to best document AccessKit. The C header file doesn't contain much so the Rust documentation will be more helpful. Most of the node's properties come from the ARIA web standard so you can also use this specification as a reference. |
|
egui code for filling accesskit node info from widget info: https://github.com/emilk/egui/blob/7bd6f83f18af7317f75e7a32413a9a9eb747db7f/crates/egui/src/response.rs#L1010 @DataTriny What's the minimal code in C that constructs the following tree and send it to accesskit?
What's the code if it changes to
How do I submit the changes? Do I need to calculate the delta or does accesskit do it and announce it to the screen reader? It would be nice if you can provide us with this code in C or Zig so we can use it as a starting point.
Yes, it's mostly the Zig-specific pointers that provide ergonomic. We will likely have our own Zig Question about the build system.accesskit-c need CMake and Cargo to build. Is it better for accesskit to ship the prebuilt binaries or how are we going to build it? |
Tree updatesThe very first tree update is a bit special as you must provide information about the root window and possibly the name of your application. It is not shown here, please consult the SDL example program linked above to see how it is done. Second tree update (could be part of the first tree update if the whole UI is available as soon as the window is shown): // Giving the container an ID of 1, assuming 0 refers to the root window.
accesskit_node_id CONTAINER_ID = 1;
accesskit_node_id BUTTON_B_ID = 2;
accesskit_node_id LABEL_C_ID = 3;
accesskit_node_builder *container_builder = accesskit_node_builder_new(ACCESSKIT_ROLE_GENERIC_CONTAINER);
accesskit_node_builder_set_bounds(container_builder, ...);
accesskit_node_builder_push_child(container_builder, BUTTON_B_ID);
accesskit_node_builder_push_child(container_builder, LABEL_C_ID);
accesskit_node_builder *button_builder = accesskit_node_builder_new(ACCESSKIT_ROLE_BUTTON);
accesskit_node_builder_set_bounds(button_builder, ...);
accesskit_node_builder_set_name(button_builder, "B");
accesskit_node_builder *label_builder = accesskit_node_builder_new(ACCESSKIT_ROLE_LABEL);
accesskit_node_builder_set_bounds(label_builder, ...);
accesskit_node_builder_set_name(label_builder, "C");
// The first update would need to include information about the root window, it is not shown here.
// We always have to give information about the focus. Here we put it on the button, but it could be set to the root window if we don't have something better.
accesskit_tree_update *second_update = accesskit_tree_update_with_capacity_and_focus(3, BUTTON_B_ID);
accesskit_tree_update_push_node(second_update, CONTAINER_ID, accesskit_node_builder_build(container_builder));
accesskit_tree_update_push_node(second_update, CONTAINER_ID, accesskit_node_builder_build(button_builder));
accesskit_tree_update_push_node(second_update, CONTAINER_ID, accesskit_node_builder_build(label_builder));
// second_update is now complete and can be pushed to a platform adapter.Third tree update, if you can't persist any state: accesskit_node_id LABEL_D_ID = 4;
accesskit_node_builder *container_builder = accesskit_node_builder_new(ACCESSKIT_ROLE_GENERIC_CONTAINER);
accesskit_node_builder_set_bounds(container_builder, ...);
// The order in which children IDs are pushed is the order in which they will be exposed in the tree.
accesskit_node_builder_push_child(container_builder, BUTTON_B_ID);
accesskit_node_builder_push_child(container_builder, LABEL_D_ID);
accesskit_node_builder_push_child(container_builder, LABEL_C_ID);
accesskit_node_builder *button_builder = accesskit_node_builder_new(ACCESSKIT_ROLE_BUTTON);
accesskit_node_builder_set_bounds(button_builder, ...);
accesskit_node_builder_set_name(button_builder, "B");
accesskit_node_builder *label_d_builder = accesskit_node_builder_new(ACCESSKIT_ROLE_LABEL);
accesskit_node_builder_set_bounds(label_d_builder, ...);
accesskit_node_builder_set_name(label_d_builder, "D");
accesskit_node_builder *label_c_builder = accesskit_node_builder_new(ACCESSKIT_ROLE_LABEL);
accesskit_node_builder_set_bounds(label_c_builder, ...);
accesskit_node_builder_set_name(label_c_builder, "C");
accesskit_tree_update *third_update = accesskit_tree_update_with_capacity_and_focus(4, BUTTON_B_ID);
accesskit_tree_update_push_node(third_update, CONTAINER_ID, accesskit_node_builder_build(container_builder));
accesskit_tree_update_push_node(third_update, CONTAINER_ID, accesskit_node_builder_build(button_builder));
accesskit_tree_update_push_node(third_update, CONTAINER_ID, accesskit_node_builder_build(label_d_builder));
accesskit_tree_update_push_node(third_update, CONTAINER_ID, accesskit_node_builder_build(label_c_builder));
// third_update is now complete and can be pushed to a platform adapter.AccessKit will compute the diff between the two updates, only add or remove nodes as necessary and emit the appropriate events. Alternative tree update if you can retain some state, this is obviously more efficient but not practical for immediate mode UI toolkits: accesskit_node_id LABEL_D_ID = 4;
// Since container C's children have changed, we must re-create it.
accesskit_node_builder *container_builder = accesskit_node_builder_new(ACCESSKIT_ROLE_GENERIC_CONTAINER);
accesskit_node_builder_set_bounds(container_builder, ...);
// The order in which children IDs are pushed is the order in which they will be exposed in the tree.
accesskit_node_builder_push_child(container_builder, BUTTON_B_ID);
accesskit_node_builder_push_child(container_builder, LABEL_D_ID);
accesskit_node_builder_push_child(container_builder, LABEL_C_ID);
accesskit_node_builder *label_d_builder = accesskit_node_builder_new(ACCESSKIT_ROLE_LABEL);
accesskit_node_builder_set_bounds(label_d_builder, ...);
accesskit_node_builder_set_name(label_d_builder, "D");
// We can still put the focus to button B even if it is not in the update, AccessKit still remembers it.
accesskit_tree_update *third_update = accesskit_tree_update_with_capacity_and_focus(2, BUTTON_B_ID);
accesskit_tree_update_push_node(third_update, CONTAINER_ID, accesskit_node_builder_build(container_builder));
// Button B and label C haven't changed, we don't need to include them in the update.
accesskit_tree_update_push_node(third_update, CONTAINER_ID, accesskit_node_builder_build(label_d_builder));
// third_update is now complete and can be pushed to a platform adapter.Build system integrationI currently know nothing about typical Zig build systems but we offer some pre-built binaries here. They are generated in our CI so you can count on them being available. We recently extracted the bindings into their own repository, which explains why there are so few releases in there. We also plan to add support for Meson, I don't know if that would be of any help. I can also point you to the work-in-progress integration of AccessKit into the Godot game engine. They are currently using our pre-built binaries as well. |
|
Thank you for the detailed explanation and code samples! |
|
The SDL example contains platform specific code. https://github.com/AccessKit/accesskit-c/blob/48061861ebe14873a29b65488728ce2a1160f72b/examples/sdl/hello_world.c#L72-L92 |
|
There is just a bit more platform-specific code @iacore, but it is nothing compared to what you would have to write yourself if you decided to implement accessibility for each platform from the ground up, the stacks are quite different. We do our best to keep the core schema platform neutral; the way you build AccessKit nodes and tree updates is completely decoupled from the platform. |
|
Could this be a library, or is it like platform glue code that each project have a copy of and customize it? I'm not sure how to treat it. I think maybe accesskit-c could use a |
|
AccessKit is an entire library, it's just that we don't provide integrations for popular windowing providers such as SDL, SFML or GLFW. We officially support the most popular Rust windowing provider though, which you can find in the platforms/winit directory of the main repository. If people show interest in using AccessKit with windowing providers written in other languages than Rust, maybe we will spend time building such components. Help from the dvui team would be welcome! |
Pretty much a hard requirement. There are no pure right-to-left writing systems. All real-world usage is bidirectional (BiDi). In Arabic and Hebrew, numbers are most commonly "western" digits (Arabic-Indic, technically) and written left-to-right embedded in the text. Same with foreign words used in a sentence, which are written in their native directionality. Of course, BiDi and IME are Internationalization, and not Accessibility, the subject of this issue. Really everything under "Language Support" is mostly orthogonal to accessibility. |
|
I agree that this issue's scope is probably too big. It should be turned into a tracking issue instead. If the project's maintainers agree that AccessKit is the answer to the accessibility tree part, I can open a dedicated issue for that where this topic can be discussed. |
Yes please - that would be wonderful. Could you put in the description what pieces of accessibility integrating with AccessKit will help with? Is it more than screen readers? Thanks! |
|
@david-vanderson I've filed #151. I haven't expanded on actionable steps yet, but I have described the motivation. I've elaborated a bit on what assistive technology means. If you didn't know, the automation testing aspect might interest you as implementing AccessKit would mean that dvui users could rely on existing testing software for the QA of their apps. |
|
Thank you! |
It might be worthwhile to consider whether or not we want to support accessibility functionality. Accessibility usually only gets considered very late into open source development as an afterthought. We should consider
The text was updated successfully, but these errors were encountered: