Inert subtrees mapping

[asurkov]

Need to spec out how inert subtrees are exposed to AT. Dialog-backed inert content perhaps makes no interest for screen readers and may be safely cut off, however screen magnifiers might need it. HTML:inert, which is re-added in [1], seems adds a bunch of new use cases which vary from offscreen accessible elements to image carousel controls, thus I'm not quite sure what mapping could make the best experience for AT.

Note, inert subtrees spec [2] allows user agents to override the restrictions on search and text selection in inert subtrees, which may speak out against cut-off approach.

Perhaps, the safest thing to do is to expose an accessible state or object attribute in case of ATK/IA2 on inert subtrees to let AT to decide how to deal with inert content.

Thoughts?

cc'ing @jcsteh, @MarcoZehe, @alice, @cookiecrook

[1] https://github.com/alice/html/tree/re-add-inert
[2] https://html.spec.whatwg.org/multipage/interaction.html#inert-subtrees

[aleventhal]

@cookiecrook is the only way to deal with inert content on Mac to prune it from the tree?

[alice]

Could we expose it the same way we do aria-hidden? i.e. expose it but mark it as hidden, and prune from the tree for APIs which don't have that concept.

Also, we could support aria-hidden=false on inert roots, if authors really wanted to override that behaviour.

[jcsteh]

Could we expose it the same way we do aria-hidden? i.e. expose it but mark it as hidden

That's not what the spec says, nor what any current browser does, at least on Windows (including Chrome from a brief test I just did, though I believe Chrome makes an exception for focused/focusable content). aria-hidden prunes the tree. There was an attempt to change this, but it's getting a fair amount of pushback (including from me) due to backwards compatibility problems, etc.

That said, I tend to think we should do this (expose but mark) for inert. While I don't support backwards incompatible changes for aria-hidden, I do think we should avoid making the same mistakes in the first place with inert.

[asurkov]

That said, I tend to think we should do this (expose but mark) for inert. While I don't support backwards incompatible changes for aria-hidden, I do think we should avoid making the same mistakes in the first place with inert.

Supposedly Accessibility APIs would need a new state for inert state? I assume no existing mapping makes a good match?

[alice]

Good question. What would we hope AT users would be able to do with inert nodes?

[jcsteh]

In Chrome's intent to ship thread, @alice wrote:

not exposing inert nodes to AT APIs will almost always be the best thing to do,

Looking at the inert explainer, I'm no longer convinced this is true. The problem is that some of the use cases for inert have very different semantics, even though we want the same focus/pointer behaviour. We're trying to have inert expose semantics to a11y (hidden, not hidden, specific state), but I'm not sure we can do that when the semantics for various use cases are so different.

For "Temporarily offscreen/hidden content" use cases, I think it makes sense to hide the content from a11y. But for "On-screen but non-interactive content" use cases, it's much less clear, and in fact hiding the content might be problematic. Let's look at some of the examples in the explainer:

• Any of the use cases for blockingElement[s]: For screen readers, it makes sense to hide the inert content. But magnifiers might need this; it's still visible (even if that visibility is effectively decorative), so it's reasonable that a magnifier user might try to see it. I think we need to reach out to magnifier vendors to see what their thoughts are on this.
• A slide show or "cover flow" style carousel may have non-active items partially visible, as a preview: Similar to blocking elements.
• Form content which is not currently relevant: Even for screen reader users, "disabled" semantics feel more appropriate here than "hidden" semantics.
• Disabling the entire UI while in an inconsistent state: Similar to form content which is not currently relevant. Even for a screen reader user, it feels weird that the entire reviewable UI would suddenly completely disappear and be replaced by a throbber or similar.

If we specify that inert hides content for a11y by default, we're effectively saying that "Temporarily offscreen/hidden content" is the main use case. Worse, it's not clear to authors that other use cases require them to do extra a11y specific work (e.g. aria-hidden="false", even assuming that were supported).

On the other hand, just exposing a state effectively passes the responsibility for poor semantic differentiation to ATs. Authors use inert in whatever way they wish, but an AT has no way of knowing which use case (and thus what semantics) were intended. ATs will probably implement heuristics and those heuristics will probably be wrong at least some of the time.

Unless we're going to split these different use cases/semantics into separate markup (attribute, attribute value, whatever), I wonder whether it might be better to have inert do nothing as far as a11y is concerned. That way, we can be clear that it's the author's responsibility to determine the semantics. For example: if it's temporarily offscreen/hidden content, use aria-hidden; if it's form content which is not currently relevant, set aria-disabled on the form fields.

In summary, I understand (and agree) that ideally, we want to expose default semantics wherever possible so more content is accessible without authors having to do accessibility specific work. In this case, though, I don't think we can infer enough information about semantics from "inertness" to expose a widely applicable default.

What's particularly concerning here is that inert has been widely discussed as something to solve a11y problems. Not exposing any semantics at all is very contrary to that. Even so, I'm pretty worried about the harm that could be caused here given the widely differing use cases.

[alice]

If we specify that inert hides content for a11y by default, we're effectively saying that "Temporarily offscreen/hidden content" is the main use case. Worse, it's not clear to authors that other use cases require them to do extra a11y specific work (e.g. aria-hidden="false", even assuming that were supported).

Conversely, though, if we don't hide nodes from a11y by default, the same logic applies when that is the desired outcome.

If we assume authors are more often than not going to neglect to consider how things should be exposed to AT, is it a better default to expose everything (including content which shouldn't be exposed), or expose nothing (including content which isn't interactive but might be visibly available)?

If it's a matter of whether aria-hidden=false is supported, we can ensure that it is.

[jcsteh]

If we specify that inert hides content for a11y by default, we're effectively saying that "Temporarily offscreen/hidden content" is the main use case. Worse, it's not clear to authors that other use cases require them to do extra a11y specific work (e.g. aria-hidden="false", even assuming that were supported).

Conversely, though, if we don't hide nodes from a11y by default, the same logic applies when that is the desired outcome.

The effect is the same, but the logic is different. In not doing anything, we're saying "inert does not imply semantics. You must explicitly define the semantics yourself." (That would need to be specified.) In hiding the content, we're implying that a specific set of use cases are the "intended" (or at least primary) use cases. Either way, the spec needs to define the semantics (or lack thereof), and something? needs to give authors guidance on common patterns and how to get the appropriate semantics.

If we assume authors are more often than not going to neglect to consider how things should be exposed to AT, is it a better default to expose everything (including content which shouldn't be exposed), or expose nothing (including content which isn't interactive but might be visibly available)?

I'd argue exposing the content is less "dangerous", only because the default for content in general (in the absence of display: none, visibility: hidden, aria-hidden, etc.) is to expose it.

If it's a matter of whether aria-hidden=false is supported, we can ensure that it is.

I don't think it's just that. I'm really concerned by the overloading of use cases here combined with the bias of semantics towards one specific set of use cases. If inert is about conveying a specific set of semantics, we should specify those semantics in absolute terms. If it's about enabling a particular "behaviour" but does not convey specific semantics, we shouldn't imply semantics at all. Ultimately, I'm not clear on what the intent is here.

[alice]

I'd argue exposing the content is less "dangerous", only because the default for content in general (in the absence of display: none, visibility: hidden, aria-hidden, etc.) is to expose it.

This is a very reasonable point.

I guess my sense is that the vast majority of use cases will tend towards cases where content is visually obscured in some way, but not in ways that impact the mapping to AT (which means the disabled form example was an especially poor choice).

My concern is that the API has less value if we're still placing the onus on authors to consider AT API mapping in the vast majority of cases - meaning that the expectation is that the mapping will be wrong in the vast majority of cases (because authors tend not to attempt ARIA to begin with, and when they do it tends to be error-prone as well).

However, I take your point that erring on the side of exposing content unnecessarily is safer than failing to expose content where it should be exposed.

Do you think there is a way that we could advise authors to use the API such that the risk of harm is minimised, if we do ship it more or less as currently implemented?

[jcsteh]

I guess my sense is that the vast majority of use cases will tend towards cases where content is visually obscured in some way, but not in ways that impact the mapping to AT

Note that even this may be problematic for magnifiers, though that's a separate issue from the semantics. Aside from my own guesswork here, Mozilla had a complaint from Dolphin when we implemented aria-hidden as specified (pruning the tree) because it impacted magnification. I feel that magnifiers weren't considered sufficiently when aria-hidden happened and I worry about making the same mistake here.

My concern is that the API has less value if we're still placing the onus on authors to consider AT API mapping in the vast majority of cases - meaning that the expectation is that the mapping will be wrong in the vast majority of cases (because authors tend not to attempt ARIA to begin with, and when they do it tends to be error-prone as well).

That's very fair... and I agree that is not an ideal situation.

Do you think there is a way that we could advise authors to use the API such that the risk of harm is minimised, if we do ship it more or less as currently implemented?

Perhaps we could specify (in the spec) that this is primarily intended for temporarily hidden/off-screen content, that this will hide the content from all assistive technology products and that any other uses of inert will thus require the author to alter the semantics via other means. We could include "visually obscured" in that list, but I worry about the magnifier case as explained above; I think we need input from someone with expertise in that area to make a good/responsible judgement on that.

[alice]

I feel that magnifiers weren't considered sufficiently when aria-hidden happened and I worry about making the same mistake here.

Yeah, good point.

I think we need input from someone with expertise in that area to make a good/responsible judgement on that.

That would be good. I'll try and ask around my network; do you have any suggestions on who might have the relevant expertise here?

[jcsteh]

You could try Mike Hill from Dolphin; he was the person who raised concerns about Firefox's change to prune aria-hidden from the tree as per the spec. I do have one engineering contact working on ZoomText, but I'm not sure if they would be the right person or not.

[alice]

I'm asking around, but of course it's always tricky across the international date line!

Meanwhile, I just remembered that the spec does have some advice on this:

By default, there is no persistent visual indication of a subtree being inert. Authors are encouraged to clearly mark what parts of their document are active and which are inert, to avoid user confusion. In particular, it is worth remembering that not all users can see all parts of a page at once; for example, users of screen readers, users on small devices or with magnifiers, and even users just using particularly small windows might not be able to see the active part of a page and might get frustrated if inert sections are not obviously inert. For individual controls, the disabled attribute is probably more appropriate.

Perhaps we could make that advice stronger?

[jcsteh]

By default, there is no persistent visual indication of a subtree being inert. Authors are encouraged to clearly mark what parts of their document are active and which are inert, to avoid user confusion.

At risk of belabouring my earlier point, this text itself makes me consider again whether it's more appropriate for the author to explicitly specify the semantics given that they have to explicitly specify the visual presentation. It could be confusing to say "authors must explicitly specify the visual presentation" but also "the default semantics are hidden, but you should override this if...". But it's hard to ignore the fact that, as you say, authors probably won't do it or will do it wrong. 😩

[alice]

100% agree that as written, the advice does leave it open to interpretation that any "visual indication of a subtree being inert" is appropriate, including one that leave the information fully visible. It was clearly geared towards avoiding a situation where content appears interactive but, confusingly, isn't.

I'm wondering if we could rework that advice to instead make the intention clear that content which is inert should also be visually obscured in some way, and that it generally won't be perceivable to all users.

Obviously there's no way to prevent authors "misusing" it, but making it clear that using inert for content which is intended to remain perceivable is misuse might allow, for example, accessibility testing libraries to flag misuse.

[alice]

@jcsteh How's this for a first attempt at a replacement for the above (non-normative) note?

An inert subtree shouldn't contain any content which is critical to understanding the page while it is inert. Content in an inert subtree might not be perceivable by all users, and won't be interactive, so content shouldn't be inert unless it is also visually obscured in some way. For individual controls, the disabled attribute is probably more appropriate. As an example of visually obscuring inert content, content which is made inert by a modal dialog element will typically be either completely obscured by the dialog, or partially obscured by the dialog's backdrop.

Might need some thought on whether it should either be normative, or have the normative sounding "shouldn't"s removed, but do you think this roughly captures what we've been talking about?

[jcsteh]

@jcsteh How's this for a first attempt at a replacement for the above (non-normative) note?

Thanks. I think it's a great start, though I'd flag that this doesn't address the magnifier case, which still worries me greatly.

Minor wordsmithing:

An inert subtree shouldn't contain any content which is critical to understanding the page while it is inert.

Can we drop "while it is inert" given the mention of "inert subtree" earlier? That is, it isn't inert while it isn't in an inert subtree.

Content in an inert subtree might not be perceivable by all users,

I'd be inclined to say "will not" rather than "might not" here, since we know it won't be perceivable to screen reader users at least.

Might need some thought on whether it should either be normative, or have the normative sounding "shouldn't"s removed,

I'd argue it should be normative because this will be the mapping for APIs. This is not a "maybe"; it's a "definite" (as per spec) that some users absolutely will not perceive the content.

Also, while I realise explainers are not specs, I'd suggest considering removal of disabled use cases from the explainer altogether, at least until we figure out whether we're going to allow aria-hidden="false" or the like (which raises a whole other can of worms), and even then, it would ideally be noted that the author would be responsible for overriding the default semantics in this case. I worry that explainers will at least to some extent be used as a reference, particularly for use cases, which could lead to problematic usage as currently written.

[nt1m]

I think the main use case for the inert attribute really is temporary/partially offscreen content. The other use cases commonly mentioned: overlays/focus trapping, are ones where <dialog> is more appropriate (and potentially easier to use).

I guess it might be a problem that they all share the underlying primitive (inert subtrees), but it shouldn't prevent implementations from treating <dialog> and the inert attribute differently in terms of assistive technologies.

[scottaohara]

Have a much longer response that I’ve been working on today for this, but I’m not thinking there should be a difference unless it was a user opt in.

More on this later, but just quickly responding since I have been talking with others on this subject and wanted to meet with Steve before I posted here and in the html thread.

[scottaohara]

from my POV, this needs to be specified to indicate that the elements of the inert subtree need to be treated as hidden. Additionally, focusable elements within the subtree would need to be reported as, not focusable - e.g., UIA isKeyboardFocusable=False.

I do not think form controls should be mapped as disabled. From an a11y standpoint they shouldn't be accessible anyway, so no need to communicate this state. And two, there are very real situations where form controls within an inert subtree would still need their values included in a form submission. So disabling these doesn't seem appropriate.