Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stateful Rules #1953

Open
tbostic32 opened this issue Oct 13, 2022 · 8 comments
Open

Stateful Rules #1953

tbostic32 opened this issue Oct 13, 2022 · 8 comments
Assignees

Comments

@tbostic32
Copy link
Collaborator

Problem: Defining states and transitions between states can be very difficult due to the number of ways in which states can be implemented within web pages.

Aria spec definition of state:

State: A state is a dynamic property expressing characteristics of an object that may change in response to user action or automated processes. States do not affect the essential nature of the object, but represent data associated with the object or user interaction possibilities. See clarification of states versus properties.

In the past, in 2021, we decided that the states we cared about should be tied to CSS psuedo classes since they are an inherently objective measurement of state. https://www.w3.org/TR/selectors-4/#useraction-pseudos. This would include things like:
1. Focus
2. Hover
3. Visited (for links)
4. Active
5. Valid/Invalid (Only with html input specification)
6. Enabled/Disabled
7. Required
8. not Checked/Checked
9. And possibly more

As each of these only denotes a single state, we could also potentially expand the definition to include transitions between states (e.g., between not checked and checked), allowing us to test some user interactions with the page. However, the downside of this approach is that CSS-psuedo classes cannot represent all possible states or state changes of interest that might occur. A quote from our findings in early 2021:

CSS pseudo-classes are well-defined and easy to refer to. Javascript manipulations of the DOM that are not reflected in the CSS pseudo-class selectors are ill defined and extremely hard to specify in the objective manner that is requested for ACT rules.

So, to give examples, with just CSS Psuedo selectors we would struggle to handle the following scenarios:
1. An error message pops up as a result of bad form input (would only trigger :invalid if HTML5 input specs used)
2. A dropdown menu is expanded (initial element would be hovered and eventually active, but its difficult to tie that to the "expansion")
3. An image carousel rotates either due to human action or on a timer

So what can we do:
1. We can stay with CSS psuedo selectors as our measurement of state, understanding its gaps and working around them. Perhaps try to estimate the number of rules that would be difficult to write with this definition.
2. Expand our state definition to include states given by other W3C standards such as aria. For example, aria would (at least) include the states busy (loading) and expanded.
3. Allow subjective applicability that allows us to define new states as needed, likely requiring new glossary entry's.

Pros and Cons of 1&2
Pro - CSS psuedo selectors and other W3C have well-defined, and agreed upon definitions.
Con - ^^ cannot accurately represent all states, some of which may be very important.
Con - Rule understanding may be difficult. "Applicable States" will use CSS psuedo classes when they may not even be present within the rule.

Pros and Cons of 3:
Pro - Allows ability to define all states as needed.
Con - How to ensure accurate understanding by all users of the rules?
Con - Significant deviation from current ACT format which could cause problems.

@tbostic32
Copy link
Collaborator Author

Another suggestion that was brought up was to include states and transitions in the Input Aspects of the rule. In this thread, I would make the proposals:

  1. We included states and transitions as a sub-section of the DOM input aspect
  2. We specifically label the states and transitions, e.g.,
- DOM
    - Focused (state)
    - Collapsed to Expanded (transition)
  1. For states captured by current documents, we can simply cite those (as required by the ACT format). For states not captured, we should add a glossary entry describing the state and giving examples.
  2. I think a slight modification to the Input Aspects for Atomic rules is probably needed to allow the nesting from above ^^. We should also add states to the Common Aspects Note

@tbostic32
Copy link
Collaborator Author

A couple of comments from our meeting:

  • Helen - Should we separate out same page states and page load states?
  • Chris - Partial page load and focus states, how to consider status update states?

@tbostic32
Copy link
Collaborator Author

tbostic32 commented Jan 19, 2023

Link has minimum contrast when in link, visited, focus, and hover states

Applicability

Any element that matches the :any-link css psuedo class (i.e., <a>and <area> elements with an href attribute) with a visible text-node and matching one of the following css psuedo selectors:

  • :link
  • :visited
  • :focus
  • :hover

Expectation

For each test target, the highest possible contrast between the foreground colors and background colors is at least 4.5:1 or 3.0:1 for larger scale text, except if the test target is part of a text node that is purely decorative or does not express anything in human language.

Comments:

  • Putting the states in the title to make the rule more specific feels okay. It still requires putting the states in the applicability or expectation.

Expandable element has valid aria-expanded attribute in expanded and collapsed states

Applicability

Any element where interacting with the element toggles the visibility of child elements.

Note: In the case where an element is not controlling its children, another rule for aria-controls would be needed.

Expectation

When the element is expanded (children are visible) aria-expanded is set to true. When the element is collapsed (children are hidden), aria-expanded is set to false.

Comments:

  • The applicability is quite general here, it likely does not follow the objective requirement, but its implementation is difficult.
  • How do we handle timing issues with checking aria-expanded.

Expandable element is keyboard accessible in expanded and collapsed states

Applicability

Any element where interacting with the element toggles the visibility of child elements.

Note: In the case where an element is not controlling its children, another rule for aria-controls would be needed.

Expectation

The element for toggling between the expanded and collapsed states (altering the visibility of the children elements) can be activated using both the mouse and keyboard (i.e., mouse and keyboard event types)

Assumption

Assumes another method for expanding the element is not available (e.g., an “Expand All” button at the top of the page)

Comments:

  • This rule would seem weird. Like why not just generalize to all instances of mouse control has corresponding keyboard control? I'm not saying this is a good thing, but I don't have a justification for only considering expandable elements.

Status Messages Available to User (Awful Title)

Applicability

Any element whose activation causes a status message to be sent to the user.

Expectation

Status message element has role=status, role=alert, or aria-live that is not set to none.

See Sufficient Techniques for 4.1.3

Comments:

  • This rule is wholly too broad, but its unclear where to slice it. Do we pick certain widgets (e.g., search, form input) or just particular message types?

@tbostic32
Copy link
Collaborator Author

tbostic32 commented Jan 25, 2023

Status Messages

Compared to Link has minimum contrast when in link, visited, focus, and hover states and Expandable element has valid aria-expanded attribute in expanded and collapsed states, status messages has the unique challenge that there is no clear way to define the applicability since any element on the page could potentially cause a status message to appear. Below are several thoughts on how we might begin approaching this.

Applicability - Specify event and element types as much as possible

Any <button>, <input>, <select>, or <textarea> with a mouse event handler for at least one of click, mousedown, or mouseup, and a keyboard event handler for at least one of keyup, keydown, or keypress.

Expectation - Requiring the tester to make an action

The success criterion is automatically satisfied if initiating an event handler on the test target:

  • Does not cause visible text to be added to the page
  • Causes visible text to appear, but the message does not provide information to the user on the success or results of an action, on the waiting state of an application, on the progress of a process, or on the existence of errors.
  • Causes visible text to appear, but a change of context occurs, focusing the newly visible text.

If initiating an event handler on the test target causes visible text to appear, has no change of context, and that text provides information to the user on the success or results of an action, on the waiting state of an application, on the progress of a process, or on the existence of errors, then the element(s) containing the newly visible text must have one of the following:

  • A role attribute with a value of alert
  • An aria-live attribute where the value is either polite or assertive

Discussion Points:
1. The list of applicable elements is quite small and somewhat arbitrary; however, not specifying a set of elements would mean testing every element that matches the event handler criteria. Non-semantic HTML elements (e.g., <div> and <span>) are very difficult to include due to their nature. Handled by 4.1.2 as a pre-check
2. This rule ignores elements where a parent has an event handler responsible for the page change. It is not uncommon for elements to bubble events to a higher level for handling.

  • Option 1: Take only the element with events attached, missing any that would bubble to a higher level
  • Option 2: Grab all elements that have events attached to them or any ancestor. This can cause an explosion of elements to test.
  • Option 3: ????
  1. From discussion points 1 and 2, it is difficult to define the applicability in terms of element and event types. The current example would also be difficult (and misleading) for manual and semi-automated testers. From a practical standpoint, it is actually easier to consider elements perceived as initiating a status message, for example a <div role="button"> styled like a button that says "Validate" or "Search". Humans can do this quite well and modern ML methods would also likely be able to do this (some prototypes exist). We should consider how we might better write rules like this that are more easily implementable to the tools and procedures that are most likely to test for them. In short:
  • Identify elements likely to trigger status updates
  • Searching and triggering may require understanding additional context, such as how to cause a form error or autocomplete box to appear.

@tbostic32 tbostic32 self-assigned this Jan 26, 2023
@tbostic32
Copy link
Collaborator Author

tbostic32 commented Mar 2, 2023

In our last conversation, Wilco mentioned viewing Carlos' rule Error message describes invalid form field value since it has the hint of statefulness to it. Lets first review that rule and see how it was setup.

Applicability - In short, a better version than what I had above. It grabs all HTML elements that meet the following conditions:

Which is basically to say, things that can take input.

Then, for the expectation, Carlos does something clever, and uses the following wording to start each expectation:

Each test target either has no form field error indicators, or at least one of the form field error indicators

With this wording, Carlos essentially isn't requiring that a form error exists (it might not if the form field is filled out correctly), but instead shifts the focus to if there is a form error, it must meet these expectations. This is a clever way to get to the error state without needing to tell the tester to trigger an error specifically (i.e., it implicitly handles the page state).

The question then, is could we generalize this to the status messages problem above? Unfortunately, I'm not so sure. There are a couple key differences that make it difficult.

  1. Status Message Definition: The first difference is that in Carlos' rule, he is able to use inputs for the applicability since each of the form errors must reference the form field that is in error. This acts as the starting point to evaluate form field errors. For Status Messages I think the closest solution would be to create a definition for status messages similar to the definition of Form Field Error Indicator and use that as the applicability.
  2. Change of Context: The second problem is that status messages require that there not be a change in context, which means #1 does not solve our case and puts us back into needing to trigger some state change to see if the context does or does not change when the status message activates.

I'm honestly not sure if we can handle the change in context requirement without referencing state more explicitly than we have done in previous rules. It seems like we need to reference some type of interaction that causes a status message to appear.

@tbostic32
Copy link
Collaborator Author

tbostic32 commented Mar 8, 2023

In our last discussion, we determined the need to have some sort of "Action Taken" heading or sub-heading in order to allow for the change of context requirement in the Status Messages. Below are possible attempts at how we might do this.

The proposal below is to add a new section above the applicability called Preparatory Steps. This section holds one or more steps that lead up to the evaluation of the rule. Each step itself must include an applicability and an action. The applicability should objectively describe the elements that will be subject to specified action.

Preparatory Steps

Step 1

Applicability: HTML element that has one of the following semantic roles
button,
checkbox,
combobox,
listbox,
menuitemcheckbox,
menuitemradio,
radio,
searchbox,
slider,
spinbutton,
switch or
textbox

Note: The list of applicable semantic roles is derived by taking all the ARIA 1.1 roles that:

Action: Any action that initiates a change in the applicable element (e.g., entering text, clicking, mouse drag)

Applicability (copied from Carlos' error rule)

This rule applies to any text that becomes visible after executing the preparatory steps and for which:

  • The text provides information to the user on the success or results of an action, on the waiting state of an application, on the progress of a process, or on the existence of errors.
  • No change of context occurs (i.e., the text is not focused after executing the preparatory steps)

Expectation

The test target (i.e., the element containing the newly visible text) must meet one of the following criteria

  • A role attribute with a value of alert or status
  • An aria-live attribute where the value is either polite or assertive

Questions

  1. The phrase "applies to any text that becomes visible after running the preparatory steps" feels like it is doing a lot of lifting here. What it really means is that some text that was not visible before running the preparatory steps is now visible and should thus be evaluated to see if it is status message text. Perhaps simply more description could help here, but are there better ways to formulate this?
  2. Are there cases where we would want to chain actions? E.g., Have multiple "Applicability After Action" sections in a row somehow? E.g., what about testing for results/messages on a different page, the applicability could be like <input> -> <link/submit to new page> -> <error message>

Error message describes invalid form field value (Rewrite)

Preparatory Steps

Step 1

Applicability: HTML element that has one of the following semantic roles
button,
checkbox,
combobox,
listbox,
menuitemcheckbox,
menuitemradio,
radio,
searchbox,
slider,
spinbutton,
switch or
textbox

Note: The list of applicable semantic roles is derived by taking all the ARIA 1.1 roles that:

Action: Any action that initiates a change in the applicable element (e.g., entering text, clicking, mouse drag)

Applicability (copied from Carlos' error rule)

This rule applies to all form field error indicators on the current page after running the preparatory steps.

Expectation

Each of the conditions below is satisfied by at least one form field indicator.

Questions

  1. form field error indicators is not an objective definition, so we still have the issue of having a non-objective applicability. Perhaps we could narrow the scope of the applicability and then use the form field error indicator definition in the expectation.

@WilcoFiers
Copy link
Member

WilcoFiers commented Mar 27, 2023

Some alternatives, trying to write this without new sections:

Approach using "where-after" language

Applicability

For each [HTML element] containing [visible text], where this text appeared after the [activation] of an [HTML element] with one of the following rules:

  • button
  • checkbox
  • ...

Exception:

  • The text does not provide information to the user on the success or results of an action, on the waiting state of an application, on the progress of a process, or on the existence of errors.
  • A change of context occurs (i.e., the text is focused after the activation)

Expectation

The test target (i.e., the element containing the newly visible text) must meet one of the following criteria

  • A role attribute with a value of alert, log, status, or progressbar
  • An aria-live attribute where the value is either polite or assertive

"Where-after" language for modals

One of the more "procedural" tests I know is on returning focus to the modal trigger, after the modal is closed. We could write it like this:

Applicability

For each [HTML element] with an [aria-modal] attribute, where a [semantic] modal becomes [visible] after this [HTML element] is [activated].

Expectation

Each test target receives [focus] after the modal is closed, except if the test target is not [included in the accessibility tree]

@tbostic32
Copy link
Collaborator Author

tbostic32 commented Apr 4, 2023

Here are a couple examples of different test cases that we can use for discussion:

Successful Form (Manual)

<html lang="en">
  <head></head>
  <body>
    <form id="example_form">
      <label for="username">Username</label>
      <input type="text" id="username"/>
      <input type="submit" />
    </form>
    <p role="status" id="submit_message"></p>
  </body>
  <script>
    let form = document.querySelector('#example_form');
    let alert = document.querySelector('#submit_message');
    let username = document.querySelector('#username');
    form.addEventListener('submit', function(event) {
      event.preventDefault();
      if(username.value.length > 0) {
        alert.textContent = "Success";
      }
    })
  </script>
</html>

Successful Form (Auto)

<html lang="en">
  <head></head>
  <body>
    <form id="example_form">
      <label for="username">Username</label>
      <input type="text" id="username"/>
      <input type="submit" id="submit"/>
    </form>
    <p role="status" id="submit_message"></p>
    <script>
      // Handler for sending success notification
      let form = document.querySelector('#example_form');
      let username = document.querySelector('#username');
      let submit = document.querySelector('#submit');
      let alert = document.querySelector('#submit_message');
  
      form.addEventListener('submit', function(event) {
        event.preventDefault();
        if(username.value.length > 0) {
          alert.textContent = "Success";
        }
      });
  
      // Automatically fill out form
      setTimeout(() => {username.value = 'W3C_User';}, 1000);
      setTimeout(() => {submit.click();}, 2000);
    </script>
  </body>
</html>

Log Message Updates

<html lang="en">
  <head></head>
  <body>
    <div id="weatherlog" role="log">
      <h4 id="logHeading">Weather Alerts</h4>
      <ul id="log-entries">
      </ul>
    </div>
    <script>
      // Simulate weather updates
      let weatherTypes = ['Sunny', 'Cloudy', 'Rainy'];
      let logIndex = 0;
      let logEntries = document.querySelector('#log-entries');
      setInterval(() => {
        let li = document.createElement('LI');
        let timeOfEntry = `${((2 * logIndex) % 24).toString().padStart(2, '0')}:00`
        li.textContent = `At ${timeOfEntry} it is ${weatherTypes[logIndex % 3]}`;
        logEntries.prepend(li);
        logIndex += 1;
      }, 3000);
    </script>
  </body>
</html>

A few questions that came up while making these examples:

  1. Is using timed steps for "automation" a reasonable thing to do? Are there technical problems to consider like time for a tool to startup? For example, what if we start changing the page before the testing tool has loaded? What other alternatives are there?
  2. Are there hooks that a tool might use that we should be aware of? For example, if a tool waits for the pages DOM to be unchanged for 3 seconds before running, it will never run (at least not until a timeout) for the weather log example above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants