Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve speech output for UIA notifications, honoring all NotificationProcessing_* constants #16381

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

michaelDCurran
Copy link
Member

@michaelDCurran michaelDCurran commented Apr 11, 2024

this is a prototype to see how NVDA's UIA notification support can be improved, both for the bennofit of UIA notifications in general, but also laying groundwork for supporting the proposed ariaNotify specification allowing web authors to instruct screen readers to speak messages without resorting to live regions: https://github.com/WICG/aom/blob/gh-pages/notification-api.md

Link to issue number:

None

Summary of the issue:

Currently, NVDA's speech output for UIA notifications is very basic in that it just cancels speech and then speaks the display string. It does not honor the importants, or whether only the most recent notification should be spoken etc.
NVDA should appropriately support all NotificationProcessing_* constants:

  • ImportantAll
  • ImportantMostRecent
  • ImportantCurrentThenMostRecent
  • MostRecent
  • CurrentThenMostRecent

Description of user facing changes

NVDA now honors the importants of UI Automation notifications sent by applications.

Description of development approach

ui.message: add optional arguments:

  • messageID: a unique string identify for this message. This could be used to filter or customize the message, and is used with the following arguments to configure how multiple messages with the same message ID interrupt each other.
    • speakOnlyMostRecent: If a message with the same message ID has already been queued, the already queued message/s are dropped and only the most recent one is spoken.
    • interuptCurrentIfSameID: if speakOnlyMostRecentIfSameID is also true, but this argument is false, the first existing message in the queue is not dropped, thus a full message is always spoken. E.g. a progress bar quickly going from 0 to 100 in jumps of 1, might say "0, 25, 55, 100" rather than just stuttering until it gets to 100.

UIA NvDAObject's event_UIA_notify: suitably map the UIA notificationProgress argument to appropriate values for speakOnlyMostRecentIfSameID and interuptCurrentIfSameID. Also map them to an appropriate speech priority. E.g.:

  • NotificationProcessing_Important*: priority=NOW
  • NotificationProcessing_*ThenMostRecent: speakOnlyMostRecentIfSameID=True
  • NotificationProcessing_*CurrentThenMostRecent: interuptcurrentIfSameID=False
  • NotificationProcessing_*: priority=NEXT

Testing strategy:

Private testing done, but will be outlined when browser prototypes become public.

Known issues with pull request:

this is a prototype draft. It is functional, but the code could be better abstracted.

Code Review Checklist:

  • Documentation:
    • Change log entry
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • Testing:
    • Unit tests
    • System (end to end) tests
    • Manual testing
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English
  • API is compatible with existing add-ons.
  • Security precautions taken.

@seanbudd seanbudd added the conceptApproved Similar 'triaged' for issues, PR accepted in theory, implementation needs review. label Apr 16, 2024
@seanbudd seanbudd added this to the 2024.3 milestone Apr 18, 2024
@seanbudd seanbudd removed this from the 2024.3 milestone May 24, 2024
* messageID: a unique string identify for this message. This could be used to filter or customize the message, and is used with the following  arguments to configure how multiple messages with the same message ID interupt each other.
speakOnlyMostRecent: If a message with the same message ID has already been queued, the already queued message/s are dropped and only the most recent one is spoken.
* interuptCurrentIfSameID: if speakOnlyMostRecentIfSameID is true, the first existing message in the queue is not dropped, thus a full message is always psoken. E.g. a progress bar quickly going from 0 to 100 in jumps of 1, might say "0, 25, 55, 100" rather than just stuttering until it gets to 100..

UIA NvDAObject's event_UIA_notify: suitable map the UIA notificationProgress argument to appropriate values for speakOnlyMostRecentIfSameID and interuptCurrentIfSameID. Also map them to an appropriate speech priority. E.g.:
* NotificationProcessing_Important*: priority=NOW
* NotificationProcessing_*ThenMostRecent: speakOnlyMostRecentIfSameID=True
* NotificationProcessing_*CurrentThenMostRecent: interuptcurrentIfSameID=False
* NotificationProcessing_*: priority=NEXT
@AppVeyorBot
Copy link

See test results for failed build of commit cb54f80afb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
conceptApproved Similar 'triaged' for issues, PR accepted in theory, implementation needs review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants