[Tracking]: Unified a11y tests

[valentinpalkovic]

🧑‍🤝‍🧑 Who: @kasperpeulen and @valentinpalkovic

This is a tracking issue for the Unified a11y tests project. Its purpose is to keep track of the overall status of the project and tasks and plan everything around it.

⚠️ Problem

Accessibility (a11y) is increasingly required in modern UIs. This is already enforced for many government services around the world, and is also becoming mandated for private services as well, e.g. European accessibility act which goes into law in June 2025.

Today, Storybook’s accessibility addon addon-a11y is one of the most popular addons not bundled with Storybook at 1.6M weekly npm dl’s. It provides a way to interactively test and debug a11y violations in stories

The biggest a11y product gap today is that you have to manually click on each story to see the a11y violations for that specific story. There is no way to do this across all stories in batch.

If we can solve this problem in the context of our Unified UI Testing project, it will go a long way towards improving support for A11y workflows in Storybook AND make our unified testing UI that much more compelling.

🏁 Goals

1. As a Storybook user, I want to be able to run a11y tests across all the stories in my Storybook, so that I can see the full a11y status of my project.

🚩 Milestones

Finalize API's - Deadline: 13th Nov

Beta Give feedback

1. Finalize Reporting API specification - Event name and payload conventions | capabilities
2. Finalize afterEach specification
3. Investigate discrepancy cases and how to deal with them

Technical Preparations - Deadline: 22th Nov

Beta Give feedback

1. Implement afterEach Hook
2. Implement Reporting API

A11y Implementations - Deadline: 6th Dec

Beta Give feedback

1. Register a11y Sub Test Provider (Batch run and single story run)
2. Change a11y reporting mechanism - Use Reporting API
3. Implement Vitest report proxy of a11y test results
4. Implement parameter-based options to control a11y behaviour in UI and Vitest
5. Handle discrepancy between Vitest run and Browser run

After QA

Beta Give feedback

1. Blog post. “Component testing for Accessibility” highlighting the new test behavior, filtering, and debugging workflow.

Nice to have

Beta Give feedback

1. A11y addon improvements. Rework addon experience per Accessibility addon V2 design
2. Investigate: Optimize accessibility testing to manage inherited violations in Storybook components
3. fix(addon-a11y): Addon-a11y displays visual bug when running the tests #29552
   addon: a11y bug ci:normal +3
   
4. Improve accessibility of our own Storybook components

🚩 Out of scope

• Create a unified API to replace Status API, Testing Module API and Reporting API.