18 – Accessibility (Pa11y)
The test audits the page with Pa11y and counts the accessibility errors that can be found mechanically.
Automation isn't enough
Automated tools find roughly a third of all WCAG problems. They can see that an image is missing alternative text, but not whether the text describes the image. A score of 5 here means no machine-detectable errors remain. It does not mean the website is accessible.
What is tested?
- The number of errors
- The number of distinct error types
If Pa11y finds nothing, the page is run again with Axe as a second opinion, since the tools cover partly different rules.
How it affects the score
Both the number of errors and the number of error types are weighed in. Fifty instances of the same error lower the score less than fifty different kinds of error. Take a table with consistently wrong header markup. That is usually one fix in one place.
| Score | Meaning |
|---|---|
| 5.0 | No apparent accessibility problems |
| 4.0 | Could be more accessible, but is rather good |
| 3.0 | About average, needs to get better |
| 2.0 | Quite bad for users with disabilities |
| 1.0 | Really bad, probably for everyone |
Running the test
python default.py -u https://example.com -t 18 -r
With --setting general.review.details=true each error type is listed with a count, which is what you want when you're about to fix them.
See also
- 26 – Accessibility Statement (Alfa): the legal requirement
- 30 – Lighthouse: Google's accessibility score