Automated accessibility testing - do’s, don’ts and maybes

What accessibility is and isn’t

Accessibility is ensuring you can communicate your content effectively to your audience. This means making your content consumable for your users; regardless of the medium and regardless of any user impairments.

It takes many people to build a website, and it's likely that not everyone involved will have the same understanding about how the decisions they make will be perceived by the end user. So how do we ensure accessibility? Unfortunately there’s no silver bullet. Automation is possible for some aspects, but it won't ensure your content is accessible.

So what is accessibility testing?

Accessibility testing is:

• Continual — e.g. each new article requires a test of its accessibility at some level.

• Evolutionary — technology changes, and websites should keep up with users.

Accessibility testing is not:

• An afterthought — though many sites consider it to be. Often systemic accessibility issues are seeded early in a project's design. Considering accessibility upfront will lead to better websites.

• Automated "set and forget" — not everything can be automated.

• Easy — but you can make it easier by keeping things simple.

What can and can’t accessibility tools check?

Automated tools exist to help with accessibility checking, but automation and tooling alone cannot ensure accessibility. It's not just the mechanics of the site, but also the meaning of the content.

Let's explore what tools can and can't provide.

Tools can check:

• Code and markup syntax

• Helps ensure browsers display your site correctly

• Colour contrasts

• Helps with visibility for poor-sighted users

• Missing tags

• Often related to accessibility for screen-readers (alt tags, aria tags)

Tools can't check:

• Semantically correct use of tags

• "Is this list of links site navigation, citation or something else?"

• "Yes this is a heading, but should it be?"

• Missing content

• e.g. a video with no transcript — "Does the video need a transcript?"

• Custom component functionality

• If you build your own component, how can you ensure it's functional across all devices? Test scripts may help check basic functionality, but this will probably need manual testing

• Content

• Is the language appropriate for the user?

• Is the article fit for purpose?

• Is there any missing information?

Considerations for accessibility testing

Accepting we cannot go all the way with automation, is there a benefit to using automation at all? Well every bit helps! If your website uses a continuous integration (CI) pipeline, this may be a good place to include automated accessibility tests, as often other unit tests and end-to-end tests will be run here as well.

These automated tests could include the use of Powermapper SortSite. We have discussed in a previous article the benefits of SortSite; it's a tool that validates web pages for HTML, spelling, dead links, accessibility issues and more. The developer edition supports the use of command line for use in CI pipelines, but it’s quite expensive.

When it comes to manual content checking, a solution could include:

• Creating a checklist of expectations for content

• A human tester to check compliance with the above

• An additional step in the publishing workflow

The publishing workflow could look like this:

Draft → Content review → Accessibility review → Published → Archived

Including Accessibility review as an additional step reinforces the need to test with user accessibility in mind.

Salsa's take

Salsa is always looking for ways to improve the quality of our sites. Currently, our accessibility testing is undertaken as part of our quality assurance step in development. This is a manual process, and does take additional time and resources. In line with the tips and techniques presented in this article, Salsa makes every effort to automate where we can. We hope in sharing the ideas above others can consider the case for automation too.