A Reference Manual for Automating Accessibility Testing in React/Gatsby(i)
From various testing framework integrated with axe-core, to linting plugins and full page auditing…
Not long ago, I raised a question regarding “Accessibility” on one of our main repo (running in Gatsby and serve as our main client web application). And my boss asked me to dig deep into this topic and plan some actions. So I went on and did some research on my own, and I feel it’s worth sharing my findings to a wider group.
Note: the findings presented here is customised to fit in Gatsby development. But since Gatsby is built on top of React, any React users can find it helpful as well.
Accessibility is important in web development. Unfortunately, it can be de- prioritised to other factors like performance in early stage of a web application. Luckily, there are more and more automated accessibility testing tools available to the community, so developers can make accessibility an integral part of their testing philosophy from start of the application development without much hassle.
It is claimed that automated testing tool can help identify 30% issues regarding Accessibility. While it’s not ideal, but it’s better than nothing. As we should aim to automate as much as we can, since it alleviates the burden from replying on pure manual testing and narrow the focus of manual testing from including everything in Accessibility to limited areas (assistive technologies for example).
!Note: this guide is for automated accessibility testing only. But it is still advised to include manual checking, auditing, and user testing altogether to underpin your Accessibility pillar.
Here we are going to discuss 3 categories of automated testing tool:
- Tools that integrated with existing testing libraries/frameworks
- Linting tools (link)
- Full page web report (link)
Tools that integrated with existing testing libraries/frameworks
When we talk about automated testing library for accessibility, it’s difficult to not talk about axe-core. To keep it brief:
- It’s open source
- It’s backed up by Deque Systems, a major accessibility vendor
- It’s designed to work on all modern browsers and with whatever tools, frameworks, libraries and environments you use today.
- It returns zero false positives (bugs notwithstanding).
It also allows users to customise settings from a list of rules available.
Here we are going to have a look at some of the tools that have axe-core built at its core and at the same time, smoothly connect with a major testing library/framework.
- axe-webdriverjs
- cypress-axe
- axe-testcafe
- react-axe(gatsby-plugin-react-axe)
- jest-axe
Axe-webdriverjs
This is built on top of Selenium WebDriver with an Axe API. So in order to use it you need both axe-webdriverjs and selenium-webdriver .
Since it’s run on top of Selenium WebDriver, it is for e2e test, as you can see that below you need to insert an url link.
//Somewhere in your project create a js file to include:const axeBuilder = require('axe-webdriverjs');
const webDriver = require('selenium-webdriver');// create a PhantomJS WebDriver instance since it uses PhantomJS to //show the result
const driver = new webDriver.Builder()
.forBrowser('phantomjs')
.build();// run the tests and output the results in the console
driver
.get('http://YOURSITE.com') //your site to audit goes here
.then(() => {
axeBuilder(driver)
.analyze((results) => {
console.log(results);
});
});
And to run the test, if you have PhantomJS installed in your project you can run it from console. Otherwise you need to use node axe.js (Of course you can put the command as a npm script like "test": "node axe.js") .
All is good. But the inconvenience is that you need to set up your test script for accessibility separately as your existing testing framework. Or if you prefer not, you can integrate it with your own testing framework. (Still another layer of complexity)
//example using mocha test runnerconst assert = require('chai').assert;
const axeBuilder = require('axe-webdriverjs');
const webDriver = require('selenium-webdriver');const driver = new webDriver.Builder()
.forBrowser('phantomjs')
.build();describe('aXe test', () => {
it('should check the main page', () => {
return driver.get('http://YOURSITE.com/')
.then(() => {
return new Promise((resolve) => {
axeBuilder(driver).analyze((results) => {
assert.equal(results.violations.length, 0); resolve()
});
});
})
.then(() => driver.quit())
})
.timeout(0);
});
React-Axe
In comparison, if you want an almost zero configuration tool and you are using React. Then maybe tryreact-axe which provides a React wrapper around the axe-core testing tool.
After you do npm install --save-dev react-axe , the only config you need to do is to initialize the module in index.js to dynamic import the library so it’s only load in a non-production environment and not to be included in the final production bundle.
if (process.env.NODE_ENV !== 'production') {
import('react-axe').then(axe => {
axe(React, ReactDOM, 1000);
ReactDOM.render(<App />, document.getElementById('root'));
});
} else {
ReactDOM.render(<App />, document.getElementById('root'));
}And now when you are in development, any violations will be printed directly to Chrome DevTools console.
Gatsby-plugin-react-axe
A very similar tool for users of Gatsby specifically is this Gatsby plugin. As you can expect from its name, it’s really just another wrapper of React-axe to fit into the Gatsby framework.
After npm install relevant plugin, simply include it in the gatsby-config.js under plugins array:
// gatsby-config.jsmodule.exports = {
plugins: [
{
resolve: 'gatsby-plugin-react-axe',
options: {
// Integrate react-axe in production. This defaults to false.
showInProduction: false,
// Options to pass to axe-core.
axeOptions: {
// Your axe-core options.
},
// Context to pass to axe-core.
axeContext: {
// Your axe-core context.
}
},
},
],
}
And just like React-axe, it will automatically print violations to the console in dev mode.
While both are convenient in terms of providing real-time feedback during development, but it still reply on human interventions in that these violations have to be dealt with. In other words, a lazy developer like me can totally ignore it and pretend that everything is on the happy path.
So we need something that is more stringent and can potentially be built into the CI pipeline.
Axe-testcafe
I admit that the primary reason to evaluate this tool in more details instead of the next one with Cypress is because we are using Testcafe instead of Cypress as our E2E tool.
(I don’t want to be off the topic, but our preference to be able testing across different browsers using Testcafe overweighs the convenience of have recording videos or nice GUI displays using Cypress).
To use axe integration with Testcafe, you need to install both axe-core and axe-testcafe .
Once install, you can start to use the tool as an example illustrates below:
import { axeCheck, createReport } from 'axe-testcafe';fixture `TestCafe tests with Axe`
.page `http://YOURSITE.com`;test('Automated accessibility testing', async t => {
const { error, violations } = await axeCheck(t);
await t.expect(violations.length === 0).ok(createReport(violations));
});
You can also configure your axe-core functionality per your preference. (example in TypeScript)
declare module ‘axe-testcafe’ {
import { ElementContext, RunOnly, AxeResults, Result,} from ‘axe-core’;
import ‘testcafe’;
export function axeCheck(
t: TestController,
context?: ElementContext,
options?: {
runOnly?: RunOnly;
rules?: Record<string, any>;
iframes?: boolean;
elementRef?: boolean;
selectors?: boolean;
}
): Promise<AxeResults>; export function createReport(violations: Result[]): string;
}
Cypress-axe
For people who prefer using Cypress as an e2e testing tool instead, similar set up can be found in the head link.
Jest-axe
Similar to the options listed above, at its core, jest-axe is just a nice wrapper outside axe-core for Jest tests.
Different to the other E2E tools above, it can be ran at a component or page level as preferred.
To use it you need to have React and ReactDOMServer installed along with jest-axe , and there’s no configuration required. An example below to illustrate how the test can be run.
import React from 'react';
import ReactDOMServer from 'react-dom/server';
import { axe } from 'jest-axe';
import 'jest-axe/extend-expect';it('has no programmatically detectable a11y issues', async () => { // pass anything that outputs html to axe
const html = ReactDOMServer.renderToString(
<MyComponent />
);
const results = await axe(html);
expect(results).toHaveNoViolations();
});
And when I did it, I got some nice printed accessibility violations along with my normal unit jest tests — and there are some nice suggestions to fix the problem:
~ jest MyComponent.spec.tsx FAIL src/components/MyComponent.spec.tsx (5.289s)
<MyComponent />
✓ should render (19ms)
✓ should render graphql image if lender name contains in the query (6ms)
✓ should display expand button correctly (4ms)
✕ has no programmatically detectable a11y issues (203ms)────────Expected the HTML found at $(‘#undefined’) to have no violations:<input type=”checkbox” class=”read__more__state” id=”undefined”>Received:
“Form elements must have labels (label)”Fix any of the following:
aria-label attribute does not exist or is empty
aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
Form element does not have an implicit (wrapped) <label>
Form element does not have an explicit <label>
Element has no title attribute or the title attribute is emptyYou can find more information on this issue here:
https://dequeuniversity.com/rules/axe/3.5/label?application=axeAPI────────Expected the HTML found at $(‘section’) to have no violations:
<section class=”table-card__container”>Received:
“All page content must be contained by landmarks (region)”Fix any of the following:
Some page content is not contained by landmarksYou can find more information on this issue here:
https://dequeuniversity.com/rules/axe/3.5/region?application=axeAPI115 | const html = ReactDOMServer.renderToString(wrapper);
116 | const results = await axe(html);
> 117 | expect(results).toHaveNoViolations();
| ^
118 | });
119 | });
120 |at src/components/MyComponent.spec.tsx:117:21
at fulfilled (src/components/MyComponent.spec.tsx:5:58)Test Suites: 1 failed, 1 total
Tests: 1 failed, 10 passed, 11 total
Snapshots: 1 passed, 1 total
Time: 6.281s
Ran all test suites matching MyComponent.spec.tsx
And similar with other tools that have axe-core as the backbone, you can configure Axe in a setup file.
// Global helper file (axe-helper.js)
const { configureAxe } = require('jest-axe')const axe = configureAxe({
rules: {
//rules go here
}
})module.exports = axe
Summary
Above all the 5 tools mentioned above, for me a personal preference is the last 3 options because:
- fully integrated with an existing testing framework/library that require no configuration between the package and the testing library (unlike axe-webdriverjs)
- built into continuous integration pipeline so it provides confidence in preventing regressions comparing to logs in console that is optional and can be omitted
For my own project, I’m considering picking jest-axe primarily because I’m already using Jest extensively in unit testing, and it suits my need to test reusable components built in Gatsby from a component rather than a whole page level when I have few fixed pages. But for you, feel free to choose a package that can work with your existing framework best.
