Accessibility Manual Testing
Step-by-step guide for testers on how to deal with the accessibility.
There are many articles for web developers about accessibility and hundreds of pages of detailed documentation. But when the task of accessibility testing turns out to be on the side of the Quality Assurance department, this information should be transformed into the requirements of users with disabilities.
This article covers some testing vectors if a test engineer has to perform an accessibility audit.
Disclaimers:
- I assume that the reader is familiar with the concept of accessibility (A11y) and technologies of its implementation (see references at the end of the article);
- The article does not cover automated testing;
- The basic checks listed below execute without using a screen reader;
- As an example, a page of an album on Yandex Music was selected.
Keyboard Navigation
The first set of tests is aimed at what can be seen with the eyes and reproduced without additional tools.
1. Focus on Interactive Elements
Try to navigate through the page by TAB key. Pay attention to the selection of active elements during navigation — interactive elements should be focusable. Users should clearly understand where they are on the site.
Actual result:
- All buttons and links do not have focus;
- Most of the elements have the property
outline: 0px
, which is an antipattern.
Expected: if the element is interactable, it will have a visible indication.
Wherefore? The highlighted focus is visual feedback for the users.
This is 2.4.7 «Focus Visible» WCAG requirement (more info).
It is recommended to provide an obvious focus style (especially if the default one is removed):
2. Sequence of Elements During Tabbing
Navigate through the page by TAB key. Interactive elements should be selected from left to right and from top to bottom.
The actual result matches with expected: navigation takes place in a logical sequence. The visual order of the elements corresponds to the real order in the DOM-tree and CSS properties do not change the order of elements.
This is 1.3.2 «Meaningful Sequence» and 2.4.3 «Focus order» WCAG requirements (more info).
3. TAB Order
Navigate through the page by TAB key. All interactive content should be intractable.
Actual result:
- Search button is missing for keyboard navigation. Users without a mouse or touchpad are unable to make a search request on the site.
- It is impossible to play an arbitrary track from the album. Interactive elements ([play], [like], and [dislike] buttons) are not a part of the tab order, and they appear only when the user hovers a mouse over them.
Expected: interactive content is intractable.
Wherefore? To use all functions of the website.
This is 2.1.1 «Keyboard» WCAG requirement (more info).
Elements can be included or excluded from the tab order by tabindex value.
4. Tabindex Values
Check the values of the tabindex
attributes in the page’s source code.
Actual result: multiple elements with tabindex
> 0.
Expected: the only one element with tabindex
> 0.
Wherefore? In order not to confuse the browser or to specify the first focus element (when using only one tabindex = 1).
Despite that the tabindex
can be greater than 0 (but not exceed 32767), it is recommended to have a single element with tabindex
> 0, because it will be the first one to receive focus from the keyboard by TAB.
Semantics
The second set of tests is aimed at how the site is ready to work with assistive technologies.
5. Language Declaration
Yandex Music has localisation — the page language could be changed into Russian, Armenian, Uzbek, Kazakh, Ukrainian, or Azerbaijani. Therefore, you need to check the declaration of the page language.
Actual result: <html>
tag does not have any attributes.
Expected: <html lang="en">
.
Wherefore? To help a speech synthesizer accurately determine the language of the page.
Each language locale must correspond to the value from the IANA registry: ru, hy, uz, kk, uk, or azb.
6. Document Structure
Check for the appropriate section heading elements.
The actual result matches with expected: titles are marked with the correct tags.
Wherefore? A well-thought-out structure of the document will help users to navigate through the page, because screen readers allow fast switching between headings.
This is 1.3.1 «Info and Relationships» WCAG requirement (more info).
If the layout consists of only <div>
s, it is still possible to include them into A11y-tree by adding the corresponding ARIA roles: role= "heading" aria-level= "1"
.
7. Landmarks
Check for ARIA Landmark Roles attributes in the page’s source code.
Actual result: nothing.
Expected: page areas are marked up by the main ARIA roles in accordance with their content: banner, complementary, contentinfo, main, navigation, search.
Wherefore? Landmarks, same as semantic headers, will help users to navigate through the page, because screen readers allow fast switching between landmarks.
This is 1.3.6 «Identify Purpose» WCAG requirement (more info).
Landmarks determine the purpose of the page areas and, in a way, duplicate content sectioning HTML5 elements: <header>
, <aside>
, <footer>
, <main>
, <nav>
. However, it is still necessary to specify HTML5 tags by additional ARAI roles to support older browsers and bypass bugs.
8. Interactive Content Description
Check the presence and clarity of the description of interactive elements (buttons and links). It is useful to try a screen reader to better understand how the text will be read.
Actual result: there are no descriptions, or they are incomprehensible.
Expected: descriptions reflect the logical meaning of interactive elements. For example:
- Instead of the standard name of the [contemporary jazz] link, it could be
aria-label="Open contemporary jazz genre page"
; - Instead of the standard name of the [play] button, it could be
aria-label="Play album"
; - Instead of missing the […] button description, it could be
aria-label="Open album menu"
.
Wherefore?
- When the interface is not visible, it can and should be described: what exactly the buttons and links do;
Aria-label
attribute is a text which will be read by a screen reader.
This is 2.4.4 «Link Purpose (In Context)» WCAG requirement (more info).
9. Image Description
Check for alt
attribute for images.
Actual result: images do not have alternative text.
Expected:
- Image has attribute
alt="Album cover"
; - Site logo has a title. Despite the fact that the logo is not being placed through
<img>
tag, the link should have an alternative text:aria-label="Yandex Music"
.
Wherefore? It is necessary to provide a descriptive alternative text for contextual images, so that the users with a screen reader can get an idea of the images or image buttons.
This is 1.1.1 «Non-text Content» WCAG requirement (more info).
10. Color Vision
Check:
- Displaying the site in simulating color blindness modes by using color vision simulation built-in Firefox or by third-party browser extension (see references at the end of the article);
- Hover on links in each of the color blindness modes;
- Text contrast ratio by using Accessibility Inspector in Firefox.
Actual result:
- Non-contrast text. «ALBUM», «Artist», «2020», and «contemporary jazz» are too light — the color contrast between background and text should be great enough to ensure legibility — minimum ratio is 4,5:1 (more info);
- Hover and active states of the [contemporary jazz] link are not visible in achromatopsia mode.
Expected:
- Contrast text;
- The links are distinguishable from the text;
- Underlined links (at least when hovered, so that they can be easily distinguished from the normal text).
Wherefore?
- In order not to use color exclusively for differences between page elements;
- To take care of users with color vision deficiency and make the text and links more readable.
This is 1.4.1 «Use of Color» and 1.4.3 «Contrast (Minimum)» WCAG requirements (more info).
11. Resize Text
Zoom page to 200%.
The actual result matches with expected: the layout has adapted to the proportions of the text.
This is 1.4.4 «Resize text» WCAG requirement (more info).
12. Self-check in Lighthouse
Run Accessibility Audit in Chrome Dev Tools.
Look what was missed during manual tests.
If you do not know what to check, then an automatic audit could be started at the beginning of the testing session. Accessibility Inspector in Firefox is usable to identify A11y issues in specific elements.
For further testing, you should start using a screen reader.
References
Similar Articles
- How To Do an Accessibility Review
- Accessibility audits
- Testing for accessibility
- Auditing For Accessibility Problems With Firefox Developer Tools
- Beyond automatic accessibility testing
Resources
- The A11Y Project
- A11y Weekly
- Accessible to all
- Web Accessibility Checklist
- Accessibility Developer Guide
- Accessibility resources, guides, communities, and more
- Accessibility Human Interface Guidelines — Apple Developer
Standards
- Accessibility
- Web Content Accessibility Guidelines (WCAG) 2.1
- Understanding Requirements (Level A/AA/AAA)
- WCAG Checklist (WebAIM)
W3C about ARIA
- WAI-ARIA Overview
- Accessible Rich Internet Applications (WAI-ARIA)
- Using ARIA
- ARIA in HTML
- WAI-ARIA Authoring Practices
- ARIA Design Pattern Examples
From MDN
From Google
Tools
- Accessibility Inspector in Firefox Developer Tools
- Accessibility In Chrome DevTools
- axe
- axe-core
- Testing Website Accessibility with Axe and WebdriverIO
- Lighthouse
Screenreaders
- VoiceOver for Mac (part of macOS)
- Using VoiceOver to Evaluate Web Accessibility
- NVDA for Windows (free)
- JAWS for Windows (paid)
- Orca for Linux (Gnome)
Browser Extensions
- ChromeVox — a screen reader for Chrome
- axe — Web Accessibility Testing — accessibility checker for developers
- Equal Access Accessibility Checker — accessibility checker by IBM
- Siteimprove Accessibility Checker — evaluates accessibility issues
- WAVE Evaluation Tool — evaluates web accessibility
- Accessibility Insights for Web — highlights accessibility issues
- Visual ARIA — displays ARIA attributes
- Web Disability Simulator — simulates color blindness, low vision, dyslexia, and more
- NoCoffee — simulates visual disturbances associated with fatigue
- High Contrast — changes color contrast
- Leonardo — contrast-based color generator
Useful Articles
- 101 Digital Accessibility (a11y) tips and tricks
- Screen Reader User Survey #9 Results
- Accessible design from the get-go
- A Complete Guide To Accessibility Tooling
- A Complete Guide To Accessible Front-End Components
- Web Accessible Code Libraries and Design Patterns
- Accessible to some
- What a Year of Learning and Teaching Accessibility Taught Me
- How accessibility trees inform assistive tech
- Building the most inaccessible site possible with a perfect Lighthouse score
- Pragmatic rules of web accessibility that will stick to your mind
- Writing HTML with accessibility in mind
- Things I learned by pretending to be blind for a week
- What we found when we tested tools on the world’s least-accessible webpage
Articles about ARIA
- What Is ARIA Even For
- Web Accessibility — ARIA
- Why, How, and When to Use Semantic HTML and ARIA
- ARIA Landmark Roles
- Accessible Landmarks
- Using WAI-ARIA Landmarks
Courses
- Free PDF book: Giving a damn about accessibility by Sheri Byrne-Haber
- Web Accessibility — free course for web developers; it partially overlaps with the videos of A11ycasts with Rob Dodson.