Techniques/UAtesting

User Agent Testing Methodology and Notes

This document outlines the proposed user agent testing methods and technologies used during the development of both ARIA and HTML techniques by the WCAG HTML techniques task force. This page contains some suggested user agent and Assistive Technology (AT) combinations that can potentially be used (in whole or in part) by task force members in testing their draft techniques.

Note: Not every member of the task force has access to the same AT during technique development and testing. However, ensuring access to any given browser is trivial.

Some sample Browser and Screen Reader combinations

The following are examples of user agent and AT combinations that may be used for testing. Each technique will contain a note that outlines what combinations were used during the development of the technique.

Firefox [>8] and JAWS [>v8]
Firefox and NVDA
Firefox and WindowEyes
Firefox and Firevox
IE and JAWS [>v8]
IE and NVDA
IE and WindowEyes
Safari [>=v5] and VoiceOver
Chrome and ChromeVox

Mobile Testing

Where possible, some techniques are tested on mobile devices and tablets that may use the following:

iOS: Safari [>=v5] and VoiceOver
Android [>=4.1]: Chrome, Firefox and Talkback
Windows Mobile: [tbc]
Nokia: [tbc] Mobile Speak
Blackberry: [tbc]

Methodology Limitations

Not everyone uses AT such as a screen reader in the same way. The spectrum of digital literacy amongst users is very broad from basic to more advanced knowledge of both a screen readers capabilities and functionality. Where possible the techniques that are developed by the task force are tested in a broad variety of ways. If required, to ensure the technique is robust also often in different contexts. This is because some WAI-ARIA states and properties may or may not be picked up by a screen reader, depending on the context within which they are found. @@needsinfo/examples.

In terms of how the author of a technique may choose to 'browse' or test a particular example, this is primarily down to their own choice and level of experience. However, care is taken by the members of the task force to try to ensure that as many levels of ability are emulated from basic tabbing through an interface to using the virtual cursors that a screen reader evokes.

For example, this may include testing widgets in 'application mode' (where the aria role="application" has been applied in whole or in part) and so on. From the feedback that the task force hopes to get when the techniques go to both internal review within the WCAG group, and then external public review - we will incorporate any feedback that makes a sensible case for the modification of a technique to better support a form of user interaction that the author may have initially missed.

User Agent Support Notes

The task force will be documenting support notes for particular user agent combinations, as the development of techniques progresses. These support notes aim to note bugs and any user agent issue, quirks and so on that the task force members find when they are drafting techniques.

At time of writing each technique includes a section called User Agent Notes. This part of the technique will give an overview of the browsers and AT used in testing and any bugs or issues that the author encountered. For the purposes of illustration, a sample from a draft technique could take the form:

Sample User Agent Notes

Jaws V.11 and greater has complete support.
ChromeVox V.1 and greater has complete support.
VoiceOver V.3 and greater supports.
NVDA 2 supports Aria Describedby.
Window Eyes as of V.7.

Approaches to Screen Reader Testing

There are many approaches to testing WCAG techniques. How and what is tested is formed largely by both an individuals access to/or the availability of any given screen reader, their level of familiarity with a particular screen readers functionality and what is being tested.

Most approaches are actually ad hoc, but below we have compiled a list of several quality resources that help to inform the process. The HTML Techniques Task Force actively helps its members to improve our overall level of digital literacy, and we openly share our knowledge as we try to improve the WCAG techniques that we are developing, as well as the methods we use to test them.