[Draft] Selecting Web Accessibility Evaluation Tools

Note: This document is a draft [see change log in progress] and should not be referenced or quoted under any circumstances. This document is under development by the Education and Outreach Working Group (EOWG), and will be offered to other W3C groups and the public for review.

1. Introduction

Web accessibility evaluation tools are software programs or online services that help determine if a Web site is accessible, and help improve Web accessibility. This document explains different features of evaluation tools and helps to determine which types of tools and features would best meet your specific needs.

WAI encourages the development and evolution of Web accessibility evaluation tools, and maintains an extensive list of evaluation, repair and transformation tools. WAI does not endorse or promote any single tool or vendor.

2. Evaluating the Conformance of Web Sites to Accessibility Guidelines

Read the section Conformance Evaluation to the Web Content Accessibility Guidelines 1.0 of the document Evaluating Web Sites for Accessibility to find out how to carry out a comprehensive evaluation which combines semi-automatic, manual, and user testing of accessibility features. There are several types of tools that help developers evaluate the conformance of Web sites to the Checkpoints of the Web Content Accessibility Guidelines 1.0, they are divided into semi-automated and manual tools as described below; sometimes evaluation tools provide both these modes.

Semi-Automated Web Accessibility Evaluation Tools

Generally, semi-automated evaluation tools attempt to evaluate Web pages with little or no user interaction. Usually these tools produce reports with the results of the checks which they carried out. Sometimes, these tools have wizard interfaces which display one result at a time; possibly with some repair options. Asking the following questions about the features of semi-automated evaluation tools help determine their efficiency:

Accessibility: How accessible is the interface of the evaluation tool for people with disabilities?

It is equally important to ensure that people with disabilities can effectively contribute to the Web, as it is for them to be able to effectively use the Web. Evaluation tool vendors can provide accessibility by following the Authoring Tool Accessibility Guidelines 1.0.

Coverage: How many checkpoints is the evaluation tool able to check?

Even though evaluation tools may not always be able to make definitive judgements on the conformance to certain checkpoints, they can help developers by at least identifying potential accessibility barriers and ruling out checkpoints which are not applicable to the content.

Precision: How correct are the results provided by the evaluation tools?

Every evaluation tool is prone to claiming false results such as not detecting potential accessibility barriers, or wrongly claiming that Web content does not conform to certain checkpoints. Accurate tools provide more reliable results and thus increase efficiency of the evaluation.

Crawling: How well does the evaluation tool traverse the entire target content?

In many cases, the conformance evaluation involves more than evaluating a single page but rather an entire Web site or a sub site. Evaluation tools should be able to evaluate groups of related pages by examining the links within the markup or by using the directory structure on the server.

Configuration: How well does the evaluation tool adapt to the requirements of the user?

Often Web developers will want to adapt the evaluation tools to accommodate their individual needs such as customizing the user interface, customizing the generated reports, or selecting which checkpoints should be tested during the evaluation.

Integration: How well does the evaluation tool integrate into the environment of the Web developer?

Some evaluation tools provide plug-in interfaces for authoring tools such as editors or content management systems, others generate reports in machine readable formats such as XML or EARL. These features increase the compatibility of the evaluation tools with the authoring tools of the Web developers.

Repair: How well does the evaluation tool assist in retrofitting Web content?

Sometimes evaluation tools can fix some accessibility barriers by correcting or adding markup but often the Web developers need to decide how to repair the error. Tools should assist the developers by providing appropriate repair suggestions and educational material.

Manual Web Accessibility Evaluation Tools

Manual Web accessibility evaluation tools are designed to involve Web developers during the evaluation process. Some tools insert icons and markup into the Web pages to highlight the results of the accessibility checks on the pages. Other tools guide Web developers through checks in a certain order; these checks are usually carried out manually by the developers. Manual Web accessibility evaluation tools help educate Web developers in understanding the impact and context of the accessibility barriers which leads to long term resolution of mistakes. Asking the following questions about the features of manual evaluation tools help determine their efficiency:

Effectiveness: How well does the evaluation tool assist Web developers in locating potential accessibility barriers?

Either by displaying precise information at the correct location of the Web pages or by suggesting an appropriate series of checks, evaluation tools can help Web developers to quickly identify problems and their context.

Education: How comprehensive is the educational material which the evaluation tool provides for Web developers?

Often it is not enough to identify potential accessibility barriers but it is just as important to educate the developers about the rationale behind the relevant checkpoints and possibilities on how to conform to them.

3. Evaluating the Accessibility Features of Web Sites

Especially during the design phase, Web developers are interested in learning about how their Web site renders on different configurations and browsers in order to identify potential accessibility barriers, identify missing accessibility features, or to enhance existing ones. Standard browser features such as turning off images, disabling scripting, or enlarging font sizes are helpful in evaluating accessibility features of Web sites. Additionally, filter and transformation tools assist the evaluation of Web sites by modifying the appearence of the Web pages; for example by displaying the content in text only, by presenting the Web pages without color, or by reading the content aloud. In order to evaluate the accessibility features of a Web site effectively, evaluation tools which provide the following should be used:

• Enlarge the font sizes to evaluate how flexible the layout design is.
• Replace images with their alternate text to evaluate how appropriate they are.
• Read the Web pages aloud to evaluate how they render on voice browsers and screen readers.
• Present the Web pages with reduced colors to evaluate how well the color contrast is.
• Linearize table cells to evaluate the reading order of the table content.
• Display Web pages without scripting to evaluate how robust the functionality is.

Last modified: $Date: 2005/02/01 21:54:57 $ by $Author: shadi $

Last updated 20 January 2005 by Shadi Abou-Zahra. Editors: Shadi Abou-Zahra and Judy Brewer, with assistance from participants of the EOWG.

Copyright © 2005 W3C (MIT, ERCIM, Keio ), All Rights Reserved. W3C liability, trademark, document useand software licensingrules apply. Your interactions with this site are in accordance with our public and Member privacy statements.