skip navigation barsW3C home pageWeb Accessibility Initiative (WAI) home page
W3C Index - W3C Search - W3C Translations
WAI Resources - WAI Site Map - About WAI

[Draft] Selecting Web Accessibility Evaluation Tools

Note: This document is a draft [see change log in progress] and should not be referenced or quoted under any circumstances. This document is under development by the Education and Outreach Working Group (EOWG), and will be offered to other W3C groups and the public for review.

1. Introduction

Web accessibility evaluation tools are software programs or online services that help determine if a Web site is accessible, and help improve Web accessibility. Read the section Conformance Evaluation to the Web Content Accessibility Guidelines 1.0 of the document Evaluating Web Sites for Accessibility to find out how to carry out a comprehensive evaluation which combines semi-automatic, manual, and user testing of accessibility features. This document explains different features of evaluation tools which can be used during a conformance evaluation and helps you to determine which types of tools and features would best meet your specific needs.

How Evaluation Tools Can Help You

Web accessibility evaluation tools can significantly reduce the time and effort of conformance evaluations by carrying out automated checks and by assisting Web developers in evaluating checkpoints which need to be evaluated manually. Specifically, evaluation tools should perform the following functions:

How Evaluation Tools Can Not Help You

Many accessibility checks remain non-automatable by current computer technology, these must be manually evaluated by experienced Web developers. The inaccuracy of evaluation tools, especially for non-automatable checkpoints, can be very misleading when Web developers lack fundamental knowledge about key accessibility requirements. Web accessibility evaluation tools can not make your Web site accessible, they can only assist you in doing so.

Disclaimer

WAI encourages the development and evolution of Web accessibility evaluation tools, and maintains an extensive list of evaluation, repair and transformation tools. WAI does not endorse or promote any single tool or vendor.

2. Types of Web Accessibility Evaluation Tools

There are several types of Web accessibility evaluation tools that help developers evaluate the conformance of Web sites to the Checkpoints of the Web Content Accessibility Guidelines 1.0, they are roughly divided into report, wizard, visual feedback, and manual evaluation tools as described below; sometimes evaluation tools can provide more than one of these operation modes.

Web Accessibility Evaluation Tools which Generate Reports

Generally, report evaluation tools attempt to evaluate Web pages with little or no user interaction, and generate reports with the results of the accessibility checks that they carried out. Report evaluation tools are very useful in quickly determining the conformance of Web content to the checkpoints which can be automatically evaluated, or for identifying which of the remaining checkpoints need to be manually evaluated.

Web Accessibility Evaluation Tools with Wizard Interfaces

Wizard-based evaluation tools guide Web developers through a series of checks in a defined sequence in order to determine the conformance of the Web content to accessibility guidelines. Sometimes these tools are able to automatically execute some of the accessibility checks, leaving the remainder of the checks to be manually carried out by the Web developers. For example, an evaluation tool with a wizard interface may display the images of a Web page and their corresponding descriptions one at a time so that Web developers can evaluate how appropriate each of these descriptions are.

Web Accessibility Evaluation Tools with Visual Feedback

Visual feedback evaluation tools display the results of automated accessibility checks on the respective locations of the Web pages by inserting icons and markup into the code of the pages. By marking the location and type of errors onto the Web pages, developers can recognize the context and impact of these errors thus increase their awareness and expertise in Web accessibility.

Web Accessibility Evaluation Tools for Manual Checkpoints

Manual evaluation tools assist Web developers by executing checks which are related to accessibility checkpoints, or by modifying the appearence of the Web pages to help identify accessibility barriers. For example, by displaying the content in text only, by presenting the Web pages without color, or by reading the content aloud, experienced Web developers can effectively evaluate and improve the accessibility features of their Web sites.

3. Evaluating Web Accessibility Evaluation Tools

According to your specific organization, different aspects of Web accessibility evaluation tools may be more relevant. Below are some questions which can be used to compare evaluation tools in order to determine which are most suitable for your specific needs:

Accessibility: How accessible is the evaluation tool for people with disabilities?
It is equally important to ensure that people with disabilities can effectively contribute to the Web, as it is for them to be able to effectively use the Web. Evaluation tool vendors can provide accessibility by following the Authoring Tool Accessibility Guidelines 1.0.
Automation: How many checkpoints is the evaluation tool able to automatically address?
The degree of automation significantly impacts the efficiency of the Web developers. Both the detection and the repair of accessibility barriers should be automatable to the highest degree possible.
Checkpoint Coverage: How many checkpoints is the evaluation tool able to adequately address?
Even though evaluation tools may not be able to automatically determine the conformance to all checkpoints, tools should address the remaining checkpoints by determining their relevance and by assisting Web developers in evaluating them.
Collaboration: How well does the evaluation tool support the collaboration of Web developers?
Evaluation tools can support the collaboration of developers in evaluating and repairing Web content by providing shared repositories of the results, or by making the results available in open data formats such as XML or EARL.
Configuration: How well does the evaluation tool adapt to the requirements of the user?
Evaluation tools should be able to adapt to the individual needs of the Web developers. For example by allowing customization of the user interface, the accessibility checks, the generated results, or the repair options.
Documentation: How informative are the educational resources supplied by the evaluation tool?
To better assist developers in learning about the Web accessibility and how to correctly implement it, evaluation tools should provide comprehensive resources to help educate the developers.
Integration: How well does the evaluation tool integrate into the environment of the Web developer?
Some evaluation tools provide plug-in interfaces for the authoring tools (such as editors or content management systems), or otherwise integrate into the development environment of the Web developers.
Platform Coverage: How well does the evaluation tool support the operating system of the Web developer?
Even though some evaluation tools may be available on more than one operating system, they may sometimes not support the same features or perform equally on all platforms.
Precision: How accurate are the results delivered by the evaluation tool?
Inaccurate results (such as not detecting accessibility barriers, or detecting wrong ones) decrease the reliability of the evaluation tool thus decrease the efficiency of the evaluation.
Repair: How well does the evaluation tool assist developers in retrofitting Web content?
There are several ways in which evaluation tools can assist developers in retrofitting inaccessible content, for example by providing repair suggestions or automating routine repair measures.
Reporting: How well does the evaluation tool support reporting for different user roles?
According to the roles of the users (for example Web developer, project manager, etc), different types of reports may be better provide targeted information to these user groups.
Site Coverage: How well does the evaluation tool cover the relevant sections of a Web site?
In many cases, Web developers will need to evaluate entire Web sites rather than single Web pages. Evaluation tools should be able to evaluate whole groups of related pages.
Technology Coverage: How well does the evaluation tool support the technologies of a Web site?
Not all evaluation tools support all currently available Web technologies (such as HTML, XHTML, CSS, SVG, MathML, etc.), and they do not support them all equally well.
User Interface: How suitable and user friendly is the interface of the evaluation tool?
Regardless which type of user interface the evaluation tool employs (for example report, wizard, visual feedback, or manual as described above), the tool should be intuitive and easy to understand.

Last modified: $Date: 2005/02/24 10:48:17 $ by $Author: shadi $

Note: This draft WAI Resource developed by W3C/WAI's Education and Outreach Working Group (EOWG). We invite review and discussion. Please address your feedback to wai-eo-editors@w3.org, a mailing list with a public archive. Change log available.

Last updated 1 February 2005 by Shadi Abou-Zahra. Editors: Shadi Abou-Zahra and Judy Brewer, with assistance from participants of the EOWG.

Copyright © 2005 W3C (MIT, ERCIM, Keio ), All Rights Reserved. W3C liability, trademark, document useand software licensingrules apply. Your interactions with this site are in accordance with our public and Member privacy statements.