W3CWeb Accessibility initiative

WAI: Strategies, guidelines, resources to make the Web accessible to people with disabilities

Editors Draft: $Date: 2009/12/04 20:35:16 $ [changelog]
Status: This document is a draft revision and should not be referenced or quoted under any circumstances. The published version this document is at: www.w3.org/WAI/eval/users.html. Please send comments to wai-eo-editors@w3.org (a publicly archived list).

[Draft Revision] Involving Users in Evaluating Web Accessibility

Page Contents


Web accessibility evaluation often focuses on conformance to accessibility standards such as WCAG. While conformance is important, there are many benefits to evaluating with real people to learn how your website or web tool really works for users and to better understand accessibility issues. Evaluating with users with disabilities and with older users identifies usability issues that are not discovered by conformance evaluation alone.

This page is part of a multi-page Evaluating Web Accessibility resource suite that outlines different approaches for evaluating web accessibility. It is the second of two pages on including users in web projects; please read Including Users in Web Projects for Better, Easier Accessibility, which covers broader issues of including users early in website design, tool development, standards, and other web projects.

Initial Review

A first step in evaluating web accessibility is conducting a preliminary review of the website to check for any obvious accessibility problems.

Even web developers with little accessibility knowledge can find some accessibility issues through a preliminary review. An accessibility expert with first-hand experience of how people with different disabilities interact with the web can:

The initial review identifies any significant accessibility barriers to fix before evaluating with users. It also helps define what to focus on for evaluation with users.

Range of User Evaluation

Users with disabilities and older users can be included in a wide range of evaluation activities, from brief consultations to large-scale usability studies. There are many options in between these extremes:

What type of evaluation you do depends on factors such as the stage in your project, for example, initial investigation of design ideas, testing specific areas of prototypes, or reviewing final designs.

Conducting informal evaluations throughout development is more effective than only formal usability testing at the end of a project.


In most cases, including users in evaluation involves:

See Including Users in Web Projects for Better, Easier Accessibility for guidance on getting a range of users, users' experience interacting with the web, and analyzing accessibility issues.

Just as with any evaluation with users, whether you include novice, average, or advanced users depends on your target users. For example, if you are developing a web application to be used by accountants inside a company, you probably want advanced assistive technology users; for a public website to apply for disability benefits, you want novice assistive technology users.

Caution: Carefully consider all feedback and avoid assuming that feedback from one person with a disability applies to all people with disabilities. A person with a disability does not necessarily know how other people with the same disability interact with the web, nor know enough about other disabilities to provide valid guidance on other accessibility issues.

Note: In addition to finding accessibility problems, evaluating with users with disabilities usually reveals general usability problems that impact all users, including those without disabilities.

Analyzing Accessibility Issues

Web accessibility depends on several components of web development and interaction working together, including web browsers, assistive technologies (AT), and web content. For any accessibility problems you identify, determine which components are responsible. For example, if a user who cannot use a mouse has trouble with keyboard access, it could be because:

Combine User Evaluation with Standards

Involving users with disabilities in evaluation has many benefits; however, it alone cannot determine if a website is accessible. Combine user involvement with evaluating conformance to WCAG to ensure that accessibility is provided to users with a range of disabilities and situations.

Drawing Conclusions and Reporting

Be careful drawing conclusions from limited evaluations or studies. See the Caution above.

Reports should include the scope of the study and the evaluation parameters, such as the testing methods and the user characteristics. For example, if a study included only usability testing with participants who are blind, its report should clarify that it did not evaluate conformance to accessibility guidelines and that it does not apply to all people with disabilities. Thus the report can help readers draw appropriate conclusions.

While small studies often provide useful information, they are not robust enough to provide statistical significance.

Note for usability professionals

When defining usability tests specifically to find accessibility issues, the protocol will usually be different from a typical general usability test; for example:

The For More Information section below provides additional guidance specifically for usability professionals.

More Information and Guidance

This document briefly addresses a few points of a very complex topic. Many resources on other aspects of involving users in evaluation are available on the Web.