W3C logoWeb Accessibility Initiative (WAI) logo

WAI: Strategies, guidelines, resources to make the Web accessible to people with disabilities

[EARLY ROUGH ROUGH DRAFT] Including Users in Web Accessibility Evaluation

Page Contents

Note: This document is a draft and should not be referenced or quoted under any circumstances. [requirements & changelog | early concept draft]

Introduction

Web accessibility evaluation has often been limited to assessing conformance to accessibility standards, such as the Web Content Accessibility Guidelines (WCAG). Broadening evaluation to include people with disabilities helps ensure that technical accessibility solutions are applied effectively.

For example, take a Web developer who has never heard of a screen reader. In trying to meet the Web accessibility guideline: "Provide text alternatives for all non-text content", he might code: alt="This image is a line art drawing of a dark green magnifying glass. If you click on it, it will take you to the Search page." Just one time watching a user listen to the site with a screen reader will clearly show the developer that his alt text is ineffective and alt="search" is all that is needed.

Including people who are target "users" of your Web site throughout Web site development helps your Web developers implement accessibility more effectively, thus maximizing your investment in accessibility. When development teams, managers, and other stakeholders have an opportunity to watch users with disabilities use their site, the results are almost always a powerful increase in understanding of basic user issues and accessibility solutions, and motivation for improving accessibility.

While this document focuses on including users with disabilities in evaluation, they really should be included throughout the development process, for example, integrated into a user-centered design (UCD) process.

A comprehensive evaluationof a Web site combines evaluation expertise to assess technical conformance and the experience of people with disabilities in usability testing for accessibility issues. This document introduces including users in Web accessibility evaluation, provides guidance on some considerations for including users, and lists additional information.

Range of User Involvement

Users with disabilities can be included in a wide range of evaluation activities, from limited informal evaluation focusing specifically on accessibility issues to large-scale formal usability testing. There are many options in between these extremes:

Few organizations have the resources to employ experts in usability testing. However, formal usability testing not required in most cases. Significant benefit can be realized by including users with disabilities without much investment in formal testing. Below are some tips. Learn more about doing more, too!

Include Diverse Users

People with disabilities are as diverse as any people. They have different experiences, different expectations, and different preferences. They use different interaction techniques, different adaptive strategies, and different assistive technology configurations. People have different disabilities: visual, auditory, physical, speech, cognitive, and neurological disabilities. Even within one category, there is extreme variation; for example, "visual disability" includes people who been totally blind since birth, people who have distortion in their central vision from age-related degeneration, and people who temporarily have blurry vision from an injury or disease.

Include as broad a variety of users with different disabilities as you can, and avoid the pitfall of only including people who are blind. In most cases evaluators have limited time and budget for evaluation and cannot include many users in evaluation. There are resources on the Web that provide additional guidance on "determining participant characteristics" for users with disabilities in accessibility evaluation.

When you are only able to have a small number of users, it is especially important to carefully consider all feedback and avoid assuming that feedback from one person with a disability applies to all people with disabilities. A person with a disability does not necessarily know how other people with the same disability interact with the Web, nor know enough about other disabilities to provide valid guidance. Don't make accessibility design decisions based only on the recommendations of one person with a disability. What works for one person might not work for everyone with that disability or other disabilities. Be careful not to assume that all users, including users with disabilities, use your product the same way.

Users' Experience Interacting with the Web

A primary consideration in choosing users to help with evaluation is their experience interacting with the Web. Some people with disabilities use assistive technologies (AT) such as screen readers that read aloud the Web page, screen magnifiers, and different pointing devices instead of mice. Some AT is complicated and difficult to learn. A user with insufficient experience may not know how to use their AT effectively. Problems identified may be due to their lack of knowledge of the AT (for example, not knowing how to read data tables or navigate through headings using a screen reader), not problems with the Web site being evaluated. On the other hand, a very advanced user might know uncommon work-arounds to overcome problems that would be impossible for the average user to handle.

Just as with any evaluation with users, whether you include novice users, "average" users, or advanced users depends on your target users. For example, If you are developing a Web application to be used by accountants inside a company, you probably want advanced AT users; for a public Web site to apply for disability benefits, you want novice AT users.

Understanding [Findings/Results/Issues]
@@ [Placing the Blame on the Correct Component :]

Web accessibility depends on several components of Web development and interaction working together, including Web browsers, AT, and Web content. Users are likely to identify accessibility issues in different components; for example, when evaluating a Web site, the user might find an accessibility problem with the AT. Here are some points related to identifying and addressing issues:

Additionally, users with disabilities will almost certainly find general usability problems that impact all users, including users without disabilities. When the evaluation is being used to improve a Web site, it is rarely necessary to distinguish between usability and accessibility issues. However, when the evaluation data will be used to make public statements about accessibility, it can be vital to distinguish between usability and accessibility issues.

Drawing Conclusions and Reporting

It is very important that any reporting of accessibility evaluations clearly indicate the evaluation parameters, such as the methodology and the user characteristics. The report should clearly indicate what it asserts and what it does not assert, especially when people who are not familiar with accessibility evaluation will likely read the report. For example, successful user evaluation with a limited number of users does not guarantee that the Web site is accessible to all people with disabilities, nor that it conforms to specific guidelines.

For More Information

This document just touches on a few points on topics that are vast. There is lots more information available out there on all aspects of including users in evaluation.

The other pages in the Evaluating Web Sites for Accessibility resource suite provide related information; for example, before evaluating with users, conduct a preliminary evaluation to make sure that there are no serious accessibility problems that will prevent users with disabilities from using the Web site. [Other WAI resources that provide relevant information including How People with Disabilities Use the Web.]

Additional resources are available on the Web on topics such as: