W3C logoWeb Accessibility Initiative (WAI) logo

WAI: Strategies, guidelines, resources to make the Web accessible to people with disabilities

[DRAFT] Involving Users in Web Accessibility Evaluation

Page Contents

Note: This document is a draft and should not be referenced or quoted under any circumstances. [requirements & changelog | early concept draft]

Introduction

Web accessibility evaluation often focuses on assessing conformance to accessibility standards, such as the Web Content Accessibility Guidelines (WCAG). Broadening evaluation to involve people with disabilities helps better understand accessibility issues and implement more effective accessibility solutions.

For example, take a Web developer who does not know what it is like to use a screen reader. To meet the Web accessibility guideline "Provide text alternatives for all non-text content", the developer might code: alt="This image is a line art drawing of a dark green magnifying glass. If you click on it, it will take you to the Search page." However, observing a person use the site with a screen reader will clearly show the developer that the alt text is ineffective and alt="search" is all that is needed.

When Web developers, managers, and other project stakeholders see people with disabilities use their Web site, most are highly motivated by a new understanding of accessibility issues. Collaborating with people with disabilities who are target "users" of a Web site from early on in a project helps Web developers be more efficient in addressing accessibility, thus maximizing the results from investment in accessibility.

While involving users with disabilities in evaluation has many benefits, it alone cannot determine if a Web site is accessible. It is an integral part of a comprehensive evaluation of Web accessibility that includes:

This document introduces involving users in Web accessibility evaluation, provides guidance on some considerations for involving users, and links to additional information. It is part of a multi-page Evaluating Web Accessibility resource suite that outlines different approaches for evaluating Web accessibility.

Optimizing User Involvement

A first step in evaluating Web accessibility is conducting a preliminary review of the Web site to check for any obvious accessibility problems. This allows you to fix any significant barriers before spending time evaluating with users with disabilities.

Users with disabilities can be included in a wide range of evaluation activities, from brief consultations to large-scale usability studies. There are many options in between these extremes:

Conducting informal evaluations throughout development is more effective than formal usability testing at the end of a project. In most cases, including users in evaluation involves finding a few people with disabilities and including them throughout the development process to complete sample tasks on developing prototypes and discuss accessibility issues and solutions.

It is especially important when you are able to involve only a few users to carefully consider all feedback and avoid assuming that feedback from one person with a disability applies to all people with disabilities. A person with a disability does not necessarily know how other people with the same disability interact with the Web, nor know enough about other disabilities to provide valid guidance on other accessibility issues.

Including Diverse Users

People with disabilities are as diverse as any people. They have different experiences, different expectations, and different preferences. They use different interaction techniques, different adaptive strategies, and different assistive technology configurations. People have different disabilities: visual, auditory, physical, speech, cognitive, and neurological disabilities — and some people have multiple disabilities. Even within one category, there is extreme variation; for example, "visual disability" includes people who have been totally blind since birth, people who have distortion in their central vision from age-related degeneration, and people who temporarily have blurry vision from an injury or disease.

Includes users with a variety of different disabilities and user characteristics. Selecting an optimum number of users with the best suited characteristics is important, challenging, and different based on the situation. In most cases evaluators have limited time and budget and cannot include many users in evaluation. There are resources on the Web that provide guidance on determining participant characteristics for a particular situation and on finding participants with disabilities.

Users' Experience Interacting with the Web

A primary consideration in selecting users to help with evaluation is their experience interacting with the Web. For example, some assistive technologies (AT) are complicated and difficult to learn. A user with insufficient experience may not know how to use their AT effectively. Problems identified may be due to the user's lack of knowledge of the AT, not problems with the Web site being evaluated. On the other hand, a very advanced user might know uncommon work-arounds to overcome problems in the site that the "average" user would not be able to handle.

Just as with any evaluation with users, whether you include novice users, average users, or advanced users depends on your target users. For example, If you are developing a Web application to be used by accountants inside a company, you probably want advanced AT users; for a public Web site to apply for disability benefits, you want novice AT users.

Diagnosing Accessibility Problems

Web accessibility depends on several components of Web development and interaction working together, including Web browsers, assistive technologies, and Web content. For any accessibility problems you identify, determine which components are responsible. For example, if a user has trouble with a data table, it could be because:

In addition to revealing accessibility problems, evaluating with users with disabilities usually reveals general usability problems that impact all users, including users without disabilities.

Drawing Conclusions and Reporting

As explained in the introduction, involving users with disabilities makes accessibility efforts more effective and more efficient. However, it alone cannot determine if a Web site is accessible; even large-scale usability studies cannot cover the diversity of disabilities, individuals, and situations. Combining user involvement with evaluating conformance to WCAG ensures that the broad range of accessibility issues are covered.

Reports should include the scope of the study and the evaluation parameters, such as the testing methods and the user characteristics. For example, if a study included only usability testing with participants who are blind, its report should clarify that it did not evaluate conformance to accessibility guidelines and it does not apply to all people with disabilities, in order to help readers draw appropriate conclusions.

For More Information

This document briefly addresses a few points of a very complex topic. Many resources on other aspects of involving users in evaluation are available on the Web.

Terminology and Notes

user characteristics
User characteristics typcally include things like age, job responsibilities, software, hardware, environment (for example, home, shared office, private office, shared public terminal), computer experience, and Web experience.User characteristics can also include type of disability, adaptive strategies ued, and experience with specific assisitive technologies.
adaptive strategies
Adaptive strategies are techniques that people with disabilities use to improve interaction with the Web, such as increasing the font size in a common browser. Adaptive strategies include techniques with mainstream browsers or with assistive technologies.
assistive technologies
Assistive technologies are software or equipment that is used by people with disabilities to improve interaction with the Web, such as screen readers that read aloud Web pages for people who cannot see or read text, and voice recognition software and selection switches for people who cannot use a keyboard or mouse.
Web content
Web "content" generally refers to the information in a Web page or Web application, including text, images, forms, sounds, and such. More specific definitions are available in the WCAG documents, which are linked from the Web Content Accessibility Guidelines (WCAG) Overview.

Optimizing Usability Testing for Accessibility Issues

Note for usability professionals: When defining usability tests specifically to find accessibility issues, the protocol will be different from typical general usability test; for example: