W3C logoWeb Accessibility Initiative (WAI)         logo

WAI: Strategies, guidelines, resources to make the Web accessible to people with disabilities

[DRAFT] Involving Users in Web Accessibility Evaluation

Page Contents

Introduction

Web accessibility evaluation often focuses on evaluating conformance to accessibility standards, such as the Web Content Accessibility Guidelines (WCAG). Broadening evaluation to involve people with disabilities helps better understand accessibility issues and implement more effective accessibility solutions.

One of the benefits of including people with disabilities is that Web developers can learn how people with disabilities interact with the Web and with assistive technologies. For example, consider a developer who does not know what it is like to use a screen reader. To meet the Web accessibility guideline "Provide text alternatives for all non-text content", the developer might provide the alternative text: "This image is a line art drawing of a dark green magnifying glass. If you click on it, it will take you to the Search page." However, observing a person use the site with a screen reader will clearly show the developer that his alternative text is ineffective and "search" is all that is needed.

When Web developers, managers, and other project stakeholders see people with disabilities use their Web site, most are highly motivated by a new understanding of accessibility issues. Collaborating with people with disabilities who are target "users" of a Web site early in a project helps Web developers be more efficient in addressing accessibility, thus maximizing the results from investment in accessibility.

Involving users with disabilities in evaluation has many benefits; however, it alone cannot determine if a Web site is accessible. Combine user involvement with evaluating conformance to WCAG to ensure that accessibility is provided to users with a range of disabilities and situations.

This document introduces involving users in Web accessibility evaluation, provides guidance on some considerations for involving users, and links to additional information. It is part of a multi-page Evaluating Web Accessibility resource suite that outlines different approaches for evaluating Web accessibility.

Involving Users Effectively

A first step in evaluating Web accessibility is conducting a preliminary review of the Web site to check for any obvious accessibility problems. This allows you to fix any significant accessibility barriers before spending time evaluating with users.

Even Web developers with little accessibility knowledge can find some accessibility issues through a preliminary review. An accessibility expert with first-hand experience of how people with different disabilities interact with the Web can:

Users with disabilities can be included in a wide range of evaluation activities, from brief consultations to large-scale usability studies. There are many options in between these extremes:

Conducting informal evaluations throughout development is more effective than formal usability testing at the end of a project. In most cases, including users in evaluation involves:

Carefully consider all feedback and avoid assuming that feedback from one person with a disability applies to all people with disabilities. A person with a disability does not necessarily know how other people with the same disability interact with the Web, nor know enough about other disabilities to provide valid guidance on other accessibility issues.

Including Diverse Users

People with disabilities are as diverse as any people. They have diverse experiences, expectations, and preferences. They use diverse interaction techniques, adaptive strategies, and assistive technology configurations. People have different disabilities: visual, auditory, physical, speech, cognitive, and neurological — and some have multiple disabilities. Even within one category, there is extreme variation; for example, "visual disability" includes people who have been totally blind since birth, people who have distortion in their central vision from age-related degeneration, and people who temporarily have blurry vision from an injury or disease.

Include users with a variety of disabilities and user characteristics. In most cases evaluators have limited time and budget and cannot include many users in evaluation. Selecting the optimum number of users with the best suited characteristics can be difficult. There are resources on the Web that provide guidance on determining participant characteristics for a particular situation and on finding participants with disabilities.

Users' Experience Interacting with the Web

A primary consideration in selecting users to help with evaluation is their experience interacting with the Web. For example, some assistive technologies (AT) are complicated and difficult to learn. A user with insufficient experience may not know how to use the AT effectively. Problems identified may be due to the user's lack of knowledge of the AT, not problems with the Web site being evaluated. On the other hand, a very advanced user might know uncommon work-arounds to overcome problems in the site that the "average" user would not be able to handle.

Just as with any evaluation with users, whether you include novice, average, or advanced users depends on your target users. For example, If you are developing a Web application to be used by accountants inside a company, you probably want advanced AT users; for a public Web site to apply for disability benefits, you want novice AT users.

Analyzing Accessibility Problems

Web accessibility depends on several components of Web development and interaction working together, including Web browsers, assistive technologies, and Web content. For any accessibility problems you identify, determine which components are responsible. For example, if a user has trouble with a data table, it could be because:

In addition to finding accessibility problems, evaluating with users with disabilities usually reveals general usability problems that impact all users, including those without disabilities.

Drawing Conclusions and Reporting

As explained in the introduction, involving users with disabilities makes accessibility efforts more effective and more efficient. However, it alone cannot determine if a Web site is accessible; even large-scale usability studies cannot cover the diversity of disabilities, individuals, and situations. Combining user involvement with evaluating conformance to WCAG ensures that the broad range of accessibility issues are covered.

Reports should include the scope of the study and the evaluation parameters, such as the testing methods and the user characteristics. For example, if a study included only usability testing with participants who are blind, its report should clarify that it did not evaluate conformance to accessibility guidelines and that it does not apply to all people with disabilities. Thus the report can help readers draw appropriate conclusions.

For More Information

This document briefly addresses a few points of a very complex topic. Many resources on other aspects of involving users in evaluation are available on the Web.

Terminology and Notes

adaptive strategies
Adaptive strategies are techniques that people with disabilities use to improve interaction with the Web, such as increasing the font size in a common browser. Adaptive strategies include techniques with mainstream browsers or with assistive technologies.
assistive technologies
Assistive technologies are software or equipment that people with disabilities use to improve interaction with the Web, such as screen readers that read aloud Web pages for people who cannot see or read text, and voice recognition software and selection switches for people who cannot use a keyboard or mouse.
user characteristics
User characteristics typically include things like age, job responsibilities, software, hardware, environment (for example, home, shared office, private office, shared public terminal), computer experience, and Web experience. User characteristics can also include type of disability, adaptive strategies used, and experience with specific assistive technologies.
Web content
Web "content" generally refers to the information in a Web page or Web application, including text, images, forms, sounds, and such. More specific definitions are available in the WCAG documents, which are linked from the Web Content Accessibility Guidelines (WCAG) Overview.

Optimizing Usability Testing for Accessibility Issues

Note for usability professionals: When defining usability tests specifically to find accessibility issues, the protocol will be different from a typical general usability test; for example: