Editors Draft: $Date: 2009/10/26 22:46:03 $ [changelog]
Status: This document is a draft and should not be referenced or quoted under any circumstances. The published version of this document's predecessor is at www.w3.org/WAI/eval/users.html. Please send comments to wai-eo-editors@w3.org (a publicly archived list).

[Draft] Including Users for Better, Easier Accessibility

Introduction

[@@more clearly that it's always good to include users but usually not done for accessibility] A key aspect of designing websites [[@@ products and services here and later can include standards develoeprs, website developers, browser developers @@]] for usability is including the perspectives of target users throughout project development. However, when designing for accessibility, developers (designers, project managers, etc.) often use only accessibility standards or guidelines and don't involve users at all, or only at the end of the development process for testing. Involving people with disabilities from the beginning helps you better understand accessibility issues and implement more effective accessibility solutions.

This page gets you started reaping the benefits of involving users with accessibility needs, including people with disabilities and older people, throughout your web development projects. When you're at the stage of evaluating accessibility, see also Involving Users in Web Accessibility Evaluation.

How Involving Users Early Helps

Including users in the development process helps you more efficiently develop accessible websites that work well for people with accessibility needs, thus maximizing your return on any investment (ROI) in accessibility.

e.g.,

As an example, consider a developer who does not know what it's like to use a screen reader. To meet the web accessibility guideline "Provide text alternatives for all non-text content", the developer might provide alt text such as: "This image is a line art drawing of a dark green magnifying glass. If you click on it, it will take you to the Search page." Such alt text is going to be a lot of work for each image. Now, consider another developer who involved users in her project early and has observed people using the web with screen readers. This developer knows that for such images, the only alt needed is "Search". She gets the job done quicker, and better for users.

The first developers' time has been wasted and the result is annoying for users. And, once the problem is discovered (at testing end or after rollout when users complain), it's going to take more time, effort, and money to go back and fix it.

When you involve users with accessibility needs early in the project, you get:

Efficiency: You can more quickly develop accessibility solutions, and spend less time guessing and having to go back and fix problems.
Effectiveness: The better you understand the issues, the better you can implement more effective accessibility solutions; that is, your website will work better and be more usable for people with disabilities, older users, and others. Also, making websites more effective for people with a range of disabilities improves general usability for everybody.
Motivation: When product designers, managers, and other project stakeholders see people with disabilities use their website, most are highly motivated by a new understanding of accessibility issues.

Learning how real people use your website is powerful. However, you can't include enough users to cover all issues. That's where accessibility guidelines come in. Combine user involvement with learning WCAG and evaluating conformance to ensure that accessibility is provided to users with a range of disabilities and situations.

How to Involve Users

If you are already using user-centered design processes (UCD) or other usability methodologies and techniques, simply ensure that your usability goals, user analysis, personas, scenarios, workflows, etc. include people with disabilities and older users.

If you aren't using UCD, you can still include people with accessibility needs in your design and development. As early as possible in your project:

Learn the basics of how people with disabilities use the web by reading online resources and watching videos.
Find people with disabilities, with a range of characteristics. (more on this below)
@@ including considerations in user cases, etc. @@
Ask them to show you websites that work well for them. Then, ask them to show you problems in websites that do not work well. Ask lots of questions to help you understand their accessibility issues.
When you are considering a design aspect, such as expanding navigation, find sites on the web that are already doing it and have users explore with you what works well and what does not.
Throughout design, ask users to review prototypes. Give them specific tasks to complete and see how the different aspects of the design and coding could be improved. For more in this, see Involving Users in Web Accessibility Evaluation. Ask lots of questions.

Caution: Carefully consider all feedback and avoid assuming that feedback from one person with a disability applies to all people with disabilities. A person with a disability does not necessarily know how other people with the same disability interact with the Web, nor know enough about other disabilities to provide valid guidance on other accessibility issues.

Getting a Range of Users

People with disabilities are as diverse as any people. They have diverse experiences, expectations, and preferences. They use diverse interaction techniques, adaptive strategies, and assistive technology configurations. People have different disabilities: visual, auditory, physical, speech, cognitive, and neurological — and some have multiple disabilities. Even within one category, there is extreme variation; for example, "visual disability" includes people who have been totally blind since birth, people who have distortion in their central vision from age-related degeneration, and people who temporarily have blurry vision from an injury or disease.

Include users with a variety of disabilities and user characteristics. In most cases evaluators have limited time and budget and cannot include many users in evaluation. Selecting the optimum number of users with the best suited characteristics can be difficult. There are resources on the Web that provide guidance on determining participant characteristics for a particular situation and on finding participants with disabilities.

Users' Experience Interacting with the Web

A primary consideration in selecting users is their experience interacting with the Web. For example, some assistive technologies (AT) are complicated and difficult to learn. A user with insufficient experience may not know how to use the AT effectively. Problems identified may be due to the user's lack of knowledge of the AT, not problems with the website being evaluated. On the other hand, a very advanced user might know uncommon work-arounds to overcome problems in the site that the "average" user would not be able to handle.

In the early stages, especially when you are first learning how people with disabilities and older users interact with the web, it is usually best to get people with a fairly high experience level.

When you get to later evaluation phases, whether you include novice, average, or advanced users depends on your target users. For example, if you are developing a Web application to be used by accountants inside a company, you probably want advanced AT users; for a public website to apply for disability benefits, you want novice AT users.

Analyzing Accessibility Issues

Web accessibility depends on several components of Web development and interaction working together, including Web browsers, assistive technologies, and Web content. For any accessibility problems you identify, determine which components are responsible. For example, if a user has trouble with a data table, it could be because:

the developer did not markup/code the data table properly, or
the user's AT does not facilitate reading data tables effectively, or
the user does not know how to use the ATs table reading feature.
@@ browser, too ?

In addition to finding accessibility problems, evaluating with users with disabilities usually reveals general usability problems that impact all users, including those without disabilities.

Evaluating with Users

A first step in evaluating Web accessibility is conducting a preliminary review of the Web site to check for any obvious accessibility problems. This allows you to fix any significant accessibility barriers before spending time evaluating with users.

Even Web developers with little accessibility knowledge can find some accessibility issues through a preliminary review. An accessibility expert with first-hand experience of how people with different disabilities interact with the Web can:

evaluate accessibility issues for a broad range of users, which might not be found by individual users;
help fix any known barriers before bringing in users; and
focus the evaluation with users on potential areas of concern.

Users with disabilities can be included in a wide range of evaluation activities, from brief consultations to large-scale usability studies. There are many options in between these extremes:

Informal evaluation of a specific accessibility issue can be as simple as asking a person in your office who uses a screen reader to find some data in an early draft of a data table that you are developing, observing her interaction, and discussing issues.
Formal usability testing of a Web site follows established protocols to gather quantitative and qualitative data from representative users performing specific tasks. Formal usability tests can be optimized to focus on accessibility issues.

Conducting informal evaluations throughout development is more effective than formal usability testing at the end of a project. In most cases, including users in evaluation involves:

finding a few people with disabilities,
including them throughout the development process to complete sample tasks on prototypes, and
discussing accessibility issues with them.

Remember: Carefully consider all feedback and avoid assuming that feedback from one person with a disability applies to all people with disabilities. A person with a disability does not necessarily know how other people with the same disability interact with the Web, nor know enough about other disabilities to provide valid guidance on other accessibility issues.

Drawing Conclusions and Reporting

As explained in the introduction, involving users with disabilities makes accessibility efforts more effective and more efficient. However, it alone cannot determine if a website is accessible; even large-scale usability studies cannot cover the diversity of disabilities, adaptive strategies, and assistive technologies. Combining user involvement with evaluating conformance to WCAG ensures that the broad range of accessibility issues are covered.

Evaluation reports should include the scope of the study and the evaluation parameters, such as the testing methods and the user characteristics. For example, if a study included only usability testing with participants who are blind, its report should clarify that it did not evaluate conformance to accessibility guidelines and that it does not apply to all people with disabilities. Thus the report can help readers draw appropriate conclusions.

For More Information

This document briefly addresses a few points of a very complex topic. Many resources on other aspects of involving users throughout design are available on the Web.

[Relationship between Web Accessibility and Usability@@]
@@ Consider which (or all) of the existing references should be included
@@ Consider if any of the external references should be included

Terminology and Notes

@@ point to definitions in a central location and delete here? or don't make them jump to a new page?

adaptive strategies: Adaptive strategies are techniques that people with disabilities use to improve interaction with the Web, such as increasing the font size in a common browser. Adaptive strategies include techniques with mainstream browsers or with assistive technologies.
assistive technologies: Assistive technologies are software or equipment that people with disabilities use to improve interaction with the Web, such as screen readers that read aloud Web pages for people who cannot read text, screen magnifiers for people with some types of low vision, and voice recognition software and selection switches for people who cannot use a keyboard or mouse.
user characteristics: User characteristics typically include things like age, job responsibilities, software, hardware, environment (for example, home, shared office, private office, shared public terminal), computer experience, and Web experience.User characteristics can also include type of disability, adaptive strategies used, and experience with specific assistive technologies.
Web content: Web "content" generally refers to the information in a Web page or Web application, including text, images, forms, sounds, and such. More specific definitions are available in the WCAG documents, which are linked from the Web Content Accessibility Guidelines (WCAG) Overview.

Optimizing Usability Testing for Accessibility Issues

Note for usability professionals: When defining usability tests specifically to find accessibility issues, the protocol will be different from a typical general usability test; for example:

you would likely use a think-out-loud technique with high facilitator interaction;
data collection would focus on understanding errors related to accessibility issues, rather than on time-on-task or user satisfaction; and
tasks would concentrate on specific areas of concern for potential accessibility problems, rather than general site usage.

@@ do we want to provide more advice specifically for usability professionals? - and make this an <h2> ?

Site Navigation