This is an old draft. The published version of this document is at www.w3.org/WAI/bcase/.
Evaluating Web Sites for Accessibility
This is an old draft. The published version of this document is at www.w3.org/WAI/bcase/.
Conformance Evaluation to WCAG 1.0
Considerations for specific contexts:
This document outlines approaches for preliminary review Web site accessibility,
and for evaluation of conformance to the Web Content
Accessibility Guidelines 1.0. It includes tips for evaluation during
development of Web sites, and for monitoring of established Web sites. The
measures described here are intended to supplement an organization's existing
procedures for content management and quality assurance on their Web sites.
There are a variety of tools and approaches for evaluating Web site
accessibility. No single evaluation tool yet provides comprehensive information
or captures all problems with regard to the accessibility of a site; therefore
evaluation involves a combination of approaches. Goals for evaluating Web
sites vary, and require different approaches to meet those goals:
Preliminary review can:
identify general kinds of barriers on a Web site.
Conformance evaluation can:
catch major problems during development phase of a new site;
determine the WCAG 1.0 conformance level for an existing Web site;
demonstrate that a Web site meets a given WCAG 1.0 conformance level.
Conformance evaluation, plus review of procedures for
ongoing monitoring, can:
help ensure that a site will maintain a given conformance level in the future.
A preliminary review may help to quickly identify the scope of problems on
a Web site. However, the preliminary review will not catch all of the problems
on a site and should not be used to determine conformance level. A preliminary
review does not include perspectives from a variety of users with disabilities
nor does it touch or test every aspect of a site.
A preliminary review combines some manual checking of representative pages
on a Web site, along with the use of several semi-automatic accessibility
checkers. Reviewers do not need to know Web mark-up languages, but should
be able to download software and familiarize themselves with some online
tools, and change certain settings on their browser.
To conduct a preliminary review, complete all five steps below.
Select a representative sampling of different kinds of pages from the Web
site to be reviewed; must include entry page(s) ("welcome page" etc.)
Use a graphical user interface (GUI) browser
(such as Internet
Navigator, or Opera) and examine
the selection of pages while adjusting the browser settings as follows (NOTE:
For reviewers who have disabilities, certain of the following steps may need
to be done with another person who does not have the same disability.)
turn off images, and make sure that the information is presented in an
appropriate sequence relative to the visual presentation on the GUI site.
turn off the sound, and make sure audio content is still available through
change the font size (larger and smaller) in the browser, and observe whether
the page is still readable.
set screen resolution to 640 x 480 and observe whether or not this forces
the page into horizontal scrolling
change the display color to black and white (or print out page on black and
white printer) and observe whether color contrast is adequate.
put away the mouse and tab through the links and form controls on a page,
making sure that you can access all links and form controls, and that the
links clearly indicate what they lead to.
voice browser (such
as Home Page Reader) or
a text browser (such as Lynx) and
examine the Web site while answering these questions (NOTE: experienced users
of screen readers may substitute a screen reader for a voice or text browser,
but if blind, may need a sighted partner to compare information available
visually; if sighted, listen to it with eyes closed, then open eyes and confirm
whether the information is equivalent)
is equivalent information available
through the voice or text browser as is available through the GUI browser?
is the information presented in a similar logical order as when viewed through
the GUI browser?
evaluation tools and note any problems indicated by the tools, for example:
online accessibility assessment tool that flags any items on a Web page which
should be examined for potential accessibility problems, and provides a
description of what the problem might be)
Bobby (an online or downloadable
accessibility checker which provides a semi-automated assessment of accessibility
problems on a Web page or group of Web pages; it can identify many problems
on sites, and lists problems which it is not able to evaluate automatically
and which require manual review)
A-Prompt (a tool which identifies
potential accessibility problems and provides guided editing to correct the
summarize the types of problems encountered, as well as best practices that
should be continued or expanded on the site
indicate the method by which problems were identified, and clearly state
that this was not a full conformance evaluation
recommend follow-up steps, including full conformance evaluation which includes
validation of markup and other tests, and ways to address any problems
A comprehensive evaluation combines semi-automatic, manual, and usability
testing. Comprehensive evaluations require familiarity with Web mark-up
languages; initial downloading and/or training on a variety of evaluation
tools and approaches; configuration of browser settings; and coordination
with reviewers with a variety of disabilities. Evaluation with users is important
as it helps to identify problems in how the technical solutions are being
A properly conducted comprehensive evaluation can identify potentially major
problems during the development phase for a new site; determine what level
of accessibility a Web site meets; and/or provide assurance that a Web site
meets a required level of accessibility.
A comprehensive evaluation includes all of the steps below except those that
are explicitly identified as alternatives or optional:
Identify scope of site to be evaluated, and the targeted conformance level
for the evaluation:
identify a "page selection" which includes at least one
of each different type of page on the site, and all top pages or entry pages
to the site.
identify and clearly disclose either:
the "entire Web site" including all pages at a base URL;
or an "expanded page selection," to be clearly explained
and disclosed on the Web site. Suggestions for inclusions in this expanded
page selection: pages from different sections of the Web site; pages representing
different "look & feel"; pages representing different development tools
and processes including those generated from databases; pages produced under
different guidelines; "contact us" pages; pages critical to your business;
etc. If any area of a site is excluded from evaluation, be sure to disclose
identify the target conformance level of
Semi-automatic and automatic evaluation
Validate markup including syntax and style sheets, using all applicable
validators, on page selection. Run at least one validation tool
across entire Web site or expanded page selection.
Use at least two accessibility evaluation tools, on page selection,
and run at least one tool across entire Web site.
Examine page selection using relevant level checkpoints from the
Checkpoints for Web Content Accessibility Guidelines 1.0.
Examine page selection with graphical user interface
(GUI) browsers: select at least three different configurations from among
the following variables: different graphical user interface browsers (Internet
Explorer, Netscape, Opera), in different versions (latest, older), running
on different platforms (Windows, Linux, Mac) and making the following
use all approaches listed under #2 in preliminary review
also examine page with scripts, style sheets, and applets not loaded
Examine page selection with one text browser (such as
Lynx) AND one voice browser (such
as Home Page Reader), and
answer the following questions. (NOTE: For settings where there is limited
choice of assistive technologies, also perform a manual evaluation of
the Web site with those assistive technologies; for instance, JAWS is the
only screen reader translated into Danish, and therefore in Denmark, a trained
evaluator should evaluate the Web site using JAWS.)
use both questions listed under #3 in preliminary review
is the text clear and simple to the extent appropriate for the purpose of
the Web site? (For English sites, consider using
and Appropriate Language and Design (CLAD) test.)
Have people with different disabilities, different levels of technical expertise,
and different levels of familiarity with the site, using a variety of assistive
technologies and adaptive strategies, review page selection and
explore freely across entire Web site or expanded page selection.
[NOTE: Sometimes this is done in the context of a testing laboratory,
but it is also valuable for people to evaluate the site in their typical
Web environment. Different levels of user technical experience and familiarity
with the site are important, and note that these will change over time, as
people participate in reviewing the site.]
Ask testers to try to find answers to the most common questions for which
people visit the Web site. Note areas where it is difficult or impossible
the use the Web site.
Summarize and follow-up
Summarize any problems and best practices identified for each page type and
a representative URL, and method by which they were identified
Recommend follow-up steps, potentially including:
repair of accessibility barriers identified through conformance evaluation
process. NOTE: for evaluations using "expanded page selection" instead of
"entire Web site," apply what you've learned to other pages
expanding best practices on site
ongoing maintenance and monitoring of site
Evaluation during the development process is essential. It can sometimes
be difficult, as both in-house and subcontracted Web developers sometimes
prefer to establish the site design and demonstrate their progress before
getting feedback. However, accessibility issues identified early are easier
to correct and avoid. Effective evaluation during the design period can include:
establishing clear requirements for the expected accessibility conformance
involvement in initial planning meetings for the site
agreeing on a review schedule during the development process
providing information on evaluation approaches so that the developers can
at least do preliminary reviews on their own
To maximize likelihood that a Web site will maintain a given conformance
level in the future, the following provisions should be in place:
clear statement of expected conformance level and scope of Web site it applies
clearly identified individuals responsible for monitoring the site, and follow-up
procedures they can use to rapidly bring non-conformant pages into conformance
clear expectations with regard to frequency, method, and scope of evaluations
processes for validating and evaluating all new types of pages before they
are added to the site
software to facilitate evaluation
incorporation in Web site of address for feedback on accessibility of site
Occasionally Web sites that are "frozen" (legacy; no longer actively maintained
) are found to have substantial accessibility problems. It can be difficult
to determine how to address these. It is helpful to:
identify who the current owner is, and whether they have any obligation or
interest in making the site accessible;
after evaluating the site, outline the changes that would be required to
retrofit the site for accessibility;
identify and propose resources and a timeline for retrofitting the site;
disclose accessibility problems on the site.
Last updated 5 October 2001 by Judy Brewer
(email@example.com) with assistance from
members of EOWG.
© 2001 W3C
Keio ), All Rights Reserved. W3C
document use and
rules apply. Your interactions with this site are in accordance with