W3C logoWeb Accessibility Initiative (WAI) logo

WAI: Strategies, guidelines, resources to make the Web accessible to people with disabilities

[Draft] Conformance Evaluation of Web Sites for Accessibility

Page Contents

Introduction

A conformance evaluation determines if a Web site meets accessibility standards, such as the Web Content Accessibility Guidelines (WCAG). This page describes a conformance evaluation method that combines automatic, semi-automatic, and manual testing of Web site accessibility. It can be used when developing a new site, or to evaluate an existing site.

This page focuses on technical assessment and does not include involving users with disabilities, which will be addressed in a separate document. Including users in evaluation helps ensure that technical accessibility solutions are applied effectively. Evaluations that combine technical assessment and usability testing of accessibility can be called comprehensive evaluations.

The conformance evaluation method described below requires:

A conformance evaluation includes all of the tasks below except those that are explicitly identified as alternatives or optional. Note: while determining the scope of the evaluation is a key first task, and summarizing and reporting the results of evaluation is the logical conclusion, the order of the intervening tasks is not crucial.

Determine the scope of the evaluation

Determine and disclose scope of site to be evaluated and the targeted conformance level for the evaluation. Note: disclosure should be internal to the organization during evaluation; if conformance is claimed publicly, disclose externally (for example, on the Web site).

  1. Determine and disclose the target conformance level of WCAG 1.0.
  2. Select a representative sampling of pages for manual evaluation that match the following criteria:
    • Include all pages on which people are more likely to enter your site ("welcome page", etc.)
    • Include a variety of pages with different layouts and functionality, for example:
      • Web pages with tables, forms, or dynamically generated results;
      • Web pages with informative images such as diagrams or graphs;
      • Web pages with scripts or applications that perform functionality.
    Note: there are special considerations for Web sites with database driven dynamically generated Web content.
  3. Identify and disclose the entire Web site including all pages at a base URL for automatic and semi-automatic evaluation; Note: if testing of the entire site is not feasible (for example, because of its unusual size or dynamic nature) identify an expanded page selection, to be clearly explained and disclosed on the Web site. Suggestions for inclusions in this expanded page selection: pages from different sections of the Web site; pages representing different "look & feel"; pages representing different development tools and processes including those generated from databases; pages produced under different guidelines; "contact us" pages; pages critical to your business; etc. If any area of a site is excluded from evaluation, be sure to disclose this information.

Use Web accessibility evaluation tools

  1. Validate markup including syntax and style sheets, using all applicable validators, on the selected sample of pages. Run at least one validation tool across entire Web site (or expanded page selection):
  2. Use at least two Web accessibility evaluation tools on the selected sample of pages and run at least one tool across entire Web site (or expanded page selection). Note any problems indicated by the tools. Note: Using at least two evaluation tools helps catch potential misidentification of accessibility problems that might result from using a single evaluation tool.

Manually evaluate representative page sample

Apply accessibility checklist to page sample

Examine the selected sample of pages using checkpoints from the Checklist of Checkpoints for Web Content Accessibility Guidelines 1.0 that are applicable to your site. Note: Applicable can mean checkpoints that cannot be evaluated by automatic or semi-automatic tools; checkpoints that actually apply to the site (for example, if site contains no audio content, skip those); and, as a minimum, those checkpoints that apply to the level of conformance you are evaluating.

Examine pages using graphical browsers

Examine the selected sample of pages with graphical user interface (GUI) browsers: select at least three different configurations from among the following variables: different graphical user interface browsers (such as Internet Explorer, Mozilla Firefox, Netscape Navigator, Opera, Safari, or others), in different versions (latest, older), running on different platforms (Windows, Linux, Mac) and making the following adjustments:

  1. Turn off images, and check whether appropriate alternative text is available.
  2. Turn off the sound, and make sure audio content is still available through text equivalents.
  3. Use browser controls to vary font-size: verify that the font size changes on the screen accordingly; and that the page is still usable at larger font sizes.
  4. Test with different screen resolution, and/or by resizing the application window to less than maximum, to verify that horizontal scrolling is not required (caution: test with different browsers, or examine code for absolute sizing, to ensure that it is a content problem not a browser problem).
  5. Change the display color to gray scale (or print out page in gray scale or black and white) and observe whether the color contrast is adequate.
  6. Without using the mouse, use the keyboard to navigate through the links and form controls on a page (for example, using the "Tab" key), making sure that you can access all links and form controls, and that the links clearly indicate what they lead to.
  7. Also examine page with scripts, style sheets, applets, and other embedded objects not loaded.

Note: Browser extensions and other plug-in evaluation tools (such as AIS Toolbar for Internet Explorer, WAVE Toolbar for Firefox, Internet Explorer, and Netscape Navigator, or Web Developer Extension for Firefox) provide functionality to help perform many of these manual checks.

Note: For reviewers who have disabilities, certain of the following tasks may need to be done with another person who does not have the same disability.

Examine pages using specialized browsers

Examine the selected sample of pages with one text browser (such as Lynx) and one voice browser (such as Home Page Reader), and answer the following questions:

With text browser:
Is equivalent information and function (for example, links and scripted events) available through the text browser as is available through the GUI browser?
Is the information presented in a meaningful order when read serially?
With voice browser:
Is equivalent information available through the voice browser as is available through the GUI browser?
Is the information presented in a meaningful order when spoken serially?
Note: for settings where there is limited choice of assistive technologies, also perform a manual evaluation of the Web site with those assistive technologies; for instance, JAWS is the only screen reader translated into Danish, and therefore in Denmark a trained evaluator should evaluate the Web site using JAWS.

Read and evaluate page content

Read over the selected sample of pages: is the text clear and simple to the extent appropriate for the purpose of the Web site? (For English sites, consider using Clear and Appropriate Language and Design (CLAD) test.)

Summarize and report findings

Summarize any problems and best practices identified for each page type and a representative URL, and method by which they were identified. Recommend follow-up steps, potentially including:

Note: for evaluations using an expanded page selection instead of the entire Web site, apply what you've learned to other pages.

Related pages

This document is part of a multi-page Evaluating Web Accessibility resource suite that outlines different approaches for evaluating Web accessibility.