This document specifies goals and requirements for Website Accessibility Evaluation Methodology for Web Content Accessibility Guidelines (WCAG) 2.0.

Status of this document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This 15 September 2011 Editors Draft of Website Accessibility Evaluation Methodology for WCAG 2.0 Requirements is an initial contribution to capture requirements for future work. This document is intended to be published and maintained as a W3C Working Group Note after review and refinement.

The WCAG 2.0 Evaluation Methodology Task Force (Eval TF) invites discussion and feedback about this document by developers, evaluators, researchers, and other practitioners who have interest in web accessibility evaluation. In particular, Eval TF is looking for feedback on how complete, precise, and achievable these requirements are.

Please send comments on this Website Accessibility Evaluation Methodology for WCAG 2.0 Requirements document to public-wai-evaltf@w3.org (publicly visible mailing list archive).

Publication as Editor Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

Table of Contents

  1. Goals
  2. Scope
  3. Requirements
  4. References

1. Goals

The main goal is an internationally harmonized methodology for evaluating the conformance of websites to WCAG 2.0. This methodology will support different contexts, such as for self-assessment or third-party evaluation of small or larger websites.

The Methodology intends to cover recommendations for sampling web pages and for expressing the scope of a conformance claim, critical path analyses, computer assisted content selection, manual content selection, the evaluation of web pages, integration and aggregation of the evaluation results and conformance statements. The methodology should also address tolerance metrics and include recommendations for harmonized (machine-readable) reporting.

Editor note: discussion on terminology: critical interaction, critical sequence, critical process

2. Scope

Eval TF has the following dependencies:

Eval TF will also coordinate with EOWG where necessary to improve the writing, presentation, and educational value of the methodology.

2.1 Target Audience

There are different possible scenarios for use. The primary scenario would be people who want to evaluate a webpage or website(s). This includes

They could use tools and/or do manual evaluations.

Other (secondary) scenarios include:

The person(s) using the Methodology should be knowledgeable of the WCAG2.0 Guidelines and of people with disabilities.

3. Requirements

3.1 Terminology

3.2 General Requirements

R01: Define methods for evaluating WCAG 2.0 conformance
Our main focus for the Methodology is conformance with WCAG 2.0. The level of the Methodology does not stretch into techniques and tests.
R02: Tool and browser independent
The use and application of the Methodology is independent from (not limited to) specific tools or browsers.
R03: Unique interpretation
The Methodology itself is uniquely interpretable to people who want to use it. It should make it clear to users what they can do if they choose a certain evaluation approach in the document.
Editor note: for discussion - Keep this one? change wording into: unambiguous? Agreed interpretation, objective interpretation? Is this terminology standardized enough?
R04: Reliable and replicable
Different Web accessibility evaluators using the same methods on the same webpages or website(s) should get the same results (Related to R14).
R05: Translatable
It should be translatable into different languages. This also means the use of standards terminology if possible and available.
R06: Point to existing WCAG techniques
It points to the existing WCAG 2.0 tests in the techniques documents and does not reproduce them.
R07: Support for both manual and automated evaluation
It supports people doing automated evaluation and/or manual evaluation. It is not limited to only manual or only automated evaluation. One of the methods would be to provide a way to report on the use automated and/or manual evaluation.
R08: Address the needs of the target audience
(see also the section on target audience). The Methodology is usable for the target audiences. The document will give a short description of the knowledge necessary for using the Methodology for evaluations.
Editor note: for discussion - propose to leave this one out as it is already described in the earlier section.
R09: Support for different contexts
It supports different contexts like self-assessment, third-party evaluation of small or larger websites.
R10: Recommendations for scope and sampling
It includes recommendations for sampling web pages and for expressing the scope of a conformance claim.
R11: Critical path analyses
It includes adding critical paths into the sample of an evaluation. I.e. for a shopping site all pages that are part of the steps in an ordering process would be in the evaluation sample (the webpages that will be evaluated).
Editor note: for discussion - This is already inside the guidelines; should we take it out here?
R12: computer assisted and manual content selection
It covers computer assisted (automated) and manual selection of pages or content for the evaluation sample. Computers can spider websites and choose pages with certain characteristics (like images, tables etc.) and/or choose random pages. This can also be done manually. The Methodology will support both manual and computer assisted or automated.
R13: Includes integration and aggregation of the evaluation results and related conformance statements
It includes methods for aggregation of evaluation results and proposal for conformance statements based on such aggregation of data.
R14: Includes tolerance metrics
Depending on the amount of tolerance, a failure could fall within a certain tolerance level meaning that the page or website would be considered conformant even though there is a failure.
R15: Machine-readable reporting
The Methodology includes recommendations for harmonized (machine-readable) reporting. It provides a format for machine-readable reports using EARL.

4. References

To be developed.