[contents]
This document specifies goals and requirements for a website accessibility evaluation methodology for Web Content Accessibility Guidelines (WCAG) 2.0.
This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.
This 12 October 2011 Editors Draft of Website Accessibility Evaluation Methodology for WCAG 2.0 Requirements is an initial contribution to capture requirements for future work. This document is intended to be published and maintained as a W3C Working Group Note after review and refinement.
The WCAG 2.0 Evaluation Methodology Task Force (Eval TF) invites discussion and feedback about this document by developers, evaluators, researchers, and other practitioners who have interest in web accessibility evaluation. In particular, Eval TF is looking for feedback on how complete, precise, and achievable these requirements are.
Please send comments on this Website Accessibility Evaluation Methodology for WCAG 2.0 Requirements document to public-wai-evaltf@w3.org (publicly visible mailing list archive).
Publication as Editor Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.
Evaluating the conformance of entire websites to Web Content Accessibility Guidelines (WCAG) 2.0 requires a reliable strategy for selecting representative samples of web pages, as it is usually not possible to test every web page. It also requires effective application of WCAG 2.0 Techniques (in particular Failure Techniques) and aggregation of the individual evaluation findings into overall conformance claims. It also includes consideration for occasional (non-systematic) errors such as minor oversight failures that are frequently found on most websites.
This document specifies goals and requirements for a website accessibility evaluation methodology for WCAG 2.0 that defines methods for sampling web pages, provides guidance for applying WCAG 2.0 Techniques, and defines methods for aggregation and reporting evaluation findings. The Methodology addresses the need for an internationally harmonized standard for evaluating the accessibility of websites for accessibility.
The main goal is to define an internationally harmonized methodology for evaluating the conformance of websites to WCAG 2.0. This Methodology supports evaluation in different contexts, such as self-assessment and third-party evaluation of websites. It is applicable to all websites (including web applications) regardless of size and it is independent of any particular evaluation tools, browsers, and assistive technology.
The Methodology defines manual and computer assisted methods for selecting representative samples of web pages from websites that include complete processes. It provides guidance for evaluating the selected web pages to WCAG 2.0, and defines methods for integration and aggregation of the evaluation results into structured reports and conformance claims.
The Methodology provides guidance on evaluation throughout the development process but it is specifically designed for conformance evaluation of existing websites. It extends the existing WAI resource Conformance Evaluation of Websites for Accessibility. Complementary WAI resources such as Preliminary Review of Websites for Accessibility and Involving Users in Evaluating Web Accessibility provide further advice on evaluation during other stages of the development process.
The primary target audience of the Methodology is anyone, including individuals and organizations, wanting to evaluate the conformance of existing websites to WCAG 2.0. This includes but is not limited to:
- Website developers, suppliers, procurers, and owners
- Web accessibility consultants and evaluation services
- Web accessibility monitoring and benchmarking activities
- Web accessibility researchers and disability advocates
Other target audiences of the Methodology include:
- Developers of web accessibility evaluation and repair tools, and authoring tools
- Website developers who want to do evaluation during the development process
- People who want to use the Methodology for education and outreach activities
- Policy makers, project managers, and other decision makers who need a standard
Users of the Methodology are assumed to be knowledgeable of WCAG 2.0, accessible web design, assistive technology, and of how people with different disabilities use the Web.
Important terms used throughout this document include:
- Complete process
- As defined by WCAG 2.0 in: http://www.w3.org/TR/WCAG20/#cc3
- Equivalent results
- High degree of correlation between the evaluation findings so that the final evaluation result is consistent, even when different web pages are selected as part of the representative sample or when different tools are used for evaluation.
- Web page
- As defined by WCAG 2.0 in: http://www.w3.org/TR/WCAG20/#webpagedef.
- Website
- A coherent collection of one or more related web pages that together provide common use or functionality. It includes static web pages, dynamically generated web pages, and web applications.
- Website developer
- Anyone involved in the website development process including but not limited to content authors, designers, programmers, quality assurance testers, and project managers.
- RQ 01: Scope and target audience of the Methodology
- The Methodology defines methods for evaluating the conformance of websites, sections of websites, and web applications, regardless of size, to WCAG 2.0. The Methodology addresses the target audience.
- RQ 02: Clear, understandable, and translatable language
- The Methodology is written in clear and unambiguous language, understandable to the target audience, and capable of translation to other languages.
- RQ 03: Increase inter-rater reliability and correlation
- The Methodology ensures that different web accessibility evaluators yield equivalent results when applying the Methodology on the same websites.
- RQ 04: Independent of any particular tools and browsers
- The use and application of the Methodology is independent of any particular evaluation tools, browsers, and assistive technology. It requires combined use of manual and automated testing approaches to carry out a full evaluation according to the Methodology.
- RQ 05: Conforms to the W3C Specification Guidelines
- The Methodology conforms to the W3C Quality Assurance Framework: Specification Guidelines.
- RQ 06: Harmonized reporting and conformance claims
- The Methodology includes requirements and recommendations for how to report what was evaluated and the evaluation findings, and to express conformance claims. These are based on the Template for Accessibility Evaluation Reports and include recommendations for supplemental machine-readable reporting using Evaluation and Report Language (EARL).
- RQ 07: Reuse of existing WCAG 2.0 testing procedures
- The Methodology employs existing testing procedures within WCAG 2.0 Techniques rather than to replicate them. This is not limited to WCAG 2.0 Techniques published by W3C/WAI.
- RQ 08: Methods for selecting representative samples
- The Methodology defines manual and computer assisted methods for selecting representative samples of web pages from websites that include complete processes.
- RQ 09: Consideration for occassional oversight errors
- The Methodology defines methods for calculating nearness of conformance. These methods are intended to tolerate occassional (non-systematic) errors such as minor oversight failures that do not cause significant accessibility barriers.
- RQ 10: Pre-requisits for applying the Methodology
- The Methodology defines the prerequisites and knowledge necessary for applying the Methodology in practice. Users of the Methodology are assumed to be knowledgeable of WCAG 2.0, accessible web design, assistive technology, and of how people with different disabilities use the Web.
- Evaluation and Report Language (EARL) 1.0
- http://www.w3.org/WAI/intro/earl
- Evaluating Websites for Accessibility
- http://www.w3.org/WAI/eval/
- Web Content Accessibility Guidelines (WCAG) 2.0
- http://www.w3.org/WAI/intro/wcag
- WCAG 2.0 Techniques published by W3C/WAI
- http://www.w3.org/TR/WCAG20-TECHS/
- Quality Assurance Framework: Specification Guidelines
- http://www.w3.org/TR/qaframe-spec/