[contents]

Abstract

This document specifies goals and requirements for Website Accessibility Evaluation Methodology for Web Content Accessibility Guidelines (WCAG) 2.0.

1. Goals

The main goal is an internationally harmonized methodology for evaluating the conformance of websites to WCAG 2.0. This methodology will support different contexts, such as for self-assessment or third-party evaluation of small or larger websites.

The Methodology intends to cover recommendations for sampling web pages and for expressing the scope of a conformance claim, critical path analyses, computer assisted content selection, manual content selection, the evaluation of web pages, integration and aggregation of the evaluation results and conformance statements. The methodology should also address tolerance metrics and include recommendations for harmonized (machine-readable) reporting.

Editor note: discussion on terminology: critical interaction, critical sequence, critical process

2. Scope

Eval TF has the following dependencies:

• WCAG WG - to ensure accurate interpretation and representation of WCAG requirements
• ERT WG - to facilitate coordination with other W3C and WAI testing efforts

Eval TF will also coordinate with EOWG where necessary to improve the writing, presentation, and educational value of the methodology.

2.1 Target Audience

There are different possible scenarios for use. The primary scenario would be people who want to evaluate a webpage or website(s). This includes

• Web accessibility benchmarking activities
• Web content producers wishing to evaluate their content
• Developers of Evaluation and Repair Tools
• Policy makers and Web site owners wishing to evaluate websites
• Others (to be specified..)

They could use tools and/or do manual evaluations.

Other (secondary) scenarios include:

• People who want to use the Methodology for education and outreach activities.
• Designers and developers wishing to evaluate in advance of the production or design of a website or webpage.

The person(s) using the Methodology should be knowledgeable of the WCAG2.0 Guidelines and of people with disabilities.

3. Requirements

3.1 Terminology

3.2 General Requirements

R01: Define methods for evaluating WCAG 2.0 conformance

Our main focus for the Methodology is conformance with WCAG 2.0. The level of the Methodology does not stretch into techniques and tests.

R02: Tool and browser independent

The use and application of the Methodology is independent from (not limited to) specific tools or browsers.

R03: Unique interpretation

The Methodology itself is uniquely interpretable to people who want to use it. It should make it clear to users what they can do if they choose a certain evaluation approach in the document.

Editor note: for discussion - Keep this one? change wording into: unambiguous? Agreed interpretation, objective interpretation? Is this terminology standardized enough?

R04: Reliable and replicable

Different Web accessibility evaluators using the same methods on the same webpages or website(s) should get the same results (Related to R14).

R05: Translatable

It should be translatable into different languages. This also means the use of standards terminology if possible and available.

R06: Point to existing WCAG techniques

It points to the existing WCAG 2.0 tests in the techniques documents and does not reproduce them.

R07: Support for both manual and automated evaluation

It supports people doing automated evaluation and/or manual evaluation. It is not limited to only manual or only automated evaluation. One of the methods would be to provide a way to report on the use automated and/or manual evaluation.

R08: Address the needs of the target audience

(see also the section on target audience). The Methodology is usable for the target audiences. The document will give a short description of the knowledge necessary for using the Methodology for evaluations.

Editor note: for discussion - propose to leave this one out as it is already described in the earlier section.

R09: Support for different contexts

It supports different contexts like self-assessment, third-party evaluation of small or larger websites.

R10: Recommendations for scope and sampling

It includes recommendations for sampling web pages and for expressing the scope of a conformance claim.

R11: Critical path analyses

It includes adding critical paths into the sample of an evaluation. I.e. for a shopping site all pages that are part of the steps in an ordering process would be in the evaluation sample (the webpages that will be evaluated).

Editor note: for discussion - This is already inside the guidelines; should we take it out here?

R12: computer assisted and manual content selection

It covers computer assisted (automated) and manual selection of pages or content for the evaluation sample. Computers can spider websites and choose pages with certain characteristics (like images, tables etc.) and/or choose random pages. This can also be done manually. The Methodology will support both manual and computer assisted or automated.

R13: Includes integration and aggregation of the evaluation results and related conformance statements

It includes methods for aggregation of evaluation results and proposal for conformance statements based on such aggregation of data.

R14: Includes tolerance metrics

Depending on the amount of tolerance, a failure could fall within a certain tolerance level meaning that the page or website would be considered conformant even though there is a failure.

R15: Machine-readable reporting

The Methodology includes recommendations for harmonized (machine-readable) reporting. It provides a format for machine-readable reports using EARL.

4. References

To be developed.