Requirements for "Techniques for Automated and Semi-Automated Evaluation Tools"

This document provides a set of initial requirements that need to be incorporated in the document "Techniques for Automated and Semi-Automated Evaluation Tools". Further refinements of this document will occur under the scope of the Evaluation and Repair Tools Working Group (ERT WG) discussions.

This version:
Previous published version:
Carlos A Velasco, Fraunhofer Institute for Applied Information Technology FIT

Purpose of the document

The document presented here gathers requirements for the document "Techniques for Automated and Semi-Automated Evaluation Tools", in the following called the document. This requirements document will present also typical scenarios of the use of the main document.

The purpose of the document "Techniques for Automated and Semi-Automated Evaluation Tools" is to present introductory information for developers of accessibility evaluation tools on how to implement WCAG 2.0 and its evaluation methodology in their tools. That includes, for instance, integration of accessibility testing in different workflows, descriptions of typical features of evaluation tools, best practice examples, classification of tools, etc.

Objectives of the document

The objectives of the document "Techniques for Automated and Semi-Automated Evaluation Tools" include, among others, the following:

  1. Present ways to classify accessibility evaluation tools according to their profile, i.e., according to the features and combinations thereof that they may include.
  2. Introduce ways to classify tools according to their licensing scheme, to their target user group and to other criteria.
  3. Present different workflows for accessibility evaluation and actors that participate in them.
  4. Support developers of accessibility evaluation tools to understand the different types of techniques in WCAG 2.0.
  5. Present how to distinguish different types of web accessibility tests: automatic, semiautomatic and manual.
  6. Support developers of accessibility evaluation tools in presenting results to different audiences.
  7. Present best practice examples on the development of accessibility evaluation tools.

Audience of the document

The document "Techniques for Automated and Semi-Automated Evaluation Tools" is targeted mainly to development managers and developers of accessibility evaluation tools. Under this scope, we will not distinguish between commercial and open source developers, although there are use cases and issues that can be more relevant to one group than to the other.

A secondary audience of this document are users of accessibility evaluation tools like accessibility experts or web developers.

Types of tools included

Examples of tools that are included are:

Profile of an evaluation tool

The document will contain descriptions of different functionalities or features that are included in different evaluation tools, which help to classify them and to identify their limitations. Typical examples include:


Here we will present two or more scenarios which can put in context the recommendations of the document.

Issues not covered in this document

The following issues are not covered in this document:


  1. Web Content Accessibility Guidelines (WCAG) 2.0
  2. Website Accessibility Conformance Evaluation Methodology 1.0
  3. Developer Guide for Evaluation and Report Language (EARL) 1.0
  4. UWEM, Unified Web Evaluation Methodology version 1.2
  5. Requirements for web developers and web commissioners in ubiquitous Web 2.0 design and development (January 2012)
  6. ACCESSIBLE project

Table of contents

What follows is a preliminary table of contents for the document:

  1. Abstract
  2. Status of this document
  3. Introduction
    1. Audience of this document
    2. Document conventions
    3. Complementary resources
  4. Profile of an evaluation tool
  5. Type of evaluation tools
  6. Best practices
    1. How to implement tests for WCAG 2.0 and its techniques
    2. Integrating the testing procedure into the development workflows (only the necessary)
      1. Accessibility testing actors
      2. Evaluation procedures
      3. Types and scope of tests
      4. Testing web technologies and document types
      5. Morphology of a test
      6. Reporting for different audiences
    3. ...
  7. References