for "Techniques for Automated and Semi-Automated Evaluation
This document provides a set of initial
requirements that need to be incorporated in the document "Techniques for
Automated and Semi-Automated Evaluation Tools". Further refinements of this
document will occur under the scope of the Evaluation and Repair Tools Working Group
(ERT WG) discussions.
- This version:
- Previous published version:
- Carlos A Velasco, Fraunhofer
Institute for Applied Information Technology FIT
© 2013 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability,
use rules apply.
Purpose of the document
document presented here gathers requirements for the document "Techniques
for Automated and Semi-Automated Evaluation Tools", in the following
called the document. This requirements document will
present also typical scenarios of the use of the main document.
purpose of the document "Techniques for Automated and Semi-Automated
Evaluation Tools" is to present introductory information for developers of
accessibility evaluation tools on how to implement WCAG 2.0 and its
evaluation methodology in their tools. That includes, for instance,
integration of accessibility testing in different
workflows, descriptions of typical
features of evaluation tools, best practice
examples, classification of tools, etc.
Objectives of the document
The objectives of the
document "Techniques for Automated and Semi-Automated Evaluation Tools"
include, among others, the following:
- Present ways to classify accessibility evaluation tools according to
their profile, i.e., according to the features and combinations thereof
that they may include.
- Introduce ways to classify tools according to their licensing
scheme, to their target user group and to other criteria.
- Present different workflows for accessibility evaluation and actors
that participate in them.
- Support developers of accessibility evaluation tools to understand
the different types of techniques in WCAG 2.0.
- Present how to distinguish different types of web accessibility
tests: automatic, semiautomatic and manual.
- Support developers of accessibility evaluation tools in presenting
results to different audiences.
- Present best practice examples on the development of accessibility
Audience of the document
The document "Techniques for
Automated and Semi-Automated Evaluation Tools" is targeted mainly to
development managers and developers of accessibility evaluation
tools. Under this scope, we will not distinguish between commercial
and open source developers, although there are use cases and issues that can
be more relevant to one group than to the other.
A secondary audience
of this document are users of accessibility evaluation tools like
accessibility experts or web developers.
Types of tools
Examples of tools that are included are:
- Industrial/commercial and open source tools, which test complete
websites or web applications.
- Focused tools, which test a concrete aspect of accessibility, for
instance, testing contrast of images, accessibility of forms, ARIA
- Tools that support research with users or developers of specific
aspects of accessibility.
Profile of an evaluation tool
The document will contain
descriptions of different functionalities or features that are included in
different evaluation tools, which help to classify them and to identify
their limitations. Typical examples include:
- ability to crawl big websites or portals
- types of web technologies handled by the tool, for instance HTML
markup, stylesheets, PDF documents, Flash applications, multimedia,
- ability to integrate dynamic content generated via scripting
(dynamic modification of the Document Object Model according to the user
interaction with the application, etc.)
- support for testing APIs like the WebDriver API, for instance
- support for standard reporting languages like EARL
- support for different accessibility compliance environments in
- integration in the web development workflow as a plug-in add-on in
different Integrated Development Environments (open source or
- multilinguality and internationalisation
Here we will present two or more scenarios which
can put in context the recommendations of the document.
covered in this document
The following issues are not covered in this
- Procurement and acquisition issues for this type of tools are
outside of the scope of this document and are covered elsewhere
- Interpretation of WCAG 2.0 success criteria and
- How to interpret standards and recommendations related to web
- Web Content Accessibility
Guidelines (WCAG) 2.0
- Website Accessibility
Conformance Evaluation Methodology 1.0
- Developer Guide for
Evaluation and Report Language (EARL) 1.0
- UWEM, Unified Web
Evaluation Methodology version 1.2
- Requirements for
web developers and web commissioners in ubiquitous Web 2.0 design and
development (January 2012)
- ACCESSIBLE project
Table of contents
What follows is a preliminary table of
contents for the document:
- Status of this document
- Audience of this document
- Document conventions
- Complementary resources
- Profile of an evaluation tool
- Type of evaluation tools
- Best practices
- How to implement tests for WCAG 2.0 and its
- Integrating the testing procedure into the development workflows
(only the necessary)
- Accessibility testing actors
- Evaluation procedures
- Types and scope of tests
- Testing web technologies and document types
- Morphology of a test
- Reporting for different audiences