ACT Overview - What is ACT

See also: ACT TF homepage, ACT Deliverables

Introduction

The purpose of the Accessibility Conformance Testing (ACT) effort is to establish and document rules for testing the conformance of web content to accessibility standards, such as WCAG 2.0 and future versions. These test rules include automated, semi-automated, and manual testing procedures. The ACT Taskforce aims to make accessibility testing transparent, and so minimise the confusion caused by different interpretations of accessibility guidelines.

The ACT Task Force builds on the work of the Auto-WCAG Community Group, which has been developing for semi-automated and automated WCAG testing for over two years. Both groups are actively contributed to by several of the largest accessibility consultancies, member organizations, as well as vendors of major accessibility test tools.

Objectives

Establish consensus interpretation of accessibility requirements (Primarily WCAG) around a core set of test rules — currently there are varying and often contradictory interpretations of WCAG, which causes confusion and slows down the implementation of web accessibility
Provide transparency and comparability of test tools and implementations — currently test tools employ their own often proprietary interpretations of WCAG, making test results difficult to interpret, combine, and compare
Enable increased tool support by documenting structured testing procedures — establishing a common base of test rules allows tool developers to focus on increasing tool support, to reduce the amount of manual testing required
Support test-driven authoring of success criteria in WCAG 2.1 and beyond — providing the framework for test rules will allow the working group to author future accessibility requirements that are backed by authoritative test rules

Approach

Build on the work of auto-WCAG Community Group — much of this work has been explored and incubated for standardization
Invite organizations with testing expertise — increase participation with specific interest in this testing effort
Establish a normative framework specification — define a common approach, as a W3C standard, for writing test rules
Leverage existing repositories of test rules — encourage organizations to make their test rules publicly available
Continually evolve and add to these test rules — work with organizations to continue growing the set of test rules

Note: the test rules will not be normative, but will be written according to the normative framework specification. This allows for a certain level of stability and interoperability, with flexibility to adapt to changing technologies and requirements.

Scope

The ACT Task Force will develop the normative framework and oversee the development of an extensible ruleset by the Auto-WCAG Community Group, with rules for WCAG Level AA testing for HTML, CSS, and WAI-ARIA at the core. Because the framework is extensible, organizations can build on top of this core ruleset, with rules that they may have a particular interest in.

For example, the EPUB standard has it's own accessibility guidelines. Applicable rules from the WCAG / HTML rules can be used and new rules specific for EPUB can be added for an EPUB ruleset. Similarly, companies with their own accessibility policies can write custom rules, such as requiring a single h1 on every page.

In short, ACT rules can be created for:

Multiple accessibility standards (such as WCAG 2.0, ARIA Authoring Practices), including advanced versions as these standards evolve
Different web technologies and digital publishing technologies (such as HTML, ARIA, EPUB, PWP)
Fully automated and semi-automated test tools; The framework would also be extensible, allowing for integrate of results from manual accessibility tests. (Such as through the WCAG-EM Report Tool.)

Audience

The primary audience of ACT rules are developers of accessibility test tools. They implement rules into their products. The ACT rules serve as documentation and requirements for these tools.

A secondary audience of these rules are accessibility experts. They often assist in setting an organization's accessibility policy, so knowing what rules do, and being able to write rules will be important to them.

Users of accessibility tools (web developers, content authors, QA testers, etc.) are not expected to read the ACT rules. These rules add a layer of documentation, thus complexity to accessibility testing. It is up to accessibility tool vendors to find fitting solutions to communicate issues to their users.

Rule Development

The ACT Task Force will coordinate the development of an initial set of test rules with the Auto-WCAG community group as well as with other groups interested in designing ACT rules. These rules will focus on automated testing for WCAG 2.0 Level A and AA using HTML, CSS, and WAI-ARIA.

Many organizations already have their own rules and test procedures. The ACT Task Force will coordinate with organizations interested in contributing, to inform the framework development. These can be WCAG rules, but also accessibility best practice, and rules designed to catch potential accessibility issues.

Organizations interested in contributing rules to the ACT Rule Suite can do so through the Auto-WCAG CG. The ACT Task Force supports Auto-WCAG in adopting the ACT Framework, including migrating an existing ruleset or test procedure. Specific acceptance requirements for this will be worked out later.

ACT Deliverables

The following outline is further detailed in ACT Deliverables. The result of this effort will be the development of five products, in three areas:

ACT Framework: A W3C Recommendation defining how to write and validate test rules for accessibility testing.
ACT Benchmark
1. ACT Benchmark Method: A description of how to test the accuracy of rules (e.g. false positives and false negatives).
2. ACT Benchmark Tool: An implementation of the benchmark method
ACT Rule Suite
1. ACT Rule Suite Repository: A collection of rules for WCAG that have passed the validation and benchmarking requirements
2. ACT Rule Suite Frontend: A website for the ACT Rule Suite Repository.

For details, see ACT Deliverables.

Organization

To maximize participation of the community and minimize impact on the working group, the following structure will be pursued:

ACT Task Force — Will work under the direction of the working group to develop the normative framework specification and supporting materials needed to implement the standard

auto-WCAG CG — Will develop non-normative test rules according to the normative framework specification, and provide feedback to inform the development of this standard

Project Outline

A detailed schedule for the following is available. This project will be developed in three phases. Each phase will be completed in one year. The phases are designed to solve parts of the goal outlined in this document by developing certain products. Each phase will therefore result in concrete products ready for public use.

Phase 1: Definition

The first phase of this plan will be focused on the following activities:

Create requirements for ACT Framework
Develop the first draft of ACT Framework
Get feedback from stakeholders on this draft
Use this feedback to get the draft ready for test implementations
Work out requirements for the ACT Benchmark and the ACT Rule Suite

Milestone 1:

Last Call ACT Framework
Requirements for ACT Benchmark
Requirements for ACT Suite

Phase 2: Implementation

Call for public feedback on ACT Framework
Work with rule designers to develop (test) implementations of the ACT Framework
Update ACT Framework based on feedback from the implementation
Develop a benchmark method to ensure the quality of ACT rules
Make the first ACT ruleset of rules available

Milestone 2:

Enter Candidate Rec stage for ACT Framework
Complete ACT Benchmark (standalone Note or part of the ACT Framework)
Listing of first set of rules for ACT Suite

Phase 3: Consolidation

Develop an implementation ACT Rule benchmark
Test the quality of the ACT Rules developed in phase 2 using the benchmark
Work with rule designers to ensure the ACT Rules can pass the benchmark
Refine the ACT Framework based on lessons learned and public feedback
Organize a group to maintain the ACT Rule Suite for the future

Milestone 3:

Recommendation ACT Framework
ACT Rule Suite on the W3C website
Organization to maintain the ACT Rule Suite

Relation to other Technologies and Standards

WCAG 2.0 and beyond

Consistent and accurate interpretation is one of the important goals of this project. For this work we will have to work closely with the WCAG Working Group to ensure the way WCAG is intended to be interpreted.

The primary focus of this project will be to create a framework, on top of which ACT Rules can be developed. An ACT Rule will likely have to be tied to a specific technology and accessibility standard. But it is important for the framework to be independent of this. This ensures that the work can be maintained for future versions of WCAG, as well as for other digital accessibility standards.

WCAG Techniques

At first glance, ACT Rules might seem like the techniques that were created for WCAG 2.0. The failure techniques in particular have a few things in common with ACT Rules. There are however a few important aspects to techniques that make them unsuitable for a foundation of this project:

The language is not written for unambiguous interpretation
There is insufficient detail in techniques for consistent implementation
They have a strong reliance on human analyses
Techniques often go beyond setting the minimal requirements for WCAG

All of these stem from a difference in goals. Techniques are designed to teach developers about accessibility, not as a method of conformance testing.