DRAFT WCAG 2.0 Implementation Testing Framework

This draft attempts to collect thoughts about Implementation Testing for Web Content Accessibility Guidelines 2.0 (WCAG 2.0). It has not been reviewed by the Web Content Accessibility Guidelines Working Group (WCAG WG) and is intended to stimulate discussion within the WCAG WG.

Note: 10 August 2003, received feedback from Patrick Curran but have not incorporated.

On this page:

Frequently Asked Questions | Related resources | Principles Applicable to Test Suite Content
Implementation Testing: Variables | Additional notes (20031003)

Frequently Asked Questions

What is implementation testing?

Implementation testing generally refers to the process of testing implementations of technology specifications. This process serves the dual purpose of verifying that the specification is implementable in practice, and that implementations conform to the specification. This process helps to improve the quality and interoperability of implementations.

Patrick Curran, 10 August 2003

Testing is part of the theW3C Recommendation Track, a required criteria to meet in order to exit the Candidate Recommendation phase. Refer to section7.4.3 Call for Implementations for more information about this stage.

Section 1.3 of the QA Framework: Test Guidelines says,

The development of quality test materials helps enhance the quality of specifications by identifying aspects of the specification that are ambiguous, contradictory or not implementable. By helping to improve the clarity of the specification, the quality of implementations is also improved.

Quality test materials also help improve the conformance of implementations by providing methods of checking conformance to well defined criteria in a consistent way. By providing a consistent way to check the conformance of implementations, test materials also lead to more interoperable implementations.

When will implementation testing start?

The WCAG WG aims to begin testing Fall of 2003. We are drafting Techniques documents that are needed for testing.

What will the WCAG WG do with the feedback?

We will use it to make the WCAG 2.0 Suite of materials a better product. Feedback will likely stimulate changes and clarifications to the document while other feedback or questions might become part of a FAQ.

What support is the WCAG WG giving to implementation testers?

We anticipate having two tiers of implementation testers:

  1. Those who we are working closely with
  2. Those who are working on their own and sending feedback

The first tier is likely to have more interaction with the WCAG WG, including interpretation advice (i.e., "am i doing this right?"). We might invite them to a teleconference meeting to speak directly to the WCAG WG.

How do I send comments to the WCAG WG?

Please send comments to public-comments-wcag20@w3.org. The archives of this list are available to the public.

Is there a benefit to participating in implementation testing?

@@Jump on migration from WCAG 1.0 to 2.0, take advantage of the improvements in 2.0, be involved in the W3C process and have an effect. Be warned that document is still changing and that minor tweaks to site might be required (??) or at least interpretation of the guidelines as they change.

What is the WCAG WG attempting to test?

The WCAG WG published Requirements for WCAG 2.0 to outline the goals for WCAG 2.0. To advance through the W3C Recommendation Track we need to demonstrate that we have met our goals:

Related resources

Principles Applicable to Test Suite Content

(Modeled after Section 3.2 of SVG Conformance Test Suite -- Test Builder's Manual - just a series of questions for now)

We need to draft Test Suites Principles. Looking at the SVG Conformance Test Suite -- Test Builder's Manual, here are some questions about how this might apply to WCAG 2.0.

1. Simple or Atomic. Each test purpose should be as simple as possible and narrowly focused on a atomic (simple) functionality. Example: chose one attribute or property and exercise it through a range, while holding other variables constant.

In a WCAG 2.0 test suite, we could test the technology property and hold language and type of site consistent. But, how do we do this at an element level? Won't we need this to be at site level?

Comprehensive. The detailed tests should try to methodically vary and test all values, plus boundary conditions and extreme conditions, of each parameter, attribute, or property.

Since we are writing guidelines, it doesn't seem possible to test all values because we are not limiting what people do.

Test cases in the SVG Test Builder's Manual primarily test the renderer, what about testing the SVG itself? What about determining if a generator is generating SVG correctly? WCAG tests will be more along the lines of the latter (is the code being generated so that it is accessible).

Implementation Testing: Variables

Other possible variables

Questions and other thoughts

Implementation Framework

A set of materials for recruiting and instructing: "If you want to test WCAG 2.0..."

Possible process:

  1. fill out questionnaire to gather information about the above variables (what technologies does your site use, what languages is it written in, etc.)
  2. set up a bugzilla account as an imp tester
  3. as you work on site, use bugzilla to comment on specific issues
  4. ???

Additional notes (20031003)

  1. 1st step: group familiarize w/resources (above: QA stuff)
  2. write testing documentation (ala tim's stuff), clarify goals

$Date: 2005/12/20 23:22:29 $ $Author: wendy $