ARIA 1.0 Test Plan

This page is intended to become the primary resource for the ARIA test plan. Previous information was available in the wiki.

Todo:

Set target dates
Construct overall timeline
Flesh out the details
Find owners for each of the steps

Identify test priorities (mostly done in CR prep, but need to check for stray MUSTs)
Build / obtain test repository
Create / harvest test files
1. Unit tests (mostly auto generated)
2. Feature tests
3. Dynamic tests
4. Additional tests
Develop test procedures
Identify test tools and determine how to use for this project (see http://lists.w3.org/Archives/Member/w3c-wai-pf/2011JanMar/0066.html for example)
Build / obtain test harness
Identify test targets (UAs on OS)
1. Establish point of contact with user agents we're testing so we can address problems
2. The following implementations have been submitted for consideration:
  - @@
  - Note AIA
Build / obtain test tools
Identify / recruit testers
Train testers
Set up automatable tests?
Execute tests and record results
Perform QA checks on test results
Analyze data to determine if necessary coverage found
Write draft implementation report
Work with implementers to patch holes in coverage
Redo needed tests
Write final implementation report

Take on plan from Judy

enumerating what tests are needed
enumerating what implementations may exist; identifying gaps where nothing exists; securing commitments to fill gaps
verifying what valid implementations do exist; identifying gaps where implementations aren't yet valid; securing commitments to fix
verifying and reporting complete CR exit criteria met

Testable statements

Tests required for CR

The following is a set of types of assertions that must be tested for CR.

Proper mapping of each non-abstract role to AAPI (61)
Proper non-mapping of abstract roles to AAPI (12)
Proper mapping of singleton states and properties to AAPI, with each of their possible values
- global states and properies, on elements with no explicit ARIA role (~57)
- supported and required states and properties, on elements with ARIA role that supports them (~150)
Mapping of default values of non-provided states and properties where defined and would lead to a positive behaviour, and required by UAIG
- global states and properties
- supported states and properties
- required states and properties
- role overrides on default from "implict value for role"
Proper mapping of role to AAPI that is modified by property
- button with aria-haspopup
- button with aria-pressed
- colheader and rowheader with aria-sort
- textbox with aria-multiline
All permutations of accessible name algorithm (@@)
- In general
- Roles with specific "name from" requirements
- Host language elements with native semantics that impact accessible name
  - img
  - form controls
  - links

Tests excluded from CR

The following tests have been identified that would benefit browser interoperability verification, but are understood to be not required to advance beyond CR.

Required owned role
Required owned by role
Incorrect values for states and properties
- Not in the token list
- Duplicate tokens
- Not a proper number
- Unresolveable ID reference(s)
- Alternate lexical representations of true/false
Updates to accessible name in live regions
- impact of aria-live
- impact of aria-atomic
Impact of roles with "children are presentational" set to true
roles on each HTML element, relationship to native semantics

Test plan template

ID	Host lang	Testable statement	Type	Test file	Test procedure	Expected Result
1	HTML	mapping of alert	role	roles-plain-alert	check mapping of first child element under element with id "test"	should map to @@

Test types

name computation
role
state/property

Expected results

By platform, different test plans for each platform
test plans for MSAA, IA2, AXAPI, @@

Machine format

Site Navigation