ARIA 1.0 Test Plan
This page is intended to become the primary resource for the ARIA test plan. Previous information was available in the wiki.
Todo:
- Set target dates
- Construct overall timeline
- Flesh out the details
- Find owners for each of the steps
- Identify test priorities (mostly done in CR prep, but need to check for stray MUSTs)
- Build / obtain test repository
- Create / harvest test files
- Unit tests (mostly auto generated)
- Feature tests
- Dynamic tests
- Additional tests
- Develop test procedures
- Identify test tools and determine how to use for this project (see http://lists.w3.org/Archives/Member/w3c-wai-pf/2011JanMar/0066.html for example)
- Build / obtain test harness
- Identify test targets (UAs on OS)
- Establish point of contact with user agents we're testing so we can address problems
- The following implementations have been submitted for consideration:
- @@
- Note AIA
- Build / obtain test tools
- Identify / recruit testers
- Train testers
- Set up automatable tests?
- Execute tests and record results
- Perform QA checks on test results
- Analyze data to determine if necessary coverage found
- Write draft implementation report
- Work with implementers to patch holes in coverage
- Redo needed tests
- Write final implementation report
Take on plan from Judy
- enumerating what tests are needed
- enumerating what implementations may exist; identifying gaps where nothing exists; securing commitments to fill gaps
- verifying what valid implementations do exist; identifying gaps where implementations aren't yet valid; securing commitments to fix
- verifying and reporting complete CR exit criteria met
Testable statements
Tests required for CR
The following is a set of types of assertions that must be tested for CR.
- Proper mapping of each non-abstract role to AAPI (61)
- Proper non-mapping of abstract roles to AAPI (12)
- Proper mapping of singleton states and properties to AAPI, with each
of their possible values
- global states and properies, on elements with no explicit ARIA role (~57)
- supported and required states and properties, on elements with ARIA role that supports them (~150)
- Mapping of default values of non-provided states and properties
where defined and would lead to a positive behaviour, and required by UAIG
- global states and properties
- supported states and properties
- required states and properties
- role overrides on default from "implict value for role"
- Proper mapping of role to AAPI that is modified by property
- button with aria-haspopup
- button with aria-pressed
- colheader and rowheader with aria-sort
- textbox with aria-multiline
- All permutations of accessible name algorithm (@@)
- In general
- Roles with specific "name from" requirements
- Host language elements with native semantics that impact accessible
name
- img
- form controls
- links
Tests excluded from CR
The following tests have been identified that would benefit browser interoperability verification, but are understood to be not required to advance beyond CR.
- Required owned role
- Required owned by role
- Incorrect values for states and properties
- Not in the token list
- Duplicate tokens
- Not a proper number
- Unresolveable ID reference(s)
- Alternate lexical representations of true/false
- Updates to accessible name in live regions
- impact of aria-live
- impact of aria-atomic
- Impact of roles with "children are presentational" set to true
- roles on each HTML element, relationship to native semantics
Test plan template
ID | Host lang | Testable statement | Type | Test file | Test procedure | Expected Result |
---|---|---|---|---|---|---|
1 | HTML | mapping of alert | role | roles-plain-alert | check mapping of first child element under element with id "test" | should map to @@ |
Test types
- name computation
- role
- state/property
Expected results
- By platform, different test plans for each platform
- test plans for MSAA, IA2, AXAPI, @@
Machine format
<tests>
<test id="1" type="role">
<testfile href="roles-plain-alert"/>
<procedure/>
<result>@@</result>
</test>
</tests>