WAI-ARIA test suites

http://code.google.com/p/aria-user-agent-implementation-testing/source/browse/#svn%2Ftrunk

JG: each test generates test HTML and JavaScript
... slider - include test code and mapping for role, properties, and states for that test
... also provides means to do different types of labeling for the control to test name calculation
... JS file is for the results - generates JS object - can edit file with results
... collect results in index.html page
... edit results file - 2 types of results

DB: edit source file or objects

JG: edit source file - check out/check in
... tests themselves aren't automated - somebody has to verify the results

DB: tricky to automate and have it be cross-browser

JG: if there was a way to automate - have to build an index for the tests
... files are generated with python scripts
... if new code needs to be added to all the tests, it's fairly easy to do
... using inspect tools on each platform

DB: don't use UISpy b/c don't support UIA on FF

CS: use Inspect (not the old one - in Windows 7 SDK)
... other tool is accchecker and it's on codeplex

JG: open test case in browser - use inspect tool to move focus
... test program automatically gives focus to one of the controls
... look at mapping and accessible name calculated in the inspect tool

TB: is it clear what a pass condition would be?

JG: if it's unclear - students will enter a "?" and then we can search for the "?"s
... for checkbox - mixed state - looks like it's unclear so we call that a failure

CS: did you consider writing a client that would compare the APIs to what they should be

JG: other part is generating the test cases
... if can automate the evaluation, that would be great
... easy to add meta information programmatically

DB: have been some efforts to automate - but not been maintained
... get people with right skills and find a way to keep it going

<davidb> http://www.youtube.com/watch?v=UHTd8c8jY8Y

CS: accchecker has some of these features - tells you what the values are supposed to be
... inspect just gives you what it finds but doesn't tell you what it should be
... accchecker is pretty scriptable too

JG: have more labor than programming skills

CS: MS guy who just joined OpenAjax alliance to see if there is anything we can do there

JG: continue focusing on creating the tests - an automated tool could still run them
... could generate design patterns for how to generate tests automatically
... not as straightforward as I thought it was going to be
... WAI-ARIA process needs some results fairly soon

DB: think there is value in what you're doing - can be used for human tests with screen reader too
... if we wanted to figure out how to resource getting talent to help students automate tests, Mozilla would be interested in matching it

CS: U. Washington has STEM students with disabilities that are available to do work and UW pays them
... you get to interview them before hiring them
... part of the DOIT program

JG: sounds like a useful thing would be to add machine readable result information
... basically copy table content - not consistent for the states

CS: we need the feedback on where the UAIG is inconsistent

JG: test 1.5 in checkbox directory - MSAA set SYSTEM-CHECKED state
... SYSTEM_CHECKED = "on"?

CS: after LC, we could make them more machine readable

JG: Mac values seem straightforward
... haven't done anything with ATK because none of the students use Unix - could build a box that uses Unix to do testing
... don't know if I'll teach the class in the fall again
... students like the course because they learn something about JS

CS: if build a tool, would be using accessibility APIs - would teach them a lot of what they need to know to build an AT

JG: attending to the F2F remotely - can present this for test suite discussion
... JSON files - abandoned - converted everything to python objects - easier to deal with
... .py scripts provide the data that generates the files
... don't edit the html files - they are generated
... and the .js files

Bugs

CS: closed the one about exposing actions when an object is disabled
... no UAIG edits needed - implementation is inconsistent

DB: UA can decide what's appropriate to do when actions are exposed on disabled object

CS: almost finished with events table - have one outstanding question

AS: try to complete these by e-mail by the F2F meeting so that we are technically ready for LC even if we don't actually publish before then

HTML mapping

CS: can we start using part of this call for HTML mapping with Steven Faulkner?

AS: next week, we'll allocate half the meeting

This is scribe.perl Revision: 1.135  of Date: 2009/03/02 03:52:20  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: RRSAgent_Text_Format (score 1.00)

Succeeded: s/it's/the UAIG is/
No ScribeNick specified.  Guessing ScribeNick: Andi
Inferring Scribes: Andi
Default Present: Andi_Snow-Weaver, David_Bolter, Cynthia_Shelly, Tim_Boland
Present: Jon_Gunderson

WARNING: Fewer than 3 people found for Present list!


WARNING: No meeting title found!
You should specify the meeting title like this:
<dbooth> Meeting: Weekly Baking Club Meeting


WARNING: No meeting chair found!
You should specify the meeting chair like this:
<dbooth> Chair: dbooth

Got date from IRC log name: 21 Apr 2011
Guessing minutes URL: http://www.w3.org/2011/04/21-aapi-minutes.html
People with action items: