See also: IRC log
Hi Joanie
is the code
<ShaneM> Some notes at wiki: https://www.w3.org/wiki/ARIA_1.1_Automated_Testing
<dbrett> https://github.com/MicrosoftEdge/A11y
Shane: The WPT can hold the
tests
... How it interfaces with AT, my assumption was acting as an
AT
David: yes
Shane: The concept is to make a
channel communication between AT and the test framework
... The WPT has a framework for reporting to W3C
CS: The W3C has a report
framework
... A way to communicate the AT and the test page with some
Javascript
... The information can then be passed/fail
... We output CSV
Shane: There are security issues
with local file system, browser have work arounds
... Providing the fake AT a URL handler with the results
... Pass the data in any useful format
... Goal is to inspect the result automatically
... We want the tests to be run automatically during normal
browser development testing
CS: We care about more than
CR
... We don't need W3C CR reports everytime
Shane: The webkit team uses this,
the data is more than just reports
... Edge maybe using the WPT already, they are in the IRC
channel
CS: Somebody
Shane: Joanie has been thinking of this for other APIs
JD: Narrow it down for me
Shane: Linux
JD: Yes, accessibility APIs
... Each platform would provide their own API
CS: Can you try to do this
Shane: David how do you feel about the code, can it send code
David: Yes I think it could be extended
Shane: It is strictly browser
testing, no capability in turning on the AT connector
... You would need to start the AT emulator, then the
browser
... Does AT attach to a window or how does it work
David: The AT looks at the Accessibility API to find edge
Shane: If there is a named window the AT could find the right window
David: You can just start with the top node and then start looking for the right window
JD: One event we get is a document load event, you can key off that event
Shane: In WPT test is a load, so there would be a load event
JD: That is like ideal, the first thing you need is the document and the event tells you it ready
RS: One question is there an MSAA?
David: There was initially, but
we didn't create all the tests for MSAA
... There are old commits
JG: The WPT is feasible?
Shane: Let's talk about ARIA 1.0, how many tests?
RS: ~900
Shane: They were generally automatically generated?
CS: The expected results though
where human readible
... I was planning on working on a format
RS: Some tests required a time delay, did you allow for that
David: Yes, we can handle waiting
CS: We will start with the simpler tests
Shane: WPT and its test runner
framework, has an interesting async test module, based on
events, so it can handle delays
... The battery test API is an example, so you have to wait for
the battery percentage changes
... Let's not reinvent
CS: The format on the wiki looks OK, but is too simple
RS: You doing a comparison with a predicted states and properties, can you test events
CS: Is there anything else other
than live regions
... Can we test events?
David: It is possible
Shane: How do events get exposed?
CS: There are accessibility API event and then start plateform events
<Rich> http://rawgit.com/w3c/aria/master/core-aam/core-aam.html#mapping_events
RS: State change events...
... You have text remove..., live regions...., there are
special ones for UIA
... MSAA/IA2 they are pretty standard
CS: It is a feature, but there are many other things more important
RS: Agree
CS: We can work on that in the future
RS: They are pretty simple tests
JD: Event testing should be easy, if we are listening for events for knowing when to check, then we are already testing the events
Shane: There are 2 kinds of test,
declarative states and properties, there are more complex tests
that evaluates ....
... It sounds like the current test only sends back one
file
CS: Event testing will be a different kind of test
<ShaneM> David: There is a way to get the data back iteratively.
JD: I will be automating the even
test, I think this will be easy
... If this is going to change the harness, then that would be
a priority
David: DIfferent branch for each browser and platform
JD: We need to figure out exactly needs to be done
Shane: SpecOps has some resources
for this, what I can volunteer to do is get the infrastructure
in place
... If we set this up so there is one click to start, if the
test requires a change .....
... The low hanging fruit is where we should start
CS: We can do some interaction testing, but that is more work
Shane: If you are using web drive it works across browsers
David: It is just navigate and open
CS: We tested the date controls, but are more work to write
Shane: WPT is more than reporting, it will also a runner framework
CS: I will be looking at test cases, and port them to the fields
Shane: RS put a pointer in to
core AAM
... Aren't the assertions in the document
JD: We have a chicken and egg issue
RS: ARIA 1.1 process to CR ....
JG: Next call in two weeks
Shane: We should be looking at next time having this machine extractable
JD: Is what the core AAM does is ... the checkbox is checked is is different on each platform
<ShaneM> ACTION: ShaneM to create sample wrapper to talk with a fake AT under WPT by 7/27 [recorded in http://www.w3.org/2016/07/13-aria-test-minutes.html#action01]
<trackbot> Created ACTION-2095 - Create sample wrapper to talk with a fake at under wpt by 7/27 [on Shane McCarron - due 2016-07-20].
<cyns> ATK connector (Joanie) IA2 connector AX Connector Event testing (Joanie) Reporting with WPT (Shane) Document test case format (Cyns) Parsing JSON test case format (Brett) Create tests (working group)
ACTON: joanie ATK connector
<scribe> ACTION: joanie ATK connector [recorded in http://www.w3.org/2016/07/13-aria-test-minutes.html#action02]
<trackbot> Created ACTION-2096 - Atk connector [on Joanmarie Diggs - due 2016-07-20].
<scribe> ACTION: joanie IA2 connector AX Connector Event testing [recorded in http://www.w3.org/2016/07/13-aria-test-minutes.html#action03]
<trackbot> Created ACTION-2097 - Ia2 connector ax connector event testing [on Joanmarie Diggs - due 2016-07-20].
<scribe> ACTION: cyns Document test case format [recorded in http://www.w3.org/2016/07/13-aria-test-minutes.html#action04]
<trackbot> Created ACTION-2098 - Document test case format [on Cynthia Shelly - due 2016-07-20].
<scribe> ACTION: cyns David Brett will work on parsing JSON test case format [recorded in http://www.w3.org/2016/07/13-aria-test-minutes.html#action05]
<trackbot> Created ACTION-2099 - David brett will work on parsing json test case format [on Cynthia Shelly - due 2016-07-20].
This is scribe.perl Revision: 1.144 of Date: 2015/11/17 08:39:34 Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: RRSAgent_Text_Format (score 1.00) Succeeded: s/aria// No ScribeNick specified. Guessing ScribeNick: jongund_ Inferring Scribes: jongund_ Present: ShaneM Cyns jongund_ Joanmarie_Diggs Rich Found Date: 13 Jul 2016 Guessing minutes URL: http://www.w3.org/2016/07/13-aria-test-minutes.html People with action items: cyns joanie shanem WARNING: Input appears to use implicit continuation lines. You may need the "-implicitContinuations" option.[End of scribe.perl diagnostic output]