21:01:54 <RRSAgent> RRSAgent has joined #aria-test
21:01:54 <RRSAgent> logging to http://www.w3.org/2016/08/31-aria-test-irc
21:01:57 <fesch> https://www.w3.org/dpub/docs/scribing-guidelines.xhtml
21:02:01 <ShaneM> ShaneM has joined #aria-test
21:02:07 <jongund> trackbot, start meeting
21:02:09 <trackbot> RRSAgent, make logs world
21:02:11 <trackbot> Zakim, this will be
21:02:11 <Zakim> I don't understand 'this will be', trackbot
21:02:12 <trackbot> Meeting: Accessible Rich Internet Applications Working Group Teleconference
21:02:12 <trackbot> Date: 31 August 2016
21:02:32 <jongund> Zakim, this is ARIA Testing
21:02:32 <Zakim> got it, jongund
21:03:35 <joanie> present+ Joanmarie_Diggs
21:03:42 <fesch> present+
21:03:49 <fesch> present+ fesch
21:04:12 <ShaneM> is the code in the topic correct?
21:05:57 <jongund> Scribe: cynthia
21:06:23 <ShaneM> ShaneM has changed the topic to: ARIA Testing meeting 314 913 934
21:06:25 <jongund> Chair: jongund
21:06:59 <jongund> Meeting: ARIA Testing
21:07:00 <cyns> cyns has joined #aria-test
21:07:19 <cyns> scribe: cyns
21:07:23 <jongund> scribe: cyns
21:08:28 <cyns> zakim, take up next item
21:08:28 <Zakim> I see nothing on the agenda
21:08:51 <cyns> topic: test case updates
21:09:01 <cyns> TOPIC: test case updates
21:09:27 <jongund> https://www.w3.org/wiki/ARIA_1.1_Testable_Statements
21:09:40 <cyns> jg: do we have any update on that?
21:10:05 <cyns> fe: gave state and property statements to John F's team
21:10:19 <cyns> fe: working on script that turns wiki tables to json object
21:10:29 <jongund> https://www.w3.org/wiki/ARIA_1.1_Testable_Statements_JSON
21:11:46 <cyns> fe: this may be hard to use when there is a complex tree
21:12:01 <cyns> fe: looking for feedback on how usable the json is
21:12:59 <dbrett> dbrett has joined #aria-test
21:13:13 <cyns> https://www.w3.org/wiki/ARIA_1.1_Testable_Statements_JSON
21:13:32 <cyns> jg: is there validation?
21:13:36 <cyns> fe: no
21:13:45 <cyns> jg: script available?
21:13:49 <cyns> fe: will post soon
21:14:23 <cyns> jg: is this ok for shane?
21:14:27 <cyns> sm: yes
21:14:48 <cyns> jg: will find out what John F and team is doing
21:15:11 <jongund> https://www.w3.org/wiki/index.php?title=ARIA_1.1_Testable_Statements&action=history
21:16:29 <cyns> TOPIC: architecture
21:16:45 <cyns> jg: talked to shane about wpt solution strategy
21:16:54 <cyns> jg: also microsoft strategy
21:17:15 <cyns> jg: key to both is accessibility api shims that reflect back accessibility info to testing framework
21:17:49 <cyns> jg: is the shim self contained in msft architecture?
21:18:03 <Rich> Rich has joined #aria-test
21:18:10 <cyns> jg: I have a student who is going to start working on msaa/ia2 shim
21:18:17 <cyns> jg: what shane needed is pretty simple
21:18:47 <cyns> db: it is self contained. it runs locally. spins up pages through web driver, then evaluates against requirements.
21:19:08 <cyns> db: requirements are currently hard-coded, but will be bringing in from json.
21:19:26 <cyns> db: looking at output in a way wpt can consume
21:19:54 <cyns> db: we have api's in c# that allow us to talk to uia, which then talks to the browser.
21:20:07 <cyns> db: when you talk to uia, what is it giving you back? a11y tree info?
21:20:26 <cyns> db: yes, accessibility tree objects
21:20:44 <cyns> jg: what change is needed to use ia2 in your driver?
21:21:06 <cyns> db: designed with other api/browser in mind. it's factored out.
21:21:41 <cyns> jg: shane, what is your approach
21:22:26 <cyns> sm: wpt architecture is complicated. because it's desinged to run in places where it's hard to do anything cool, like locally running code
21:23:05 <jongund> https://www.w3.org/wiki/ARIA_1.1_Automated_Testing
21:23:35 <cyns> sm: wpt has test web pages with test code embedded within them.
21:23:45 <jongund> https://www.w3.org/wiki/ARIA_1.1_Automated_Testing#Proposed_Architecture
21:23:47 <cyns> sm: can use webdriver but does't need it
21:24:38 <cyns> sm: can be a lot of interaction between test server and child window
21:25:21 <cyns> sm: idea is to have child window talk to wpt server as a relay to talk to the shim
21:26:01 <cyns> sm: can use in differnt environemtns, like android
21:26:06 <cyns> cs: not needed for aria 1.1
21:26:11 <cyns> sm: true, but good for futre
21:27:29 <cyns> sm: is msft tool pulling expected results from child window?
21:27:36 <cyns> db: not yet
21:28:37 <cyns> sm: do we want evaluative smarts in test window, so they have to be written once, or in logic that is part of platform specific
21:28:48 <cyns> sm: is it doing the evaluation?
21:28:52 <cyns> db: yes
21:29:18 <cyns> sm: are you oepn to pulling the logic up into a javascript library that might live in the test?
21:29:46 <cyns> db: I think that's possible, but a lot of work
21:30:38 <cyns> db: client code can run anywhere c# can be compiled (which is most platforms)
21:31:36 <cyns> sm: in the diagram, see "magic".  Is it possible for get the needed info from the client window?
21:31:42 <cyns> db: yes, but file would be easier
21:32:56 <cyns> sm: easier for me, maybe not for you... if the local at shim had a port, then child test window could instruct it what to do from localhost
21:33:02 <cyns> db: that makes sense to me
21:33:31 <cyns> sm: that would let me remove the server-side part on the right
21:33:39 <cyns> cs: details of "magic"
21:34:00 <cyns> sm: RESTful api, with window title and criteria, and yes or no.
21:34:08 <cyns> sm: wpt would call that subtests
21:34:37 <dbrett> dbrett has joined #aria-test
21:34:41 <ShaneM> individual results for each assertion evaluated per test window.
21:34:45 <fesch> present+ fesch
21:35:31 <Rich> present+ Rich_Schwerdtfeger
21:35:42 <cyns> sm: in wpt world, when result came back, test runner can tell system to open next test
21:36:33 <cyns> jg: would that make the microsoft and wpt tools more compatible
21:36:52 <cyns> db: yes. changing to take input from a port rather than file shouldn't be too hard
21:37:06 <cyns> sm: like that
21:37:37 <cyns> sm: my goal is that this effort is useful going forward for testing user agents as they are developed, like the rest of wpt
21:38:13 <cyns> sm: the easier it is for browser developers to know when aria is broken and fix it, the better
21:38:37 <cyns> sm: stanley, can we look at integration points
21:38:40 <cyns> sh: yes
21:39:00 <cyns> jd: sounds interesting, but need to look at code
21:39:31 <cyns> sm: there is some setup for full-scale automated testing. you'll have to start up the shim and the browser.
21:39:34 <cyns> jd: that's easy
21:40:23 <cyns> (discussion of configuration requirements for setting up wpt)
21:42:04 <cyns> jg: I have a student who will be working on IA2. she's looking at wpt now and accprobe.
21:42:15 <cyns> jg: next step will be looking at microsoft test harness
21:42:39 <cyns> db: already prototyped
21:42:53 <cyns> cs: branch is public
21:43:16 <dbrett> https://github.com/microsoftEdge/a11y/tree/aria
21:44:04 <cyns> sm: I'll do the work for the wpt pieces
21:44:27 <cyns> db: it can read any json
21:44:48 <cyns> fe: I can tweak too
21:47:26 <jongund> present+ jongund
21:47:38 <jongund> rrsagent, draft minutes
21:47:38 <RRSAgent> I have made the request to generate http://www.w3.org/2016/08/31-aria-test-minutes.html jongund
21:48:12 <jongund> rrsagent, make logs world
21:53:22 <ShaneM> I will need to mock up a listener so that I can demonstrate it.  There aren't any examples of this yet.  A lot of moving parts here.
21:55:24 <joanie> ShaneM: but what I need to do when I have a spare cycle is read and do this, correct? https://github.com/w3c/web-platform-tests/blob/master/README.md
21:56:06 <ShaneM> joanie: yes that will get the basic platform in place.
21:56:14 <joanie> cool tyvm
21:56:23 <joanie> i'll try to get to that this weekend
21:56:34 <ShaneM> awesome.  I will be around
23:00:21 <jongund> jongund has joined #aria-test
23:03:00 <jongund_> jongund_ has joined #aria-test