20:58:10 <RRSAgent> RRSAgent has joined #aria-test
20:58:10 <RRSAgent> logging to http://www.w3.org/2016/08/24-aria-test-irc
20:58:30 <jongund> trackbot, start meeting
20:58:32 <trackbot> RRSAgent, make logs world
20:58:34 <trackbot> Zakim, this will be
20:58:34 <Zakim> I don't understand 'this will be', trackbot
20:58:35 <trackbot> Meeting: Accessible Rich Internet Applications Working Group Teleconference
20:58:35 <trackbot> Date: 24 August 2016
20:58:46 <jongund> zakim, this is ARIA Testing
20:58:46 <Zakim> got it, jongund
20:58:58 <jongund> chair: jongund
20:59:08 <jongund> Meeting: ARIA Testing
20:59:49 <jongund> agenda: https://lists.w3.org/Archives/Public/public-aria-test/2016Aug/0008.html
21:02:38 <joanie> present+ Joanmarie_Diggs
21:02:48 <jongund> present+ jongund
21:03:11 <ShaneM> ShaneM has joined #aria-test
21:03:45 <cyns> cyns has joined #aria-test
21:06:08 <ShaneM> https://www.w3.org/wiki/ARIA_1.1_Testable_Statements#alert
21:06:43 <JF> JF has joined #aria-test
21:07:02 <ShaneM> Scribe: ShaneM
21:07:30 <ShaneM> Chair: joanie
21:07:36 <ShaneM> Chair: jongund
21:07:42 <ShaneM> rrsagent, make minutes
21:07:42 <RRSAgent> I have made the request to generate http://www.w3.org/2016/08/24-aria-test-minutes.html ShaneM
21:07:57 <JF> zakim, who is here?
21:07:57 <Zakim> Present: Joanmarie_Diggs, jongund
21:07:59 <Zakim> On IRC I see JF, cyns, ShaneM, RRSAgent, Zakim, jongund, MichaelC, joanie, trackbot
21:08:02 <ShaneM> rrsagent, make logs public
21:08:14 <JF> Present+ JF
21:08:23 <ShaneM> present+ cyns
21:08:38 <ShaneM> present+ ShaneM
21:08:44 <ShaneM> present+ jongund
21:08:58 <ShaneM> rrsagent, make minutes
21:08:58 <RRSAgent> I have made the request to generate http://www.w3.org/2016/08/24-aria-test-minutes.html ShaneM
21:09:09 <ShaneM> Topic: Test Case Format
21:09:27 <ShaneM> fred: putting things in the testable statement document
21:09:31 <ShaneM> ... using tables
21:09:47 <ShaneM> There are some gaps.  I don't know what a pattern is or what it means to say something is not in a pattern.
21:09:53 <jongund> https://www.w3.org/wiki/ARIA_1.1_Testable_Statements
21:09:55 <ShaneM> cyns: I didn't understand the question
21:10:51 <ShaneM> fesch: Under some statements it says "not in a certain pattern"
21:10:56 <ShaneM> cyns: Yes, that does mean something...
21:11:12 <fesch> fesch has joined #aria-test
21:11:20 <fesch> https://www.w3.org/wiki/ARIA_1.1_Testable_Statements#menuitemradio
21:11:27 <ShaneM> ShaneM: It may well, but not to any sort of automation system. We don't know what patterns are.
21:11:40 <ShaneM> fesch: Go look at "cell"
21:11:47 <jongund> https://www.w3.org/wiki/ARIA_1.1_Testable_Statements#cell
21:12:22 <ShaneM> ... it says "selection pattern TODO" or "invoke pattern TODO"
21:12:44 <ShaneM> cyns: basically it means that the invoke pattern is implemented with the default values
21:13:13 <ShaneM> ShaneM: if there were a pointer to what an invoke pattern is, then we are laughing.
21:13:25 <ShaneM> cyns: I will provide a link to the documentation
21:13:52 <cyns> https://msdn.microsoft.com/en-us/library/windows/desktop/ee671194%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396
21:13:57 <ShaneM> fesch: describes the layout
21:14:27 <ShaneM> will get rolled up into a JSON object.
21:14:46 <ShaneM> as long as it is interpretable by whatever the automation framework is, we should be good
21:15:12 <ShaneM> jongund: Why do we need MSAA?
21:15:23 <ShaneM> fesch: because it is in the table.
21:15:43 <ShaneM> jongund: do we really need to support this?  It seems to be very limited.
21:15:55 <ShaneM> cyns: an MSAA only implementation would be largely unusable.
21:16:20 <ShaneM> fesch: if there is a platform that is mixed, then we would need to evaluate both in order to cover
21:16:29 <ShaneM> cyns: Edge is only UIA only
21:17:16 <ShaneM> fesch: IAcessible2 is what Firefox.  MSAA is legacy.  IAccessible2 is not a MS product that is implemented in FF and Chrome and maybe others.  UIA is the current MS A11Y API.
21:17:51 <ShaneM> jongund: question about "if given"
21:18:06 <ShaneM> fesch: that's not used at all in the conversion
21:19:01 <ShaneM> ... as far as I know this is how the testable statements are supposed to be written
21:19:44 <ShaneM> ShaneM: I recommend we maintain it
21:19:53 <ShaneM> jongund: the information is in the table
21:20:04 <ShaneM> ShaneM: then... no need to duplicate
21:20:24 <ShaneM> jongund: if it is something we are not using then it is not adding any new information.
21:20:42 <ShaneM> fesch: it doesn't add any value
21:21:29 <ShaneM> cyns: the information in the "then" clause should be translated into the table too
21:21:57 <ShaneM> fesch: I have put notes in about information that is missing.
21:22:46 <ShaneM> ... rich gave me some information about where to find other information in a branch.  that's mroe than I wanted to do right now.
21:23:14 <ShaneM> jongund: should the test include information about these non-inherited attributes?
21:23:23 <ShaneM> fesch: The reason they are in there because this is new.
21:24:14 <ShaneM> ShaneM: I dont think it matters
21:24:37 <ShaneM> cyns: you would want to know which thing failed
21:24:56 <ShaneM> ShaneM: In WPT each assertion comes out in its own subtest result so you get the information regardless
21:25:14 <ShaneM> cyns: You can use the role without the properties.  They are not required.
21:25:36 <ShaneM> jongund: Are browsers required to compute indexes or is it just a should?
21:26:02 <ShaneM> ... most browsers don't do it automatically.  So ATs need to do it independently.
21:26:28 <ShaneM> ... What do we need to do to get out of ARIA 1.1 CR testing.  The tests should be minimal.
21:26:55 <ShaneM> jongund: would it make more sense for MS to have multiple tests?
21:27:21 <ShaneM> ???: what would this mean if there were multiple tests?
21:27:32 <ShaneM> jongund: each code snippet will generate a test file.
21:28:49 <ShaneM> ... Does MS framework need separate tests to get results?
21:29:03 <ShaneM> joanie: I am waiting for a group decision as to direction before I put in too much work.
21:29:09 <ShaneM> jongund: we are going to talk about that next.
21:29:22 <ShaneM> fesch: there are thigns not in the Core AAM and it would be great if we could get them in.
21:29:31 <ShaneM> ... colcount, rowcount, some roles.
21:30:02 <ShaneM> jongund: all the new stuff is on rich's branch and it not yet merged because some people have not signed off on it yet.
21:30:14 <ShaneM> s/jongund/joanie/
21:30:26 <ShaneM> fesch: it is just hard to do things without the branch merged.
21:30:46 <ShaneM> fesch: there are some where there is no change in the mappings.  So there is no ARIA 1.1 testing needed.
21:30:51 <ShaneM> ... but I might be wrong.
21:31:49 <ShaneM> fesch: Some things support a couple of new attributes.  When I made a test I made the role with the new attributes.  For example the only thing to test for menuitem is with the attributes...
21:32:45 <JF> +1 to cyns
21:33:21 <ShaneM> fesch: we are only checking the deltas...
21:34:37 <ShaneM> JF: we may apply a little reason and logic to the question of "what is tested when something changed from 1.0"
21:35:15 <ShaneM> cyns: it seems a little easier to just enumerate the possibilities for the changes roles than trying to figure out what is changed.
21:35:34 <ShaneM> JF: I sort of agree with Shane that we should ensure things aren't broken.  Especially since it is automated.
21:36:26 <ShaneM> fesch: Does anyone have trouble with the format as it is?
21:36:47 <ShaneM> cyns: the only question is "do we want to maintain wiki tables"?  Would a nested list be easier to maintain?
21:38:04 <ShaneM> JF: I find it easier to look at HTML than wiki tables
21:38:19 <ShaneM> fesch: The problem is that some tables are uneven.
21:38:29 <ShaneM> cyns: I am going to look for a wysiwyg table editor
21:38:34 <ShaneM> fesch: its a lot of bookkeeping
21:38:46 <ShaneM> jongund: we need to move on...
21:39:05 <ShaneM> ... fesch you got started on this.  Can some of JF people can get into this now?
21:39:40 <ShaneM> JF: Rich has some statements that need to be done.  We are going to look at that, then feed it back to the people in India, and then see how they do with a few example
21:41:27 <ShaneM> fesch: all of section 3 is availabhle to be done.  aria properties and state testable statements.  fesch is working on 4.
21:42:11 <ShaneM> ... if someone wants to work on "none".  I think it would be weird to have only one.
21:42:18 <ShaneM> cyns: yes.  It needs multiple.
21:43:02 <ShaneM> jongund: Can JF pleease share with the list what people are wokring on so there are no collisions or wasted effort.
21:43:11 <ShaneM> ... tomorrow on the ARIA call maybe we can sync up.
21:43:28 <ShaneM> Topic: Automation Platform
21:43:44 <joanie> q+
21:43:54 <ShaneM> jongund: MS is committed to their tech but only for UIA
21:44:22 <ShaneM> ... where are we at for other platforms?  joanie is committed to implementing on ATK.
21:44:24 <ShaneM> q?
21:44:28 <ShaneM> ack joanie
21:44:47 <joanie> https://www.w3.org/WAI/ARIA/wiki/ARIA_1.1_Changes
21:44:50 <ShaneM> joanie: the testable statements page is based upon the change log.
21:45:12 <ShaneM> This other page is way more detailed.  That's the one that has all of the good information.
21:45:45 <ShaneM> ... I basically did a giant diff to build that other page.
21:45:55 <ShaneM> jongund: does that page need to be merged?
21:46:04 <ShaneM> joanie: you need to at least look at it very carefully.
21:46:21 <ShaneM> fesch: who created the testable statements page?
21:46:24 <ShaneM> jongund: I did.
21:46:31 <joanie> ack me
21:46:36 <ShaneM> ... I will go through and interate joanie work.
21:47:01 <ShaneM> s/interate/integrate/
21:47:15 <ShaneM> jongund: thank you for all that work joanie
21:48:44 <ShaneM> ShaneM: I thought we were going to try to get WPT to drive tests
21:49:02 <ShaneM> cyns: I thought we were going to use the MS stuff and attempt to integrate with WPT reporting if we had time.
21:50:32 <ShaneM> jongund: ShaneM do you have examples?  We dont have a communication channel between the AT layer and the test window.
21:53:51 <ShaneM> jongund: Are there things in the changes to aria 1.1 that would require more complex "interaction" or state change testing?
21:54:26 <ShaneM> joanie: There might be things.  I don't know.
21:54:38 <ShaneM> cyns: We didnt test them in 1.0 so why would we in 1.1?
21:54:56 <ShaneM> jongund: there are some events that we might need to test.  Does UIA have those?
21:56:23 <ShaneM> jongund: we are not going to finalize this plan today.  can we meet next week?
22:01:40 <ShaneM> ShaneM: let's continue the conversatin via email.
22:01:50 <ShaneM> jongund: let's talk in a week.
22:01:52 <ShaneM> rrsagent, make minutes
22:01:52 <RRSAgent> I have made the request to generate http://www.w3.org/2016/08/24-aria-test-minutes.html ShaneM