13:56:18 <RRSAgent> RRSAgent has joined #dpub-aria
13:56:18 <RRSAgent> logging to http://www.w3.org/2015/12/03-dpub-aria-irc
13:56:40 <tzviya> rrsagent, make logs public
14:00:00 <ShaneM> Present+ ShaneM
14:00:21 <tzviya> present+ Tzviya_Siegman
14:01:02 <tzviya> rrsagent, this is DPUB-ARIA
14:01:02 <RRSAgent> I'm logging. I don't understand 'this is DPUB-ARIA', tzviya.  Try /msg RRSAgent help
14:01:20 <tzviya> this is DPUB-ARIA
14:01:35 <ShaneM> zakim, this is DPUB-ARIA
14:01:35 <Zakim> got it, ShaneM
14:02:03 <janina> present+ janina
14:05:03 <tzviya> agenda: https://lists.w3.org/Archives/Public/public-dpub-aria/2015Dec/0001.html
14:05:54 <tzviya> present+ Michael
14:06:01 <tzviya> Present+ ivan
14:06:10 <tzviya> topic: testing DPUB-AAm
14:06:31 <tzviya> s/AAm/AAM
14:08:47 <tzviya> DPUB-AAM published this morning at http://www.w3.org/TR/2015/WD-dpub-aam-1.0-20151203/
14:08:55 <MichaelC> -> https://www.w3.org/WAI/PF/testharness/ PF test harness
14:09:33 <richardschwerdtfeger> richardschwerdtfeger has joined #dpub-aria
14:09:35 <tzviya> michael: we have DPUB roles and AAM
14:09:53 <tzviya> ...we care more about getting roles to CR, but they are dependant on AAM
14:10:04 <tzviya> present+ rich
14:10:37 <tzviya> michael: expected result of roles is that it is exposed in accessibilty mapping X as X
14:11:13 <ivan> q+
14:11:24 <tzviya> ...when testing a specific role, using a browser with Accessibility API X
14:12:58 <tzviya> ... and seeing the roles implemented in at least 2 APIs should get us to CR
14:13:19 <ivan> present+ ivan
14:15:05 <tzviya> tzviya: summarizes michael's comments: proof of implementation for DPUB roles is that any role is implemented in any 2 browser/API combo
14:15:10 <tzviya> ack ivan
14:15:46 <tzviya> ivan: the goal of CR is to prove implementability, but there is also goal to show usage
14:16:16 <tzviya> ...are we also under obligation to show that the roles are in use?
14:16:42 <ShaneM> q+ to discuss other ways to test a vocabulary
14:16:44 <tzviya> ...similar to semantic web. How do you check a vocab?
14:17:29 <tzviya> michael: we are first concerned with exiting CR, partly by pre-negotiating
14:18:20 <tzviya> ...we propose our criteria to the director, and then we meet the goals
14:19:29 <tzviya> ...We should care about implementation as well. For ARIA 1.0 we didn't have the resources, and let's look at the testing
14:20:02 <tzviya> tzviya: i know of organizations IRL that will use this vocab, so we can provide real examples in the wild
14:20:15 <MichaelC> agenda+ Test Automation
14:21:01 <tzviya> shane: we don't need to have examples to exit CR, but if we do, best method is to demo with schemas in validator.nu
14:21:34 <tzviya> michael: showing manual test procedure for Aria 1
14:21:38 <MichaelC> https://www.w3.org/WAI/PF/testharness/testcases/edit?testsuite_id=1&testcase_id=183
14:21:42 <tzviya> scribenick: tzviya
14:22:54 <tzviya> michael: this is manual testing, but it will likely be in use for 1.1 as well (at least in part)
14:23:24 <tzviya> ...must indicate exactly what API, browser, etc testing
14:23:44 <tzviya> q?
14:23:59 <ShaneM> ack ShaneM
14:23:59 <Zakim> ShaneM, you wanted to discuss other ways to test a vocabulary
14:25:14 <tzviya> michael: having 2 passes is enough to pass CR
14:25:26 <tzviya> ... this can also be used for bug fixes
14:26:14 <tzviya> tzviya: who did the testing for ARIA 1.0?
14:26:44 <tzviya> michael: memebers of PF, including Jon Gunderson and others from U of Illinois
14:26:54 <richardschwerdtfeger> present+ Rich_Schwerdtfeger
14:27:52 <tzviya> ...Rich and Cynthia did a lot of testing as well
14:27:54 <ivan> q+
14:28:22 <tzviya> ...there are 500 - 800
14:28:26 <tzviya> ack iv
14:29:36 <tzviya> ivan: Other WGs had around 1000 tests as well, but the tests had reports, so the tests pushed information to the implementors
14:29:43 <tzviya> ...this may be different though
14:30:03 <tzviya> ...At the end of CR period, we need to put togethor a report
14:30:11 <MichaelC> -> https://www.w3.org/WAI/PF/testharness/testreport?testsuite_id=1 ARIA 1.0 test report
14:31:57 <tzviya> michael: this is php pulled from mySQL db
14:32:09 <tzviya> agenda next
14:32:17 <tzviya> zakim, next item
14:32:17 <Zakim> agendum 1. "Test Automation" taken up [from MichaelC]
14:32:18 <ShaneM> It should be trivial to push results from automation into a database - set up a web service
14:32:41 <tzviya> michael: this stuff should be automatable
14:32:56 <tzviya> ...looking for token, varies by API
14:33:40 <ShaneM> q+ to suggest that we not make automation dpub's problem
14:33:44 <tzviya> ...the a11y API inspector tools are scriptable, so can automated by platform, but it would be custom by platform
14:34:30 <tzviya> ...ARIA WG will talk about this further next week
14:35:16 <MichaelC> -> http://testthewebforward.org/docs/test-format-guidelines.html Test the Web Forward
14:35:18 <tzviya> We don't know how much automation we'll be able to do in next few months bc this is ongoing
14:36:01 <tzviya> ack shane
14:36:01 <Zakim> ShaneM, you wanted to suggest that we not make automation dpub's problem
14:36:34 <tzviya> shane: it's not DPUB's problem to figure out autotmation. We need to solve this for everyone.
14:39:11 <tzviya> tzviya: next steps - it looks like automation is a bit off
14:39:14 <ShaneM> q+ to ask if there is a tool to extract the mappings automatically? (assuming the answer is yes)
14:39:42 <tzviya> ...how do we move this forward?
14:40:41 <tzviya> tzviya: it would be helpful to test all the AAMs together and tap into eager student bodies
14:40:45 <tzviya> ack shan
14:40:45 <Zakim> ShaneM, you wanted to ask if there is a tool to extract the mappings automatically? (assuming the answer is yes)
14:41:04 <tzviya> shane: is there a way to extract mappings auto?
14:41:12 <tzviya> michael: we use xslt
14:41:49 <tzviya> shane: does respec produce xhtml? does respec spit out something reasonable?
14:42:45 <ShaneM> ACTION: Shane to script output of assertions from ReSpec into JSON
14:42:45 <trackbot> Created ACTION-1756 - Script output of assertions from respec into json [on Shane McCarron - due 2015-12-10].
14:44:02 <tzviya> shane: testing of DPUB is much easier than core testing, because it's just role testing
14:45:11 <tzviya> tzviya: we have a note in current draft of DPUB that we can extend the vocab. At what point do we freeze the vocab?
14:45:32 <ivan> +1 to Michael
14:45:45 <tzviya> michael: freeze when you enter CR, but also goes to question of vocab extensibility
14:46:25 <tzviya> ...do something of a virtual last call
14:47:14 <ivan> q+
14:47:47 <ShaneM> Also need to know when the AAM is stable...
14:48:01 <ShaneM> (in other words, do you trust the mappings?)
14:48:46 <tzviya> michael: we need to know when the DPUB vocab is set, when we are ready to test
14:48:52 <tzviya> ack iv
14:49:43 <tzviya> ivan: for each of our statements, we must have a test file, using the role
14:50:10 <ShaneM> I envision a tool chain
14:50:31 <tzviya> ...shane's script should create HTML files from the code samples in the document
14:52:42 <tzviya> tzviya: we plan to provide more complicated, nested samples
14:53:07 <MichaelC> q+ to talk about test file format
14:53:29 <tzviya> ack mi
14:53:29 <Zakim> MichaelC, you wanted to talk about test file format
14:53:54 <tzviya> michael: test files are the smallest granular chunks possible
14:56:22 <tzviya> tzviya: i'm thinking that we should leave existing examples, and add appendix to demo comprehensive examples
14:56:36 <ShaneM> q+ to mention something about term references
14:57:26 <ShaneM> <a>user agents</a>
14:58:18 <tzviya> ...timeline: 2PWD just published - we should be able to comment on state of the vocab in January after feedback
14:58:24 <tzviya> ack shane
14:58:24 <Zakim> ShaneM, you wanted to mention something about term references
14:58:45 <tzviya> rrsagent, make minnutes
14:58:46 <RRSAgent> I'm logging. I don't understand 'make minnutes', tzviya.  Try /msg RRSAgent help
14:59:01 <tzviya> rrsagent, make miutes
14:59:01 <RRSAgent> I'm logging. I don't understand 'make miutes', tzviya.  Try /msg RRSAgent help
14:59:09 <tzviya> rrsagent, make minutes
14:59:09 <RRSAgent> I have made the request to generate http://www.w3.org/2015/12/03-dpub-aria-minutes.html tzviya
15:02:47 <pkra> pkra has joined #dpub-aria
15:07:46 <tzviya> tzviya has joined #dpub-aria
15:13:27 <richardschwerdtfeger> richardschwerdtfeger has joined #dpub-aria
15:50:37 <ShaneM> ShaneM has joined #dpub-aria
16:01:23 <richardschwerdtfeger> richardschwerdtfeger has joined #dpub-aria
16:30:30 <ShaneM> ShaneM has joined #dpub-aria
16:44:02 <richardschwerdtfeger> richardschwerdtfeger has joined #dpub-aria
17:23:17 <richardschwerdtfeger> richardschwerdtfeger has joined #dpub-aria