See also: IRC log
this is DPUB-ARIA
DPUB-AAM published this morning at http://www.w3.org/TR/2015/WD-dpub-aam-1.0-20151203/
<MichaelC> PF test harness
michael: we have DPUB roles and
AAM
... we care more about getting roles to CR, but they are
dependant on AAM
... expected result of roles is that it is exposed in
accessibilty mapping X as X
... when testing a specific role, using a browser with
Accessibility API X
... and seeing the roles implemented in at least 2 APIs should
get us to CR
tzviya: summarizes michael's comments: proof of implementation for DPUB roles is that any role is implemented in any 2 browser/API combo
ivan: the goal of CR is to prove
implementability, but there is also goal to show usage
... are we also under obligation to show that the roles are in
use?
... similar to semantic web. How do you check a vocab?
michael: we are first concerned
with exiting CR, partly by pre-negotiating
... we propose our criteria to the director, and then we meet
the goals
... We should care about implementation as well. For ARIA 1.0
we didn't have the resources, and let's look at the testing
tzviya: i know of organizations IRL that will use this vocab, so we can provide real examples in the wild
shane: we don't need to have examples to exit CR, but if we do, best method is to demo with schemas in validator.nu
michael: showing manual test procedure for Aria 1
<MichaelC> https://www.w3.org/WAI/PF/testharness/testcases/edit?testsuite_id=1&testcase_id=183
<scribe> scribenick: tzviya
michael: this is manual testing,
but it will likely be in use for 1.1 as well (at least in
part)
... must indicate exactly what API, browser, etc testing
<Zakim> ShaneM, you wanted to discuss other ways to test a vocabulary
michael: having 2 passes is
enough to pass CR
... this can also be used for bug fixes
tzviya: who did the testing for ARIA 1.0?
michael: memebers of PF,
including Jon Gunderson and others from U of Illinois
... Rich and Cynthia did a lot of testing as well
... there are 500 - 800
ivan: Other WGs had around 1000
tests as well, but the tests had reports, so the tests pushed
information to the implementors
... this may be different though
... At the end of CR period, we need to put togethor a
report
<MichaelC> ARIA 1.0 test report
michael: this is php pulled from mySQL db
agenda next
<ShaneM> It should be trivial to push results from automation into a database - set up a web service
michael: this stuff should be
automatable
... looking for token, varies by API
... the a11y API inspector tools are scriptable, so can
automated by platform, but it would be custom by platform
... ARIA WG will talk about this further next week
<MichaelC> Test the Web Forward
We don't know how much automation we'll be able to do in next few months bc this is ongoing
<Zakim> ShaneM, you wanted to suggest that we not make automation dpub's problem
shane: it's not DPUB's problem to figure out autotmation. We need to solve this for everyone.
tzviya: next steps - it looks
like automation is a bit off
... how do we move this forward?
... it would be helpful to test all the AAMs together and tap
into eager student bodies
<Zakim> ShaneM, you wanted to ask if there is a tool to extract the mappings automatically? (assuming the answer is yes)
shane: is there a way to extract mappings auto?
michael: we use xslt
shane: does respec produce xhtml? does respec spit out something reasonable?
<ShaneM> ACTION: Shane to script output of assertions from ReSpec into JSON [recorded in http://www.w3.org/2015/12/03-dpub-aria-minutes.html#action01]
<trackbot> Created ACTION-1756 - Script output of assertions from respec into json [on Shane McCarron - due 2015-12-10].
shane: testing of DPUB is much easier than core testing, because it's just role testing
tzviya: we have a note in current draft of DPUB that we can extend the vocab. At what point do we freeze the vocab?
<ivan> +1 to Michael
michael: freeze when you enter
CR, but also goes to question of vocab extensibility
... do something of a virtual last call
<ShaneM> Also need to know when the AAM is stable...
<ShaneM> (in other words, do you trust the mappings?)
michael: we need to know when the DPUB vocab is set, when we are ready to test
ivan: for each of our statements, we must have a test file, using the role
<ShaneM> I envision a tool chain
ivan: shane's script should create HTML files from the code samples in the document
tzviya: we plan to provide more complicated, nested samples
<Zakim> MichaelC, you wanted to talk about test file format
michael: test files are the smallest granular chunks possible
tzviya: i'm thinking that we should leave existing examples, and add appendix to demo comprehensive examples
<ShaneM> <a>user
agents</a>
...timeline: 2PWD just published - we should be able to comment
on state of the vocab in January after feedback
<Zakim> ShaneM, you wanted to mention something about term references
This is scribe.perl Revision: 1.144 of Date: 2015/11/17 08:39:34 Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: RRSAgent_Text_Format (score 1.00) Succeeded: s/AAm/AAM/ Found ScribeNick: tzviya Inferring Scribes: tzviya Present: ShaneM Tzviya_Siegman janina Michael ivan rich Rich_Schwerdtfeger WARNING: No meeting title found! You should specify the meeting title like this: <dbooth> Meeting: Weekly Baking Club Meeting Agenda: https://lists.w3.org/Archives/Public/public-dpub-aria/2015Dec/0001.html WARNING: No meeting chair found! You should specify the meeting chair like this: <dbooth> Chair: dbooth Got date from IRC log name: 03 Dec 2015 Guessing minutes URL: http://www.w3.org/2015/12/03-dpub-aria-minutes.html People with action items: shane[End of scribe.perl diagnostic output]