See also: IRC log
<Kathy> scribe: Kathy
Mary Jo - questionnaire open until today
... looks like from the results that is highly probably
that we will have a meeting
... if anyone has any time that does not work please add to the survey
Mary Jo - looks like there is a list of repositories
...
and some metadata defined
... there is a repository of test cases
https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/Testing_Resources
... does anyone have any feedback on this
Alistair - Wilco put in a format and you need to run that against test suite
... model is ok
... we are supposed to collect the repositories
... bentoweb does not have a pass / fail so someone will manually need
to go through
... AXE model is different
... I think Wilco wanted someone to own this
Shadi - issue is that bentoweb is not participating
... so
can we extract what we need
... we will need someone to write a conversion tool
Alistair - I took a look but didn't see where the pass/fail was
Shadi - pretty sure that the data is there, we may need
to ask them
... metadata is in XML
<shadi> http://www.bentoweb.org/refs/TCDL2.0.html
<shadi> https://www.w3.org/WAI/ER/tests/usingTCDL
shadi: w3c took a subset of these
<ChrisLoiselle> looks like this may be where it goes into pass fail <locations expectedResult="fail">
<ChrisLoiselle> 8.1.3. Linking Locations and Techniques
shadi: if a tool runs the file, it is
expected to produce
... so expectResult is the results - pass, fail, cannotTell and
notApplicable
Mary Jo - maybe we can ask them questions
Alistair - we have tried emailing and not sure how to push out note out to people
Kathy - may want to post to the interest group
Shadi - Jon Gunderson may have a test suite and they may have a person to join
Mary Jo - proposal for review
Kathy - wouldn't we want the publication to be on the same schedule as WCAG 2.0
Mary Jo - we should sync with the WCAG 2.0 schedule
... it would be good to talk to the chairs of AG to find
out schedule
... any other comments?
<shadi> scribe: shadi
AG: isn't benchmark same as running against
test suite?
... coverage and accuracy
MJM: thought it was you run your tools against
https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/ACT_Overview_-_What_is_ACT
AG: different ways to do things
... use Jasmine and Cucumber
... may not be good to write another tool
SAZ: maybe reuse these tools rather than rewrite
AG: don't have access to all tools
... vendors need to carry out the testing themselves
<MoeKraft> I need to drop for a daily scrum call.
SAZ: think idea was to provide a runner
AG: we're not set out to develop software
here
... many years of work
... we all do it differently
... best way is to just ask people to use the test suites and send in
the results
... optionally, of course
MJM: sounds reasonable
NEXT MEETING: MAY 8th