12:25:31 <RRSAgent> RRSAgent has joined #er
12:25:31 <RRSAgent> logging to http://www.w3.org/2015/04/15-er-irc
12:25:33 <trackbot> RRSAgent, make logs public
12:25:35 <trackbot> Zakim, this will be 3794
12:25:36 <trackbot> Meeting: Evaluation and Repair Tools Working Group Teleconference
12:25:36 <trackbot> Date: 15 April 2015
12:29:10 <Wilco> Wilco has joined #er
12:29:18 <shadi> zakim, call shadi-617
12:31:06 <Wilco> Hi Shadi
12:32:25 <carlos> carlos has joined #er
12:34:19 <shadi> scribe: shadi
12:35:07 <shadi> http://www.w3.org/WAI/ER/charter5
12:35:20 <samuelm> samuelm has joined #er
12:38:03 <shadi> SAZ: looking for ways to make our current work more relevant
12:38:23 <shadi> ...also explore the possibility of rule-set for conformance evaluation
12:38:38 <shadi> ...relates to auto-wcag community group
12:39:00 <shadi> WF: came together to develop common set of tests
12:39:15 <shadi> ...focusing on automated tests
12:39:22 <shadi> ...spent some time on format and approach
12:39:30 <shadi> ...progressing fairly slow
12:39:38 <shadi> ...but reviewing them thoroughly
12:39:49 <shadi> ...about 12 tests available
12:40:00 <shadi> ...and another 12 in progress
12:40:34 <shadi> ...core group of about 8 people
12:40:50 <shadi> ...most of them from EIII project
12:41:01 <shadi> ...EC-funded project that initiated the work
12:41:35 <shadi> ...some may be able to participate beyond the project duration
12:42:13 <shadi> CV: are there more test cases being developed within the project?
12:43:44 <shadi> WF: EIII develops an automated and a non-automated tool
12:44:17 <shadi> ...the second tool is for non-technical people
12:44:32 <shadi> ...all tests development within the community group
12:46:41 <shadi> SAZ: tests in auto-wcag is also called rule-sets
12:46:50 <shadi> ...but there are also test samples
12:47:03 <shadi> ...used to benchmark evaluation tools
12:47:16 <shadi> ...how interesting are these in general?
12:47:29 <shadi> CV: unfortunately less interest in general
12:47:42 <shadi> ...we are interested in transparency but other less
12:49:30 <shadi> SAZ: think two sides of the same coin
12:49:49 <shadi> ...need test samples to validate correct implementation of rule-sets
12:50:14 <shadi> WF: agree ... there is much confusion about tool results
12:50:25 <shadi> ...need a common set of rules
12:51:04 <shadi> CV: often asked how our tool compares to others
12:51:16 <shadi> ...who verifies that?
12:51:16 <shadi> ...big test suite?
12:55:02 <shadi> ...would help demonstrate quality
12:56:51 <shadi> SAZ: is this amount of work manageable?
12:57:08 <shadi> WF: need more people with dedicated time allocation
12:57:26 <shadi> ...work can be done reasonably fast if people invest time
12:57:39 <shadi> ...think lots of organizations interested
12:57:51 <shadi> ...have been talking with some people
12:58:41 <shadi> SAZ: maybe chunk-down
12:58:53 <shadi> ...like focus on automated and on HTML first
12:59:05 <shadi> WF: lots of low-hanging fruit
12:59:16 <shadi> ...like syntactic ones
12:59:25 <shadi> ...things like images was tough, for example
12:59:51 <shadi> CV: we face the same challenge
13:00:02 <shadi> ...but it needs a coordinated effort
13:00:16 <shadi> ...not manageable by a few alone
13:00:55 <shadi> SM: should be done little by little
13:02:13 <shadi> ...huge task
13:02:33 <shadi> ...should tackle specific techniques and aspects
13:02:48 <shadi> ...automated rules can be validated more easily
13:03:02 <shadi> ...test definitions can be benchmarked
13:03:37 <shadi> ...will fail unless we focus on specific parts
13:04:20 <shadi> ...think should be done within ERT WG
13:04:30 <shadi> ...but needs adequate participation
13:05:31 <shadi> http://www.w3.org/TR/AERT
13:07:35 <shadi> WF: would support move of auto-wcag work into ERT WG
13:07:46 <shadi> ...if sufficient organizations are interested
13:08:02 <shadi> ...EARL has been an important aspect in that work, though
13:08:26 <shadi> ...using it as the output format
13:08:44 <shadi> http://www.w3.org/WAI/ER/charter5
13:10:26 <shadi> SAZ: how would we document and present the test rules?
13:10:39 <shadi> WF: like the Techniques
13:10:55 <shadi> ...individual documents that are aggregated into one main document
13:11:16 <shadi> ...can use Github to develop collaboratively
13:11:43 <shadi> SAZ: what about the test samples?
13:13:07 <shadi> CV: that's the work we did in BenToWeb
13:13:20 <shadi> ...needs a test definition language
13:13:26 <shadi> ...have an XML schema for that
13:13:36 <shadi> ...could turn it into RDF
13:13:54 <shadi> ...associate every test description with a test rule
13:14:26 <Wilco_> Wilco_ has joined #er
13:14:43 <shadi> SAZ: think there may be similar development by other organizations
13:15:08 <carlos> https://github.com/webcc/bentoweb-wcag20-test-suite-v3
13:17:00 <shadi> SAZ: so could be like 4 main deliverables
13:17:29 <shadi> ...test rules, test samples, test case description, and test results format
13:17:50 <shadi> ...could be done in task forces too
13:20:00 <shadi> ...will try to develop a new draft charter for next week
13:22:42 <shadi> present: Carlos, Samuel, Wilco, Shadi
13:22:49 <shadi> chair: Shadi
13:22:55 <shadi> trackbot, end meeting
13:22:55 <trackbot> Zakim, list attendees
13:23:03 <trackbot> RRSAgent, please draft minutes
13:23:03 <RRSAgent> I have made the request to generate http://www.w3.org/2015/04/15-er-minutes.html trackbot
13:23:04 <trackbot> RRSAgent, bye
13:23:04 <RRSAgent> I see no action items