Evaluation and Repair Tools Working Group Teleconference

EARL 1.0 Test Suite

<shadi> http://www.w3.org/WAI/ER/EARL10/tests/

Shadi: We need to have test suites in order to enter candidate recommendation state
... Without that we cannot finish EARL
... question for tool developers: does your tool already generate EARL reports?

Kostas: we generate EARL reports already
... regarding the test suite: what is actually the test suite? Do we need to create new tests or just put there sample reports of the tools?

<shadi> http://www.w3.org/WAI/ER/EARL10/tests/test0001.rdf

Shadi: Ideally every report would have the input (web page) along with it
... So we need sample input AND sample output
... If tool developers could provide such we could have a look at them in the group
... Input and output only contain anonymized information

<shadi> http://www.w3.org/WAI/demos/bad/

<cstrobbe> AccessODF also generates EARL, but you need to extract it from the ODF files.

Shadi: We want a good set of tests, various aspects of EARL, not repeat features
... So that tool developers can test their tool if they generate good EARL reports

Kostas: we could generate such reports

Shadi: Please generate two examples and then we can discuss

CarlosV: We are very busy the next weeks, after that we will concentrate on EARL reporting
... Our tool has an export mechanism of the object model to EARL, but it is not up to date

Shadi: Christophe, can you send us an example ODF file that contains an EARL report?

Christophe: will do

Kostas: Can the EARL reports contain custom RDF classes?

Shadi: It depends on the type of the additional information, maybe subclassing would be ok
... But we need to see the individual reports

<kostas> there is a noise

Shadi: Samuel and Christophe and me would be reviewers
... CarlosV, Kostas and Philip would generate the EARL reports

Website Accessibility Conformance Evaluation Methodology

<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20120910

Shadi: first public draft was in March
... processed all public comments
... it's a complete document now
... we would like to publish working draft
... Two options for ERT-WG
... 1.) review in more detail before publishing
... 2.) publish as working draft and ERT-WG will review that

<kostas> i prefer the second choice

CarlosV: Would like to review before publishing but problem is the time

Shadi: any particular concerns?

CarlosV: I only read an early draft of it, which was to concise, but I assume that is not longer the case

Samuel: I would prefer the second choice as well

<kostas> it is better to become publishable asap for comments/reviews from all interested people

<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20120910#step5c

Shadi: we don't know if this section will stay inside the document
... this would be for review of ERT-WG

Kostas: would like to have more examples in the document

Shadi: it is on our todo list

<shadi> 29-30 October, Lyon, France

<shadi> Eval TF face-to-face meeting

<shadi> http://www.w3.org/WAI/ER/2011/eval/eval-tf#f2f

Shadi: would like to have comments before this F2F meeting
... ERT-WG members are invited as well

RESOLUTION: Approval for publication as an updated working draft

Techniques For Accessibility Evaluation And Repair Tools

<shadi> http://www.w3.org/TR/2000/WD-AERT-20000426

Shadi: one of the first deliverables of this group
... idea was to develop ideas for evaluation tools
... a lot of this work has been integrated into techniques document of WCAG 2.0

<shadi> http://www.w3.org/TR/WCAG20-TECHS/

Shadi: basic objective is to inform tool developers how to check the accessibility of web content
... today there are many tools available but they implement WCAG in different ways and come to different results
... should be kind of cookbook
... evaluation in general
... how to implement checks, how they relate to success criteria and so on

Kostas: our tool is based on this document

Shadi: would you say that it would be of more help if this document would be more up to date

<shadi> AERT

Shadi: I'm referring to WCAG 2.0

Kostas: I'm referring to WCAG 2.0

<shadi> [[Status of This Document]]

CarlosV: isn't the document is totally obsolete?

Shadi: I think the techniques of WCAG may not be enough for tool developers
... You need combine different techniques, ....., WCAG 2.0 has a layer of logic on top of the techniques
... That is not all explained in the techniques document
... Title and content of the document may change but the general objective will stay the same

CarlosV: I think the documents form UWEM are more usable
... we follow this approach when developing our tools

Shadi: I think UWEM was addressing many different target user groups at the same time
... which is just a design question
... we try to break that down into more modules
... new document should address evaluation tool developers
... there will be overlap with UWEM, but hopefully it will be more standardized
... we would like to bring that document up to date
... next step would be to define the requirements of such a document

Next Meeting

<shadi> 19 September - tentative, probably not

<shadi> 26 September - confirmed