WCAG 2.0 Evaluation Methodology Task Force Teleconference

new editor draft

EV: new editor draft and disposition of comments are up, latest version 28th jan.
... almost all of the comments are adressed

survey 7 and 8

EV: survey 7 closed, survey 8 opened last tuesday
... hope to publish new editor draft monday
... things accepted in surveys will be included in next ED
... survey open until Saturday evening

make sampling optional

combining evaluations

<ericvelleman> Hi, phone just stopped, calling in now

EV: ongoing discussion about combining evaluations
... does Accessibility support need to be uniform on a website?

VC: It will be hard to qualify, especially with different evaluators that don't agree with each other in a crosschecking situation
... Concerned about conflicting results

DF: David MacDonald pointed out the difference between support amongst Assistive Technologies
... if some AT does not support some accessible function, it shouldnt be wrong because some AT does not support it

KHS: the code is more important than the AT used, necessary to document what is used

My line is gone, can someone scribe?

dialing in again

KHS: vitally important to identify all tools, AT, UA, OS
... if semantics are properly programmed, that is more important than tools, at

DF: very hard to define what AT you have to use per SC, and probably double work. Some criteria are easy to check on the basis of code

EV: we can add "require evaluators to describe what AT, etc they use"

KHS: you don't need to test every criteria with every tool

<Detlev> agree

EV: finish this discussion on list, as we seem to agree, will make a text suggestion

ME: we can maybe define the types of tools that can be used during evaluation

SAZ: not sure it is feasible to require to report every tool used per check
... It can be an option for reporting

<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130128#step1d

SAZ: natural part of evaluation to define and document tools used

DF: there is a certain risk if you define the tools used, if you do not use every tool on every sc
... We make comments describing interesting finds with AT

EV: I will suggest a text on the list

<Detlev> skip link - sure - thats part of the check in the technique!

KHS: important to also report the findings with AT and additional checks

I am disconnected again

<ericvelleman> Eric will make a proposal on the list. Please review and comment the coming days so we can include it into the next editor draft

KHS: do not prescribe that evaluators have to use it, but if you use it

SAZ: if step 1d would be : "define the tools / methods to be used (optional)"
... or combine 3.1.4 and 3.1.5

EV: see 3.4.5 step 4e

<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130128#step4e

SAZ: step 4e may indeed be a better place

EV: refer back to it from 5a (reporting)

DF: detailed report: would you have to register every check for every page for every SC

EV: good question, we do not adress this yet

KHS: raises the bar, but may be in the hands of the evaluation commissioner

EV: i will check if we have the same sort of issue in more places in the document

make sampling optional

EV: do we make random sampling optional?
... proposed resolution: keep non-optional for now

DF: We need to define 'random'
... if we keep it easy i do not object to keeping it non-optional

EV: few topics on the list

<Detlev> Martijn: I'm fine with *not* defining randim it if the process as straightforward as suggrsted by Richard

KHS: this can go hand in hand with the level of detail of testing

SAZ: Detlev suggested to move away from terminology s.a. random, sample
... we might take a more pragmatic approach: make it random-like
... different websites need different approaches

<Detlev> "targeted heuristics"

EV: discussion going on on the list, multiple threads
... i propose a non-statistical way, at least for the next working draft and following test-run

<Detlev> I was basically just playing back Richard's suggestions...

<Ryladog> CSUN?

EV: end of call, do the rest of discussions on the list/survey

csun

<shadi> https://www.w3.org/2002/09/wbs/48225/EvalTF_CSUN2013/results

SAZ: currently no meeting, not enough people

EF, TB, ME: no travel budget

EV: no official group meeting

<Sarah_Swierenga> I'd like to meet whoever is going to be at CSUN - I won't be arriving until Tuesday afternoon.

<Vivienne> What about WWW 2013?