WCAG 2.0 Evaluation Methodology Task Force Teleconference

29 Sep 2011


See also: IRC log


Vivienne, Samuel, Shadi, Kerstin, Kathy, Amy, Richard, Eric, Denis, Sarah, Mike, Katie, Elle
Tim, Alistair, Liz, Detlev



<Ryladog> http://www.w3.org/WAI/ER/conformance/ED-methodology-20110928

ev: missing introduction, is that OK Will work on it with Detlev

Just scope as it includes goals, Clearer as just scope

<vivienne> The scope looks really good now

ev: thanks

<shadi> "The methodology will be cross-tested by the Task Force and will include a reference to the test results"

shadi: unclear what cross tested means can we edit better, also look at co-ordination with other groups

shadi: If we want to monitor ATAG we need a named person - but do we really need to do that

<vivienne> I'm not happy with the term 'monitor', seems like we're supervising rather than watching

<vivienne> maybe follow rather than monitor

ev: we can take it out. The idea was that we also look at the work being done in other groups, not necessary to co-ordinate just keep a watching brief

<shadi> "The Methodology will be written to be agnostic to the context of conformance evaluation so that in can be used for self-assessment by the manufacturer or supplier, for acceptance-testing by the user or purchaser, or for third-party evaluation by an independent body. This makes the Methodology compatible with quality assurance processes such as that defined by ISO/IEC Guide 7."

ev: will look at the co-ordination section

<kerstin> I'm missing freelancer testers

ev: target audience - I have remodelled it a bit. added advocates, These are now our tarfget audience

<Zakim> shadi, you wanted to ask about "during the development process" and if "Developers of Evaluation and Repair Tools" are primary audience

shadi: Does our method primarily cover the development phase, or is it primarily a final audit?

<Ryladog> I agree with Shadi

<dboudreau> @kerstin: wouldn't freelancers be a sub-group of the web content producers group?

EV: Primary audience is those wanting to evaluate against WCAG2

<kerstin> @dboudreau don't think so, one can be accessibility consultant and not a web developer

<vivienne> I agree, leave out post-mortem

<shadi> "Web accessibility consultancies and evaluation services"?

kerstin: missing freelance consultants - ie single people without an organisation

<Sarah> maybe use 'web accessibility evaluators'

<vivienne> I think that the term encompasses both individuals and corporations under consultancies

<vivienne> web accessibility specialists and evaluation services?

<shadi> [[While the Methodology will provide guidance on evaluation throughout the development process, it is specifically designed to inform on evaluation of existing websites; complementary WAI resources will provide further advice on evaluation during other stages of the development process.]]

<vivienne> +1


<Ryladog> +1

Mike: are we being too restrictive ?

ev: We can use the methodology at any time during teh process, but users will find advice on earlier stages

<Zakim> shadi, you wanted to ask for specific suggestions to the current wording (pasted above)

Amy: We would use evaluatin during development - but not want to make a performance claim until finished

<AmyChen> conformance claim (not performance claim)

<vivienne> I'm happy with the scope now.

ev: Look at last paragraph of Scope section - is this adequate. are there any specific suggestions for changing it ?

<shadi> [[While the Methodology will provide guidance on evaluation throughout the development process, it is specifically designed to inform on evaluation of existing websites; complementary WAI resources will provide further advice on evaluation during other stages of the development process.]]

<dboudreau> I'm also happy with this suggestion

<dboudreau> +1


<Kathy> +1

<Ryladog> +1

<Mike> +1

<AmyChen> +1

<ssirois> +1

<EricVelleman> +1

shadi: Eric and Shadi will look at target audience to include existing audience but ensure secondary audience includes users at development stages

EV: most discussion on requirements 3 and 4

<Sarah> +1

Katie: Greg usually uses a term - if a number do testing and get the same result 80/20 rule would make it replicable. ie 80% get the same result

ev: perhaps we should go back to reliable for Requ 4. This makes it a bit more flexible and reliable has a bit of theis 80/20 bit. We need to bear in mind quality martix

<Mike> consistent?

<vivienne> I prefer reliable to replicable.

R4 includes word "should" that allows for some leeway

<Sarah> Referring to an 80% percentage could be an issue - how would it be measured? 80/20 is a rule of thumb, not an exact number

<Ryladog> +1

EV: R14 can we look at this first

<kerstin> Voraussetzung

<shadi> Pre-requisit

Kirsty: what is the german word for replicable and reliable

<ssirois> i agree with sarah on the 80/20 rule of thumb. i have difficulty understanding how a tolerence could be a metric at all. i see that as a "human appreciation" thing!?

<EricVelleman> shall we change R04 to reliable then?

<Ryladog> +1

<vivienne> +1

<Kathy> +1

<dboudreau> +1

ev: is everyone happy to change to reliable in R4


<kerstin> @Richard Replizierbarkeit und Reliaibit├Ąt, or in verbs replizierbar / reliabel

<Mike> +1 for reliable

<Sarah> reliable +1 - based on Mike's comments about the definitions

<ssirois> +1 for reliable, absolutely

viviene: I agree - replicable is srict, do it again exactly, so I agree with using reliable

<Ryladog> Yes I think so for human judgement

viviene: is there W3C document on the 80/30 rule ?

Shadi: Yes in tecniques it was included that they should be reliable - I will look it up again for reliablity, replicability and ambiguity ?

<shadi> http://www.w3.org/TR/WCAG20-TECHS/intro.html

EV: Any one else welcome to help
... R3 unambigous - some want it other want to lose it - Is it our interpretation of WCAG or the methodology techniques

Kathy: the end result should be unambigous ?

IMike: It can be confusing - are we talking about the methodolgy description being unambigous ?

EV: I will try to clarify it.

<EricVelleman> Proposal for rephrasing: The Methodology will be written using terminology and a style that is as easy to translate as possible; where necessary terms will be defined in a glossary.

EV: translatable - should we use new phrase?
... methodology should be easy to translate into german etc. If needbe a glossary will help

??: Yes it must be in clearest language possible - in English or whatever !

EV: Must point out R18 - Quality Assurance. If we want to be a formal document it must comply with these guidelines.

EV: Resources - some interesting stuff coming in - but what do we do with them. Could go at bottom of page - but it might get too big

ev: discuss on teh list and keep sending in resources please.

Summary of Action Items

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.136 (CVS log)
$Date: 2011/09/30 08:16:23 $