See also: IRC log
<shadi> https://www.w3.org/2002/09/wbs/48225/WCAG-EM20130208/results
<shadi> https://www.w3.org/2002/09/wbs/35422/WCAGevalTaskForce/results
shadi: first internal survey, we don't need to
fill in both surveys
... objection from Samuel - does not object to idea of adding a note; can be
discussed after public review; editor note will be added to step 5.c
... text from current working draft will be used for the public comments
draft
<shadi> [Review Note: Eval TF is particularly looking for feedback on this section.]
+1
<ericvelleman> +1
<MartijnHoutepen> +1
<Liz> +1
<Vivienne> _1
<MoeKraft> +1
<Detlev> plus one
<Kathy> +1
<Vivienne> +1
katie: +1
<Mike_Elledge> +1
peter: +1
<shadi> kathy: +1
<ericvelleman> https://www.w3.org/2002/09/wbs/35422/WCAGevalTaskForce/results
eric: most responses accept as is
<shadi> [[I think there is still sometimes ambiguity in the term WCAG 2.0 Techniques, which sometimes is used to refer to any technique that meets a WCAG success criterion and sometimes to refer to the techniques published in the Techniques document. It is not necessary for this draft, but at some point please make a review and use consistent terminology to distinguish those two meanings. You may need to define those two terms in your document to distinguish them.]]
eric: conformance for particular situations - not sure what we should do with this one; will be discussed in wcag group tonight
shadi: loretta asks for a review, so shadi and eric will look for obvious ambiguities and adjust wording for next draft
eric: proposed solution ok?
+1
<shadi> shadi: propose Eric and Shadi to take a quick pass at catching obvious ambiguity, then do a more thorough review of terminology before the next publication
+1
<ericvelleman> +1
<MartijnHoutepen> +1
<Vivienne> +1
<Detlev> fine
<Mike_Elledge> +1
<Liz> +1
<MoeKraft> +1
<Kathy> +1
<ericvelleman> https://www.w3.org/2002/09/wbs/35422/WCAGevalTaskForce/results
<ericvelleman> https://www.w3.org/2002/09/wbs/48225/WCAG-EM20130208/results
vivienne: looking for samuel's actual comment
<Kathy> I am still working on an updated version
eric: evaluation flow graphic
kathy: using waterfall diagram that Peter came up with, and taking colorful style from previous one. also, trying to show the interaction without having it be overwhelming
shadi: we need to have final version by tuesday
for approval process
... deadline for external comments is today
<shadi> [[Samuel's comments resent: http://lists.w3.org/Archives/Public/public-wai-evaltf/2013Feb/0022.html]]
shadi: final version for public review is due tuesday
kathy: will try to have revised diagram by saturday
<Detlev> fine
<MartijnHoutepen> +1
<Mike_Elledge> +1
<Vivienne> that sounds fine to me
eric: proposal to use new diagram if everybody agrees. otherwise we will use the existing diagram if there are issues with the new one
+1
<Kathy> +1
<Liz> +1
<MoeKraft> +1
<ericvelleman> +1
shadi: please fill out the survey - need to have comments if there are concerns
peter: still concerned about sampling template
and not being able to test web apps in their entirety
... should there be something other than a total conformance claims for cases
where you can't test everything, e.g. web applications and software? a
conformance (or another word) claim (or another word), like 'to our
knowledge'?
... want this to be helpful for others who haven't worked on the website
shadi: agrees that this a bigger issue than this
methodology which is bound to WCAG - we are not chartered or scoped to address
this issue. inherent aspect of wcag, that the methodology applies to the entire
scope of the website
... we will not be able to address it in this draft, even though this is an
important issue
peter: want to avoid the language, 'alternative
to a conformance claim'. this is not an alternative. really likes the phrase,
'evaluation findings.'
... is not recommending that this issue hold up the public working draft
<Vivienne> Sorry guys, I can't get back on the call
eric: can you really do a conformance claim, maybe to the 'best of our knowledge' would be good to do in the next version of the working doc.
<Vivienne> zakim keeps tellingme my passcode is not valid!
detlev: all or nothing evaluation methodology is
difficult. evaluation findings is usually more the case. we don't place
ourselves outside of the practical side of this
... using conformance claim, the risk is that people will downplay the risks,
and impact the credibility of the claim
shadi: two aspects - peter - you can't evaluate everything - scoping issue; detlev - tolerance aspect - websites will typically have some oversight or issue; maybe they need to be addressed slightly differently
peter: developers probably can't make conformance
claims. they have more knowledge, but all apps have bugs. wacg is structured
around conformance claims, but nothing is ever perfect, so a conformance claim
approach is problematic.
... this is not only scoping or complexity, but a mixture of both. could we
put this as a action item - to explore 'evaluation finding' and challenges of
making claims on dynamic, complex apps
<Detlev> thumbs up for that issue!
eric: will make this part of the "issue" list
<korn> Issue: Further explore "evaluation findings" as a concept, and language that describes the challenges of making a "conformance claim" for large sites, dynamic sites, web apps
<trackbot> Created ISSUE-13 - Further explore "evaluation findings" as a concept, and language that describes the challenges of making a "conformance claim" for large sites, dynamic sites, web apps; please complete additional details at <http://www.w3.org/WAI/ER/2011/eval/track/issues/13/edit>.
<ericvelleman> Thanks Peter
mike: conformance claims seems to be more objective than they actually are; however, for all practical purposes, even if the findings are all transparent, someone is going to compare them. they will try to create a score or metric to compare the two tools. so, if we don't establish the metric for conformance claim, someone else will.
peter: we should also be aware of this this will
be used. language is important and if we suggest that this is 'the number'
or
... or 'yes, it passes' can be risky, since it usually isn't true
shadi: i don't disagree with the discussion, but does this belong in wcag, or in this methodology? needs to be discussed in the wcat working group.
detlev: the methodology should draw a clear line between the 'check' and what it really does, which is show where the problems in the site are. more realistic and more usable.
peter: we are all members of wcag or eval tf, so anything has to be approved by the wcag working group anyway
shadi: it is our responsibility to bring back findings to the working group. so far in our draft, we have that you do a 'conformance check with reasonable confidence.'
<Detlev> I have been going on about that repeatedly but it did not find consinsus of course
<shadi> sometimes consensus building takes its time, Detlev :)
<Detlev> sure :-)
eric: watch for the revised diagram and get comments back to the group quickly