WCAG 2.0 Evaluation Methodology Task Force Teleconference

09 Jan 2014

See also: IRC log


Shadi, Kathy, Vivienne, Liz, Alistair, Martijn, Sarah, Detlev, Tim, Mike
Moe, MaryJo, Eric


open comments in DoC

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20131129

<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20131129

SAZ: we published editor draft in november, now trying to wrap up comments
... hopefully working towards our last working draft
... other groups are reviewing our draft s.a. WCAG, ERT
... some comments are still open

<shadi> http://lists.w3.org/Archives/Public/public-wai-evaltf/2014Jan/0004.html

comment 18

SAZ: comment 18,27,33: we suggested "essential functionality" instead of "core"

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20131129#comment18

SAZ: does anybody have another suggestion for this wording?

<Sarah_Swierenga> 'essential' works for me

DF: whatever we pick, we'll always have comments

SAZ: we'll keep essential for now

<Vivienne> I'm happy with essential for now

comment 53

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20131129#comment53

SAZ: comment 53 suggested we give extra guidance on responsive webdesign

KW: if content is hidden in the mobile version of the website, this could be considered as another page

SAZ: in the section "scope of applicability" the extent of the website is described, testing of mobile websites is part of another section

KW: we need guidance on what to do with mobile versions of the same site

<shadi> Suggested Text [[Note: Websites using responsive design techniques (i.e. adapting the presentation according to user hardware, software, and preferences) as opposed to redirecting the user to a different location are not considered to be independent website versions unless the site at the different breakpoints utilizes different code. In that case, the website at the different breakpoints could be considered as individual websites each for evaluation.]]

DF: Agree, different breakpoints can change the website in very different ways, we need to acknowledge this

AG: this fits under "use cases"

SAZ: in several steps, f.e. step 4, but i don't know if it constitutes different websites

AG: seeing this as different website could be problematic for users

DF: I wouldn't treat this as a different website, but we can include it as a certain "state" of a page

KW: agree with DF, we can incorporate it in step 3 & 4

SAZ: we can add it to scope of applicability, and mention it at the page states, and in step 4

KW: agree

under particulor types of website we add another item: website using responsive design

Decision: under "particular types of websites" we add another item: website using responsive design

comment 43

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20131129#comment43

<Detlev> Shadi, has the third scoring option already been deleted?

<Detlev> fine

SAZ: the "per instance score" is dropped as agreed

DF: people will be able to make scores for themselves anyway, so it doesnt matter that much. Agree with Gregg V that it's debateable. Not happy to drop it, but can see the point

SAZ: will be discussed in WCAG call tuesday

VC: People will come up with there own scoring, i think it's necessary to give some guidance on scoring, but we needn't prescribe a way how it should be done

LF: We talked about having an addendum to this document, with a paper on different kinds of scoring methodologies. "although we don't dictate how to score, see .... for further information"

LF: NIST has come up with a weighted scoring system, this could be referenced

SAZ: we now offer the most simplest form of scoring; do we need to remove this?

<shadi> http://www.w3.org/community/wai-engage/

SAZ: WAI-engage is the place for this kind of suggestions

ME: problematic to put any scoring system in the body of the document, the process of scoring websites is too complex
... an alternative: "although WCAG-em doesn't recommend any scoring method, but see reference X for guidance..."


ME: we can establish criteria to use when choosing scoring methods

<shadi> http://www.w3.org/WAI/RD/2011/metrics/

<shadi> http://www.w3.org/TR/accessibility-metrics-report/

AG: we can just remove everything, because it is too complex

SAZ: we can reference a wiki, where all scoring methods can be collected

AG: in different studies, different scoring methods are use in the context of that study, so not all methods can be extrapolated

DF: we now have the minimum of scoring methods, but i'd like to keep it in, as optional

SAZ: score methodology draws focus away from conformance

comment 50

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20131129#comment50

VF: First letter capilisation

<Detlev> fine to me

<Sarah_Swierenga> +1

comment 49

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20131129#comment49

<shadi> [[Following this methodology will help evaluators to apply good practice, avoid commonly made mistakes, and achieve more comparable results]]

VF: the "to" has to come out

<Sarah_Swierenga> out with the 'to' :-)

<agarrison> don't need the final comma either...

SAZ: please review the comments, especially your own
... if all goes well, we can get agreement with WCAG WG

Summary of Action Items

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.138 (CVS log)
$Date: 2014-01-14 16:03:19 $