WCAG 2.0 Evaluation Methodology Task Force Teleconference

Follow up discussion about latest Editor Draft and DoC

<ericvelleman> http://www.w3.org/WAI/ER/conformance/ED-methodology-20140121

EV: updated Editor Draft of 21 January
... treid to integrate most comments

<ericvelleman> http://www.w3.org/WAI/ER/conformance/comments-20131129

EV: there are two disposition of comments
... one from 26 February Working Draft
... the second from 29 November Editor Draft
... missing just a few comments to integrate
... particularly on adding a section on "responsive design"

SAZ: action on Kathy, need to check with her if she can get it in time

EV: otherwise need to add an editor note for review
... comment #72 to add arrows from each step to step 5

<Detlev> I don''t understand the issue - can you explain, Eric?

EV: graphic was from Kathy?

ME: need a section for responsive design?

SAZ: actually a paragraph in the list under "Particular Evaluation Contexts"

ME: could then shorten the note in section "website in multiple versions" if we have a specific section on responsive design

EV: need text from Kathy first
... back to comment #72, everyone OK with an arrow from each step to step 5?

DF: diagram already quite complex
... will that really make it better?

EV: agree but more precise

DF: quite self-explanatory
... but the important message is the feedback loop
... not the fact that you need reporting all the time

<Vivienne> I'm not too worried about it

<Detlev> leave as is

<ericvelleman> -1

<agarrison> -1

<Mike_Elledge> -1 (keep the same)

<MaryJo> -1

<Liz> Leave as is

[poll on making the change of adding arrows above]

RESOLUTION: keep it as-is in the upcoming draft for more discussion

<MaryJo> keep as is, lists of one are not optimal.

<ericvelleman> http://www.w3.org/WAI/ER/conformance/ED-methodology-20140121#step5d

<Detlev> Should be "Aggregated Score" not "Aggregation Score"

EV: had agreed to drop "Per Instance" score approach
... also dropped "Web Web Page" scoring
... and renamed to "Aggregated Score" to make clearer

DF: do we remove "Per Web Page" score as well?
... if one SC is missed on one page gets reflected highly

<Detlev> per page would work fine for me

SAZ: agreed on "Per Web Page" than "Per Website"
... also need to discuss opening up to other scoring approaches

MJ: think "Per Website" makes more sense

[SAZ explains "Per Web Page" vs "Per Website"]

DF: approach of calculating over all pages is what we do
... but some may argue that this could raise discussion
... "Per Website" loses quantitative aspects

EV: in The Netherlands it is done the other way around

DF: "Per Web Page" can mask issues, like CAPTCHA that is critical
... "Per Website" is more sensitive and forces people to look at the issues more closely
... people can also add their own scores approach on top

<Detlev> hello, can of worms,,

GE: maybe could come up with an approach to measure the aggregate score
... like multiply certain types of issues with a factor
... often get asked to provide a score for organizations

VC: difference between WCAG 1.0 and WCAG 2.0 in the level assignment
... that has impact on scoring
... gets into controversy

GE: more for research on analyzing different websites that an individual website

SAZ: sophisticated scoring gets very complex and controversial
... but intent is to communicate improvement over time
... "Per Website" better communicates progress but can also cascade serious issues

MK: should add disclaimer that there is no widely recognized scoring mechanism

EV: have very tentative text

DF: often have incidental issues that could have significant effect
... could be a systemic pattern but still has little impact on website after all

<Vivienne> I'd vote to add the other back in and have 2 choices

<Detlev> Vote?

<Detlev> agree with Vivienne to have both in

<Liz> both in

<gavinevans> both in

<MoeKraft> keep both

<Detlev> Rationale: having both in will gneerate more helpful comments

SAZ: add "Per Website" back in and move discussion to next publication

RESOLUTION: add "Per Website" back in and move discussion to next publication

Face-to-Face Meeting at CSUN

[in the process of getting contract for f2f room confirmed]