WCAG 2.0 Evaluation Methodology Task Force Teleconference

12 Dec 2013

See also: IRC log


Kathy, Eric, Shadi, Detlev, Martijn, Moe, Sarah, Richard, Mike, Alistair, Liz, Tim


<shadi> http://www.w3.org/WAI/ER/conformance/comments-20131129

Shadi: Many comments, mote to come. Most disputed perhaps scoring, Use of review teams.
... There was a call to review reference to review teams - suggestion to respond by chsanging language from "highly recommended' to 'beneficial'

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20131129#comment10

Kathy: Need not be highly recommended but still usefull, even by not so experienced evaluators - remains a good recommendation.

<richard> Remove +1

<Sarah_Swierenga> +1

<MoeKraft> +1

<ericvelleman> +1

Shadi: Any objections to changing from highly recommended change to something else?

<MartijnHoutepen> +1


Shadi: resolution to change language, but section stays in

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20131129#comment11

Shadi: Comment No. 11 (Michael Cooper) concerning the removal of a section

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20131129#comment18

Shadi: Comment 18 )Gregg) :Concern about the use of the term "core functionality"
... Suggestion Gregg: better use DEPENDEND COMPONENTS

Richard: The thing is about function, not component, functionality should stay - could ber either core or common

Shadi asks people to look at comment 18 closely to solve this issue

Eric: This started with primatry functionality

Detlev Dependent components is not easy to grasp

Eric: can't we keep 'core functionality'?

Shadi: lets wait for responses first, then decide
... explains difference between core and dependent..

<MartijnHoutepen> DF: i believe GVs concern is about opening the floodgates to subjective interpretations of core

<MartijnHoutepen> DF: afraid of using complicated terms

Detlev explains what Gregg arguments against core functionality were

Kathy: Agrees that core functionality may not be ideal, bu tthe main thing is to link to a terms definition "core functionality' (or whatwever other term)

Shadi: Agreement that core functionality may be subject to misunderstanding we could truy dependend components

<MartijnHoutepen> DF: trying to find an example of another term suggested bij GV; I think it was 'essential'. 'Dependent component' seems too complicated

<shadi> [[From Gregg's comment: I would advise talking about "High Frequency pages" -- and "Pages needed to complete processes".]]

Detlev core is perhaps too narrow, better term needed

Alistair: does not like dependent components, makes no sense

Shadi: Resolution: We accept Greggs critique but think his suggestino is not ideal

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20131129#comment37

Shadi Comment 37: involving people with disabilities - one comment asking that non-optonal, the othe rtoo remove that sugestino because it adds not so much to checking, mire to usability

Shadi: Proposed resolution: Change to "Involving people with disabilities and people with aging-related impairments helps identify additional accessibility barriers that are not easily discovered by expert evaluation alone".

Alistair: happy with current text

Eric: also happy with Proposed resolution, AListair?

Alistair: Agrees with second comment that says for purposes of evaluation here, it doesn't add much to involve users

Shadi: Still true (ans should be stated) that it is still beneficial to involve users

Kathy: In user testing you always cstch issues that you wouldn't have caught otherwise, much if it not strictly a11y-related (also to AT, UA, usability etc)
... we should not open this too much so we do niot confuse users

<ericvelleman> change from 'discovered by the evaluators alone' to 'discovered by expert evaluation alone'?

Alistair: thinks change weakens the text - do we have to make the change?

<Liz> no change

Detlev: Can't think aout that so wquickly...

Kathy; Expert evaluation alone will not catch all issues

Shadi: Explains rationale of proposed change to address concern


Alistair: not happy with change but go ahead

Eric: We need to keep the doc in line with general WAI guidance
... we say it's optional anyway

Shadi: WCAG docs (Intro, Understanding..., mentions the importance of involving users also

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20131129#comment28

Shadi: Comment 28 - Sample size suggestion by David (Candadian Gov)
... Difficulty starts with determining the size (including DOM states)
... Proposed Resolution: Add editor note and ask for more input on this issue, also ask David for details on source

Richard: Our job is to evaluate, not to establish statistically correct sample
... If possible, do the whole thing
... else, follow the steps (common pages, add random sample)

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20131129#comment31

Shadi: Comment by Richard that we cannot prescribe random sampling

Richard: It is up to the evaluator to specify the random sample size because the need for that will be specific to the web site under review


<shadi> http://www.w3.org/WAI/ER/conformance/comments-20131129#comment30

Shadi: There is no implication to sample.

Richard: Weoften do the whole site - should be visible as option

Shadi: If all has been tested, you can make a conformance claim
... back to sample size, reiterates proposed resolution
... Objections?


<Liz> no

<ericvelleman> Agree

Shadi: Sample size is adaptive and site size specific - What is the objection of suggwesting 10% as random-sampled, Richard?

Richard: Seems it is too detailed, to resterictive
... Would be enough to say use Random sample, let evaluator decide how to do it but record what he/she has done

Shadi: Aim was to introduce a minimunm and a threshold so it would scale

Eric: OLd versions were very briedf and were beefed up for tthis version - we need to test the methodology so we will find out if it works, that will generate relevant feedback

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20131129#comment36

Shadi: about N.A. Gregg suggesting use of "not present" instead of N.A.
... has been discuss a lot in the WG

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20131129#comment38

Shadi: about the term correlation
... Please look at the disposition of comments and use the mailing list or survey for feedback

SAZ: we'll discuss results with WCAG wg and ERT wg, sadly no publication this year

EV: thanks Shadi for overview, clustering of comment. Let's start discussions on the list

Summary of Action Items

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.138 (CVS log)
$Date: 2013/12/22 16:58:41 $