W3C

WCAG 2.0 Evaluation Methodology Task Force Teleconference

29 Nov 2012

See also: IRC log

Attendees

Present
Eric, Katie, Kathy, Martijn, Peter, Detlev, Mike, Sarah
Regrets
Alistair, Liz, Shadi, Moe
Chair
Eric
Scribe
Martijn

Contents


data from sampling survey

EV: the data has names in it, is someone able to anonimize?

DF: has anybody objected?

EV: to make it public, i would need permission of everybody

KW: i will do it

EV: send the data to the list when its done

Disposition of comments

EV: DoC is online, together with a survey

http://www.w3.org/WAI/ER/conformance/comments-20120920

https: //www.w3.org/2002/09/wbs/48225/evaltfq5/

EV: sent an email for question 8,9, 11
... starting point for discussion, more discussion on the list
... will try to take on comments in 3 ways

EV: Any questions about the survey?
... Next task would be to come up with a next Editor's draft.
... Disposition of comments (<http://www.w3.org/WAI/ER/conformance/comments-20120920>) has been grouped

KHS: Looks good.

EV: Problem with taking out the numbering now... we wouldn't have a link between dispositoin of comments and the draft. So proposal is to keep numbering in until we've finished with most of comments. And then make the change... at the end. OK?

<Detlev> agree - kep numbering in for now

<MartijnHoutepen> agree

EV: Agenda #4 - Disposition of Comments (already somewhat discussed)
... Plan is to do surveys on various questions, discuss in surveys, and then dispose of them.

<Detlev> Christ - 192 comments..

EV: Any other questions/comments?

<Detlev> bundle the little bleeders...

EV: Only got as far as section 2; but in sections 3-5 found a lot that can be grouped. So, 192 total comments, but perhaps only 50-60 unique comments/issues.
... Agenda point #5 - DoC ID 98.
... Seeking a volunteer - to work on the various terms we use (audit, evaluation, check, test, etc.)
... Which terms should we use, and where?

<MartijnHoutepen> PK: concerned about the term audit

<MartijnHoutepen> PK: seek to avoid audit

EV: Agrees, "audit" likely shouldn't appear in our document.
... So... any volunteer(s) to do this work? Perhaps on the list?

KHS: Volunteers to do this.

<MartijnHoutepen> shall we make an action ?

<MartijnHoutepen> ACTION: Ryladog to research the use of terms like audit, evaluation [recorded in http://www.w3.org/2012/11/29-eval-minutes.html#action01]

<trackbot> Sorry, couldn't find Ryladog. You can review and register nicknames at <http://www.w3.org/WAI/ER/2011/eval/track/users>.

<scribe> ACTION: Katie_Haritos-Shea to research the use of terms like audit, evaluation, etc. (regarding DoC ID 98) [recorded in http://www.w3.org/2012/11/29-eval-minutes.html#action02]

<trackbot> Sorry, couldn't find Katie_Haritos-Shea. You can review and register nicknames at <http://www.w3.org/WAI/ER/2011/eval/track/users>.

EV: Now on other issues. Through agenda faster than expected.
... To use our time, let's discuss exclusion criteria.

<ericvelleman> https://www.w3.org/2002/09/wbs/48225/evaltfq5/

<Ryladog> ACTION:Ryladog to review Disposition of comments #98 Proposed resolution: Harmonize the use of Audit, Evaluation, Check, Test etc in the document. Open issue: "propose when to use audit, evaluation, check, test etc in the document" [recorded in http://www.w3.org/2012/11/29-eval-minutes.html#action03]

EV: This is question #8

<ericvelleman> https://www.w3.org/2002/09/wbs/48225/evaltfq5/

<MartijnHoutepen> http://www.w3.org/WAI/ER/conformance/comments-20120920#section2exclusion

PK: There are four (A/AA) SCs that talk to a "set of web pages".
... Exclusion criteria might be tied to the definition of the "set".

DF: Agrees with this idea.
... Few sites will be 100% perfect everywhere; using exlcusion is also a nice way to note some subset that is perfect.
... Also for sites that are in development; excluding parts that are known to be a problem is also useful.

EV: Other comments?

<Kathy> I agree with Detlev

EV: IF we all agree with this, this is a fundamental thing. We need to be thoughtful about this.

DF: There are multiple use cases for exclusion.

EV: All in agreement?

<Detlev> we don't have that many people on the call today...

MH: If we want to allow these exclusions, we need to be very careful - to not make it some sort of excuse option.
... So it is probably easier to not allow them, but if we really want them, then we need to describe very thoroughly how to do so.

KW: This mechanism can also give people clarity on what parts are / aren't accessible.

DF: Not talking about a final site (e.g. a seal of approval on one section, another not), but rather about the use case during development/design phase.
... The real question about exclusion's use is for a final conformance claim.
... But during design/development, excluding some parts is very useful.
... Note too that we don't have as many folks on the call as usual; we should get more feedback before coming to a final decision on this.

SS: Concerned about opening this window for people using the methodology to select aspects of their sites/apps for going for full conformance, while identifying other areas as "we hope to get there, may not ever get there"
... Thought the methodology as drafted now did account for some of this - the whole sub-parts discussion. But not sure we want to go down this path.

<Detlev> yes, Eric

EV: This is primarily about opening up the methodology to other use cases.

SS: Want to make sure this doesn't get hijacked.

<Detlev> Peter: We need a way for sites to take stock of what works and what doesn't independent of conformsnce claims

PK: We have a long-standing issue for large sites & web apps that aren't perfect - but need to report on what is (and isn't) accessible.
... For those situations you can't make a 100% perfect conformance claim; and exclusion may be a useful tool as part of noting this lack of perfection.

ME: See Question #1 in questionnaire, DoC ID 4 "Scope of WCAG-EM"
... This is correlated to the situation that DF is describing.
... This issue is one that can be resolved by there being clarity in claims that are made for a website / compliance. So long as the entity that is responsible for the site states the particular limitations to the accessibility claims... we are dependent upon honesty.
... Some voluntary declarations from some entities provide little information; other cases you get a tremendous amount of detail.
... So the key question is providing sufficient information.

EV: Claim and Scope are two distinct issues.
... See at least another use case coming out of these discussions.
... Getting more clarity on what is claimed is perhaps more than just a question/topic of scope, but also for question #5.

KHS: Also really have to clarify the difference between a scoping claim relative to this methodology.

EV: Invites everyone to look at survey question #8 about exclusion. Also in the disposition of comments.
... now to final agenda item. Any other issues?
... Hopes to launch a new survey by Monday morning. Will then discuss this survey in the next call (and the Monday survey in the call after that)

Summary of Action Items

[NEW] ACTION: Katie_Haritos-Shea to research the use of terms like audit, evaluation, etc. (regarding DoC ID 98) [recorded in http://www.w3.org/2012/11/29-eval-minutes.html#action02]
[NEW] ACTION: Ryladog to research the use of terms like audit, evaluation [recorded in http://www.w3.org/2012/11/29-eval-minutes.html#action01]
[NEW] ACTION: Ryladog to review Disposition of comments #98 Proposed resolution: Harmonize the use of Audit, Evaluation, Check, Test etc in the document. Open issue: "propose when to use audit, evaluation, check, test etc in the document" [recorded in http://www.w3.org/2012/11/29-eval-minutes.html#action03]
 
[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.133 (CVS log)
$Date: 2012/12/05 13:30:12 $