W3C

WCAG 2.0 Evaluation Methodology Task Force Teleconference

21 Feb 2013

See also: IRC log

Attendees

Present
Katie, Martijn, Shadi, Alistair, Liz, Peter, Vivienne, Moe, Tim
Regrets
Kathy, Sarah, Moe, Detlev
Chair
Shadi
Scribe
Martijn

Contents


<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130219

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20130208

SAZ: Latest version online in which we processed the comments received from wcag wgf
... comment No1 added a review note on terminology

No2 Styling

3 is a grammar fix

4 added a review note on scoring

SAZ: Right now you score your score against the total SC
... suggestion to exclude non-applicable SCs in scoring
... We'll get input from testdrive

VC: the way it was originally intended: not applicable is pass. We added 'not-applicable' and need to word this carefully

SAZ: we discussed this on reporting, but not yet on scoring

<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130219#step5c

<korn> +1!

AG: Why not look at failures instead of passes?

<Vivienne> I also score via the failures

<korn> If you have 36 A/AA criteria, and n failures, then your score is 36-n (per page, etc.)

SAZ: do we need to adress this before publishing or put it in a review note

PK: publish now, but give alternative scoring methods we consider in review note
... proposal: score = 36 - (number of failures)

<shadi> [[[Review Note: Feedback on this section is particularly welcome. For example, how do these scoring approaches work in practices? Are there other simple yet effective scoring approaches? Should the scoring be based on applicable Success Criteria only?]]

SAZ: let's put this in a (new) review note

<korn> Feedback on this section is particularly welcome. For example, how do these scoring approaches work in practice? An alternate scoring approach being considered is...

<shadi> [[[Review Note: Feedback on this section is particularly welcome. For example, how do these scoring approaches work in practice? Are there other simple yet effective scoring approaches; Eval TF is considering using score based on failures rather than successes? Should the scoring be based on applicable Success Criteria only?]]

AG: 4 failures out of 25 sc can be high or low , it depends how many sc are applicable

<korn> Feedback on this section is particularly welcome.  Another approach being considered is to record success criteria failures.  A third is to also track those that are not applicable.  Are there other simple yet effective scoring approaches? Is there an approach preferred by the reviewer?  For example, how do these scoring approaches work in practices? Should the scoring be based on applicable Success Criteria only?]

<Vivienne> I'm fine with it

<agarrison> Fine

<MoeKraft> +1

+1

<Ryladog> +1

<shadi> RESOLUTION: take up Peter's wording suggestion for the review note (with minor tweaks as necessary)

<korn> FInal suggestion: [[[Feedback on this section is particularly welcome.  Another scoring approach being considered is to instead record failures of success criteria.  Also being considered is tracking those that are not applicable.  Are there other simple yet effective scoring approaches? Is there an approach preferred by the reviewer?  For example, how do these scoring approaches work in practices? Should the scoring be based on applicable Succe[CUT]

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20130208.html

SAZ: this will be updated before publishing
... Comment no5 not applicable
... No6 wording change

<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130208#rerun

SAZ: 7 rewrite of section re-runnig evaluation, include old and new pages
... 8,9,rewording of adjectives

VC: can we add smt like: while it is only one of the approaches, this is the only one developed by W3c

SAZ: Not necessary, has a high standing on its own
... comment 10: earlier comment 25 on separable area's not adressed, we added a review note
... comment 11, see comments 8 and 9
... comment 12 see 11
... comment 13 see 11 and 12

<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130208#applicability

SAZ: comment 13: the note is rewritten

<shadi> https://www.w3.org/2002/09/wbs/35422/20130221eval/results

<shadi> [[Note: WCAG 2.0 defines "Statement of Partial Conformance" for individual web pages that are known not to conform with WCAG 2.0 due to third-party content and/or languages lacking accessibility support. Such web pages may not be excluded from the scope of evaluation according to this methodology. In some cases this means that the website as a whole does not conform with WCAG 2.0 due to partially conforming web pages. Section Step 5: Report the Evaluation Findings

<shadi> provides more guidance on reporting evaluation results and making accessibility statements for entire websites.]]

<shadi> http://services.w3.org/htmldiff?doc1=http%3A%2F%2Fwww.w3.org%2FWAI%2FER%2Fconformance%2FED-methodology-20130208&doc2=http%3A%2F%2Fwww.w3.org%2FWAI%2FER%2Fconformance%2FED-methodology-20130219

<shadi> http://services.w3.org/htmldiff?doc1=http%3A%2F%2Fwww.w3.org%2FWAI%2FER%2Fconformance%2FED-methodology-20130208&doc2=http%3A%2F%2Fwww.w3.org%2FWAI%2FER%2Fconformance%2FED-methodology-20130219#applicability

<shadi> [[CHANGE

<shadi> Such web pages may not be excluded from the scope of evaluation according to this methodology.

<shadi> TO

<shadi> Such web pages may not be excluded from the scope of evaluation if this methodology is used. (or- ....if using this methodology)]]

Comment 14: on techniques from the wcag wg to techniques in general, removed some references to wcag

AG: less clear now

SAZ: ongoing discussion on techniques and the misunderstanding of the role of techniques
... wcag wg will adress this in the future
... for now they ask us to not further the misunderstanding

AG: take out the links to techniques

SAZ: we will discuss this further, maybe add in brackets to not limit ourselves to wcag techs
... i will report back on comment 14 and 16
... comment 15 needs more thourough revision, we will have real world testing after publication

PK: add a review note on appendix C

<korn> We would appreciate feedback on the report items. Should any more be added, or any removed or changed?

+1

<Vivienne> +1

<Ryladog> +1

<Tim> +1

<korn> Note: WCAG has not yet defined what conformance means for an entire website, just for individual web pages....

<korn> ...Is the language here confusing on that point?

<shadi> RESOLUTION: add review note to Appendix C as proposed by Peter

<korn> My final comment in this section was: [[[Also further down in the same bulleted list is the item "Reason for not conforming to WCAG 2.0: "third-party content" or "lack of accessibility support". This text should be a conditional, e.g. "If present, and if known, the reason for not conforming...."]]]

<korn> Any reason we can't just make that editorial change here? That should be a simple one...

<shadi> [[Conformance is defined only for Web pages. However, a conformance claim may be made to cover one page, a series of pages, or multiple related Web pages.]]

<Tim> I think WCAG talks about conformance for multiple pages as part of a process

PK: comment 17 change "Reason for not conforming to WCAG 2.0: "third-party content" or "lack of accessibility support". to "If present, and if known, the reason for

SAZ: we will discuss appendix C before a later version

NO meeting next week

Summary of Action Items

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.137 (CVS log)
$Date: 2013-02-22 18:32:29 $