WCAG 2.0 Evaluation Methodology Task Force Teleconference

23 Aug 2012

See also: IRC log


Shadi, Detlev, Martijn, Eric, Alistrair, Kathy, Peter, Moe, Mike, Tim
Sarah, Richard, Vivienne, Kerstin


Comments from WCAG WG Review (ED 30 July) http://lists.w3.org/Archives/Public/public-wai-evaltf/2012Aug/0034

<shadi> http://lists.w3.org/Archives/Public/public-wai-evaltf/2012Aug/0034

<shadi> [[This note reads very prescriptively for an informative document. If it is

<shadi> meant to be normative, the task force should recharter to produce a

<shadi> normative document. Otherwise, the "requirement" and "conformance"

<shadi> language should be softened.]]

Eric: Need to discuss the first item in the notes from WCAG 2.0 Working Group; This note reads very prescriptively for an informative document. If it is meant to be normative, the task force should recharter to produce a normative document. Otherwise, the "requirement" and "conformance" language should be softened.

<ericvelleman> http://www.w3.org/WAI/ER/2011/eval/eval-ws

<ericvelleman> “The objective of Eval TF is to develop an internationally harmonized methodology for evaluating the conformance of websites to WCAG 2.0”

Eric: Question about the sampling vs every page. That is where we differ. The goal is to create an internationally harmonized methology. Should we change the objective?
... Should this be informative?

Detlev: This means standardized methodology. There may be reasons to not be normative. Many people have noticed the optional parts and there has been discussion on the conformance claims. We should not do normative - this would over stretch our task

Peter: Agree. We should remove the word requirment or conformance to the methodology. We should put in information about the purpose of the document. Did the WCAG 2.0 working group give us specifics?

<shadi> https://www.w3.org/2002/09/wbs/35422/20120816misc/results

Eric: No, there is high level information. Language of requirement or conformance would need to be changed

<Detlev> where should the word requirement be removed?

Peter: Suggest changing the use of the words requirements and conformance. "Use as intended". We should not talk about conforming to the methodology

<Detlev> ah thanks

Allister: I see the point being made. Conformance should be made softer. Conformance is already in WCAG

<agarrison> The point was we could make a soft accessibility statement at the end of the methodology - rather than saying a website conforms

Shadi: Should we change requirement to provision? Is it just to take out the word requirement or just should we take it all out

Peter: This is the whole of the methodology - core and optional without saying these are the required part. People can take the parts that they want and ignore the other parts.

Eric: In informative document you can still have requirements. We should not have the word requirment and maybe this word should be replaced

Peter: Alternative is these are the core parts and these are the optional or something else. The core parts are designed to be used together

Detlev: Not sure what the requirements that are too strong. Some parts are required and necessary to do a review. We should look at this and these steps are needed to get on the right path

Alister: Seems like we are creating parallel conformance model. This maybe what the WCAG working group is worried about

Shadi: Not sure we break WCAG model or overriding the model. We say accessibility statement not conformance criteria

<korn> "For EvalTF to be used properly, the following steps should be taken"

<ericvelleman> a?

Shadi: The concern is that we are using language that is for normative document rather than informative document. There is a minimum set of requirement that need to be covered otherwise the methodology does not work. We need to find the middle line - not exceeding the scope

<korn> "Effective use of EvalTF involves carrying out all of the following steps"

Shadi: We cannot lose meaning by being too general

Alister: It does break the model. If you say the website conforms based on sampling - this breaks it. WCAG says every page. It does do a different job
... Methology be used to conformance claim. This needs clarification

<Zakim> shadi, you wanted to respond to Alistair

Shadi: Disagrees, this is the intent of 3.5.2 if you provide an accessibility statement. You cannot claim conformance using this metholodogy because all pages would need to be checked. The intent is not to say that it conforms to WCAG. The accessibility statement is that in the sampling it was found to be accessible

Sadhi: This is one specific use case for evaluation. T

Peter: Another suggested approach is to look at the work we are doing is to provide a sampling methology and a way to document the results. This report describes what we found.
... This goes with what is the appendix. We move away from conformance to reporting the issues found

Shadi: This is where we initially started from. Conforming to the methology - there are requirements for sampling etc. If people are to say they are following the methology then they must follow some steps

<agarrison> Is this methodology a means by which a conformance claim for a website can be confirmed as being truthful or not?

Peter: We should about how it should be properly used without saying required. Add language that it should be used as intented

Eric: This goes with core and optional parts
... Where we have required now we could say these are the core components

Peter: To be considered and EvalTF, these are the steps that may/should be taken

<shadi> [[ Requirement -> Provision | Conformance with this methodology -> Adhering with this methodology ]]

Peter: not a simple word replacements

<korn> Yes Shadi, something like that.

<korn> OI

<korn> I'd say "adhering TO this methodology", but yes.

<Tim> how about "essential step" instead of "requirement"?

Detlev: Not a big problem. Conformance is tied to single page and all states of the page. Our methology has it owns path to follow that we define indepentently - there is no issue. This should be layer on top of the WCAG conformance. Note this in the intro

<Detlev> strongly agree with Peter here

Peter: It will be more useful if we talk about the problems found on the pages. Valuable to list the issues found so that it can be determined the level of accessibility. Important that we do the evaluation on A, AA, AAA and report back more information if it does not conform

Eric: we will continue the discussion on the list

Comment #29 http://www.w3.org/WAI/ER/conformance/comments-20120730#c29

<shadi> http://lists.w3.org/Archives/Public/public-wai-evaltf/2012Aug/thread#msg77

<shadi> http://lists.w3.org/Archives/Public/public-wai-evaltf/2012Aug/0077.html

Eric: We need to discuss comment #29: 3.4.2 Step 4.b: Use WCAG 2.0 Techniques Where Possible- Requirement 4b, first bullet point
... Rewording request from Detlev

<shadi> 3.1.5 Step 1.e and 3.4.2 Step 4.b

Shadi - proposed wording is in email. Update to the role of techniques. The suggestion is to rewrite 3.1.5 step 1e and 3.4.2 4b

<Detlev> they are in Shadi#s mail *not yet* in the draft, right?

Eric: any reactions?
... we could make the change as suggested and put in the editor draft. Then can be reviewed by the team

<korn> # Rewrite for section 3.1.5 Step 1.e [[ /Techniques/ in the context of WCAG 2.0 are informative and not required for satisfying the _WCAG 2.0 conformance requirements_; WCAG 2.0 Success Criteria are written as testable statements. However, techniques provide documented ways of meeting WCAG 2.0 Success Criteria and documented failures to meet them. More information on techniques is provided in _introduction to techniques_. W3C/WAI provides a set of public

Peter: Looks redundent.

Shadi: People should look at this and commment on it - today or tonight.
... the main issue is to explain the role of the techniques for evaluators. Techniques are not required and the differences between the types and failures.
... we should do this on the list since people have not had a chance to read it
... Goal is finalize it at the end of week to send it back to WCAG working group

Peter: More effective if we have it in form of a survey. Here is the proposed language and what are your thoughts. Email is not effective.
... If the draft is going to WCAG 2.0 Working Group for next Thursday, changes from requirement - does seem possible. Too much to change.

Shadi: We are unsure about the timeline and alignment. We will see if we can save time by not having long cycles but doing them in parallel so that both groups are looking at the same time. If the group wants to review first, then we should be considerate about but it will extent the timeline
... email vs survey - both are difficult. Some people do not respond to either of them. People should be responding to both. Questions come through the mail list. We will use both approaches

Eric: Surveys are for short discussion but email is good for opinions

Mike: Clarification - looking at the editor draft disposition of comments. Have we reopened 3.1.5 1e?

Shadi: We did get comments from WCAG on the working group but also from this group. The trigger was Detlev's comments that were broader that we need to clarify the role of techniques. The two sections need to be aligned
... Only comment 28 have been reopened

<Detlev> http://lists.w3.org/Archives/Public/public-wai-evaltf/2012Aug/0038.html

Detlev: We may need more time to address the changes. Would it be appropriate to include more on processes?
... We need to be more specific about page states. Should we look at this section and in line with what the group things

Alister: If the methology is to check the conformance claim is truthful, do we need to talk about techniques. Could the conformance claim reference the techniques? Is it our place to ask for it
... If we list the techniques then it may be different from the conformance claim. What is the purpose of the methology?

<Detlev> fine if it gets delayed - just want processes discussed...

Shadi: In response to Detlev - we are behind our own schedule; so we should not delay it. Comments are good but this needs alot of consideration. Should we go down this path now or wait? We need to stay focus on getting a working draft out
... this is only our second draft

<shadi> [[Website owners, procurers, suppliers, developers, and others are frequently tasked with assessing the conformance of websites to the Web Content Accessibility Guidelines (WCAG) 2.0.]]

Shadi: Regarding Alister's question - we should not look at this only for conformance claim but it is one of the use cases. Confromance check can come at any point. It is one possibility but it is not just that
... Can carry out the evaluation and with a certain amount of certainty we can say something about the accessibility of the site. Not looking at every page for conformance claim

Eric: Document will not be perfect before sending it out. It is a working draft
... What are the most important questions to address.
... we should ask for feedback in parallel
... We will continue the discussion on the list

<agarrison> My comment was: It does not appear to be the case that techniques need to be captured in the WCAG 2.0 conformance model for a single webpage (rightly or wrongly). If we think that the methodology should be confirming if an existing conformance claim for a website is truthful - do we even need to think about referencing techniques - or would it be enough to reference a conformance claim - and suggest that whatever techniques they used should be included in th

EV: can we send the responses to both WCAG WG and Eval TF for review?

PK: not sure what this will work
... by when do you expect to have something?

EV: maybe Monday

PK: what's the timeline for that?
... is there enough time to make that happen?

<Detlev> no. fine

SAZ: Monday would provide disposition of comments and updated Editor Draft
... send that to WCAG WG on Tuesday

<Mike_Elledge> +1

<agarrison> Some of my comments where based on contents of http://lists.w3.org/Archives/Public/public-wai-evaltf/2012Aug/0071.html

SAZ: have Eval TF discussion on Thursday before WCAG WG discussion

SAZ: to get input on how the comments were addressed

Summary of Action Items

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.136 (CVS log)
$Date: 2012/08/23 15:21:31 $