See also: IRC log
<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20131014
Shadi: Updated editor draft on IRC. Still working on incorporating all changes with Eric. Shadi hs been massaging, editing. Some areas require more substantial changes.
<shadi> - Abstract (significant editorial rewriting)
<shadi> - Introduction (significant editorial rewriting)
<shadi> - Purposes for this Methodology (minor edits)
<shadi> - Relation to WCAG 2.0 Conformance Claims (significant changes)
<shadi> - Background Reading (hardly any change)
<shadi> - Terms and Definitions (added definition for "Web Page States")
<shadi> - Using This Methodology (re-organized with minor edits)
<shadi> - Scope of Applicability (re-organized with significant edits)
<shadi> - Principle of Website Enclosure (new section based on old text)
<shadi> - Particular Types of Websites (primarily "web applications")
<shadi> - Particular Evaluation Situations (still open edits to do)
Shadi: Sent back to Eric to review. Not much time to review. This is not the final, don't worry. There will be time to review.
Sh: Today's call, go thru changes, to see if any
immediate issues that I and Eric can address. Also several questions. Since
Peter's last meeting (hopefully not) Peter if you have any immediate questions
or comments will prioritize them on thsi call call.
... Peter, any comments?
Peter: Seeing for first time. I would like to look at introduction, relation to 2.o conformationce claims, then reprort findings step 5B, and probably also steps 1B-D. in that order.
S: Didn't get to latter section (steps), so may
be open comments that need to be added into doc.
... let's look section by section. Introduction heavliy rewritten. Next
subsections (Purposes ahs not changed much), but Wcag has. Let's give everyone
5 minutes to look them over. Then discuss.
P: Minor nit--
<korn> ... how much KNOWLEDGE the evaluator has to how the website was developed
A: Abstract throws out what saying in intro--by talking about conformance in intro have we not said one thing then another. Conformance accessibility Statements in introduction instead.
M: Yes
<korn> "Evaluationg the extent to which a website supports the WCAG 2.0 guidelines is...
<Detlev> Yes - I has unfortunately no time before to read the draft
P: Rather than using conformance in first sentence, might say (see above).
P: Then when get inot section "this is a document that is sampling" not using word conformance.
S: Just want to be sure that eeryone is on same page.
<Detlev> "...assessing to what extent websites adhere to WCAG"
E: Period of silence. Note things in IRC.
<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20131014
<korn> Maybe re-use text from the first sentence of the Abstract in Intro - "guidance on the process oe evaluating..."
M: supersede THEM
<korn> E.g. "The process of evaluating ... involves several steps.
<korn> end of first para of Relation to Claims: too long. And better stronger, e.g.: "known to completely satisfy each and every".
<korn> Last para of Relation: "THe majority of the use cases of WCAG-EM involve sampling a subset of "
S: Resuming discussion...
... Alistar raised relation of conformance to sample claim. Suggestion to
change from conformance to conformance claim.
<Detlev> I think "Evaluating the conformance of a website to the Web Content Accessibility Guidelines (WCAG) 2.0" should really be "Evaluating the accessibility of a website to the Web Content Accessibility Guidelines (WCAG) 2.0"
K (was P): Like guidance to the process. Borrow that type of language. Suggest "isntead of evaluating conformance of web site" say "process of evaluating extent to which website conforms is a process that involves several steps."
K: Process not conformance.
<shadi> [[Evaluating the extent to which a website ahere to the Web Content Accessibility Guidelines (WCAG) 2.0 is a process involving several steps involved. The activities carried out within these steps are influenced by aspects such as the type of website (e.g. static, dynamic, mobile, etc.), its size, complexity, and technologies used to create it (e.g. HTML, PDF, etc.), how much knowledge the evaluator has to how the website was or is being developed (e.g. white-box,
<shadi> black-box, gray-box, etc.), as well as by the purpose of the evaluation (e.g. conformance check, detailed analysis, etc.).]]
S: Word smithing will use up time. Discuss issues
instead. Peter suggests focusing on process rather than conformance to
differentiate them.
... Alister, right direction?
A: IN right direction.
D: General remark that not sure everyonoe can read while on call, evaluating accessibility rather than conformance. Guidelines ahve conformity aspect in them, but process looks for accessibility. Not just knowing wherther something is comforming or not.
S: Doesn't accessibility come from guidelines.
<AGarrison> +1
D: Still uncomfortable with conformance, can't state that website conforms as a whole. Sets expectation that we'll determine whether website conforms.
S: Extant website conforms is the direction we want to go. Will go back and look at wording.
E: About Conformance. Want to be careful how we
use it, not that can't use it.
... Still can use if we feel it is necessary.
S: Extant to which website conforms.
<ericvelleman> +1
A: 5B in third paragraph, provide conformance evaluation, In my mind what this is about. Our methodology confirms whether conformance statement is true or not. If take sample and it's okay, probably claim is ture.
A: If not, throws out their claim. Rather like 5B, because it makes that distinction.
S: Not sure the issue.
A: If go to 5B, instead of our saying if it conforms, we can say whether or not someone's statement is true.
S: Not only use of method. Also use for
evaluating conformance.
... Link to Seciton 5B or itself.
A: Whether or not we can evaluate
Not clear where you are.
A: Second paragrahp in second block, provide guidance on how to make accessibility conformance statements.
K: Read this as defniing a particular term. Another statement someone else is making is something else.
S: Is your comment that we only talk about evaluatioin statement but that don't mention it can be used to confirm statement from someone else?
A: Can't make a conformance claim ourselves.
A: S: Not sure want to make it an absolute term that can't make conformance claim. Sentence above it...claim can't be made on selection of sample of pages. When methodology used alone and sample cna't make a claim, ut if part of QA then it is possible
that you can continually use to confirm your conformacne claim. It is possible. In most cases espec. as third party it would be difficult to extrapolate to entire website.
K: Would not want to get into political question of using the methodology to review soemone else's claim. Instead describe use as in report. Wouldn't want to suggest proving or disproving someone's statement.
S: Any thoughts on updated text?
K: First paragraph feels too long, should be
stronger. Not try to justify every success criteria. make first paragraph two
sentences. C claims can only be made wrt WCAG 2.o. Can only made when
specific
... pages found to conform.
S: Or accessible alternatives. "Each and every conformance requirement" Completely is preferred.
K: want to be sure that that it is clear.
S: Only a sample...in majority okay?
K: Place where WCAG EM came from, how do we deal with situation where looking at some not all pages. Appropriate to be stronger than many uses.
E: Will be more direct in text.
K: While WCAG-EM and reports coming out cna be used when eacy and every page is evaluated, WCAG-EM was designed for situation when subsample is evaluated.
S: Let's go with Majority of use cases.
K: Also improtant to say, Make point that it is rarely possible (impossible) to evaluate every page, therefore we have WCAG-EM. Therefore can't make conformance claim in most cases.
S: but I do feel that not possible is too heavy handed. When QA and process used together can make claim. Not possible, will never lead to conformance claim is too restrictive.
K; IN a significant number of cases, bec of size and complexity, not reasonable to evaluate every page and every user interaction, thus not possible to make a conformance claim per WCAG.
S: WCAG does not say that have to evaluate every
page. Just reasonably sure that it is valid. Does not mean that each has to be
evaluated.
... Example, script that provides content; All built in similar fashion, can
say it conforms bec confident in how it's being generated to maek that
claim.
K: I understand that, makes sense, but, if all pages generated from a db and db includes icons and alt text, some of entries are without alt text, then nothing wrong with code that is generated, but result may not be.
<shadi> [[A determination of conformance can be made based on best knowledge. If a page of this type is monitored and repaired (non-conforming content is removed or brought into conformance) within two business days, then a determination or claim of conformance can be made since, except for errors in externally contributed content which are corrected or removed when encountered, the page conforms. No conformance claim can be made if it is not possible to monitor or correct
<shadi> non-conforming content]]
K: "Page satisfies all success criteria." Othrwise making an informed guess, has a high liklihood of confidence, but not a claim for every page.
S: Not best guess, best knowledge. Need additional quality assurance to remove bug in two business days...can have website with claim without testing every page. Agree that in majority of cases, need to describe
<AGarrison> Alistair - is this a methodology for determining a confidence level in a (possibly hypothetical) conformance claim for the whole website? Just a thought...
S: what it means
E: If in first paragraph state removed evaluation
claim statement indicate the relation to WCAG 2.0 conformance claimes, have to
look at every page, in effect can look at every page, but if not possible make
a good sample, based on sample what can we let people say about sample.
... Took conformance and claim out. Do we want to change this more, say can't
make any statement at all? or is this sufficient?
K: Output of WCAGEM is not a WCAG conformance claim, but a conformance statement.
S: Maybe front-load information.
MJ: When talking about best knowledge a good appraoch, can't check every page.
S: Can check with WCAG working group, but without
other mechanisms in place can make partial conformance claim.
... Output of this methodology is analysis of a sample which would not allow
you to extrapolate claim for entire site.
... How does that sound.
K: sound good to me. Although "because" language is useful.
S: Have some "because" at beginning in frist paragrahp, in second say why can't.
K: Would say "therefore" to connect the two.
S: We have "thus"...maybe needs to be earlier. Pull info to the front. More directed.
E: Think that's a good solution. Take to front, makes clear from beginning.
A: Conformance word striekds me that rasonabley obviouls that would give undersatnding ofwebsite, on other hadn saying ithe methodolgy woun't tell you about conformance. people will be
<korn> What about the title being: "WCAG Evaluation Methodology"????
S: Extant to which methodology conforms. In very rare cases this will lead to a conformance claim.
D: Am glad that Alister mention title
incoherence, conformance mixed up with C in WCAG, which is content. Think about
title that does't have conformance in it.
... Will think people will think it's about assessing website conformance,
which is misleading.
S: It is an assessment of conformance, just won't be able to claim conformance very oftern.
<Detlev> agree with Peter
K: WCAG evaluation methodology. In rare cases will be able to review every page. Lead with WCAG methodology.
Tim; WCAG does relate to webpages.
<Detlev> Well the title change would reflect a change in wemphasis that has emerged over thelast few month..
S: Wound up with WCAG-EM bec Evaluation Methodology. Would have to be very strong caser to reopen that discussion, would take 2 weeks. Disntinction between Conformance and Conformance Evaluation is asubutel one.
<ericvelleman> Thank you very much Peter for all the work you spent in this group. Great input. Many thanks
S: Want to put high threshold for reopening
discussion on revising title.
... Thanks for much for your comments and discussion. We will do our best to
continue with that. I hope our paths cross here and there.
S; Peter feel free to send any comments, didn't talk about scope of responsibility, web applications text changes. Will make sure your comments won't be lost.