See also: IRC log
Eric: welcome to the call
... tried to incorporate all changes into the recent Editor Draft
<ericvelleman> http://www.w3.org/WAI/ER/methodology/
<ericvelleman> http://www.w3.org/WAI/ER/conformance/comments-20130226
Eric: key topics to discuss, likely on the
mailing list in the coming week...
... New conformance section, changes in 1b, how to choose a sample
... in document, noted where the change comes from (e.g. disposition of
comments), why the change, etc.
<ericvelleman> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#conformanceintro
Shadi: issue with the title of the section...
"Conformance in the context of WCAG-EM"
... may need to tweek the wording of the title. Also comment AC81 (from Peter)
and the use of the word "statements"
... also should be clear we aren't proposing a new/different conformance model
here.
<Detlev> Why not "Step 5.b: Provide an Accessibility Evaluation Statement"?
Fredrick: worried about using the word "conformance" itself here. Could cause confusion.
Eric: uses the term "Accessibility Evaluation Conformance Statement", to try to be distinct.
Fredrick: Prefer to not use "conformcne" here; so "Accessibility Evaluation Statement"
Peter (from earlier in the thread): purpose / goal of this new section is to make clear that WCAG conformance claims cannot be made, by their definition, to our output, as our output is intentionally a sampling.
<Vivienne> Sorry i was late in from a meeting . Can someone fill me in on what section we're discussing?
<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#conformanceintro
Peter (still from earlier): therefore we want to lay out the issues in this section, and set forth the idea that the output of WCAG-EM is a statement addressing the extent to which conformance was found.
Shadi: likes what Tim is saying - avoiding the
words "conformance" and "conformance claim" as much as possible.
... Q (to Peter perhaps): is it that we are trying to define the scope of
WCAG-EM here?
... thinks that the second paragraph is more important; maybe flip the order.
Seems we are more describing the limitations & scope, rather than any
conformance requirements. Hence the disconnect of the title from the contents
of the section.
Peter: appreciate to try to avoid conformance but
important to talk about it when we introduce the evaluation statements or
such
... because conforms is used inherently and needs to be addressed straight
on
... need to address the many ways people will use this
... agree that title could be improved
... maybe "output from WCAG-EM"
<shadi> ...but critical to have description of these aspects of the document
Eric: suggests we suspend further discussion of this section here, and shift it to the mailing list.
<MartijnHoutepen> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#step1b
<MartijnHoutepen> Define the goal of the evaluation
<shadi> http://www.w3.org/TR/WCAG-EM/#step1b
Eric: changing 1b to be about "additional requirments", beyond what is set forth in WCAG-EM
Peter: might be good to list examples of some of
the things that are additional
... not sure if the term "requirements" is precise
... maybe better "aspects" as they are more content than process aspects
Detlev: might this also be a place to define an
eval that is more limited (e.g. to explicitly NOT evaluate against some SCs)
... or to evaluation portions of a page (since WCAG applies to entire
pages)
... thus you might skip "Page title", etc. since it wouldn't apply to a
portion of a page.
+1 to this from Peter
Shadi: paraphrasing Detlev: if you have a bunch of templates / code objects / pieces of content, and you want to evaluate those on their own, and make an eval statement about those, you would want to set that forth here in step 1b.
Eric: this would be the place to put such a thing, though maybe a bit different of an "addition" (being a subtraction) than we had at first thought.
Peter: perhaps "define any modifications to the
evaluation process"
... then in examples, we might have the one Detlev set forth
Shadi: need to look back at the section "Using
this methodology" and "scope" and "applicability". Those sections also list
types of websites.
... we encourage the use of this methodology in different situations, but the
primary focus is the website as a whole, as presented to user.
... lots of disucssion about incremental eval, development eval, regression
eval. But... adding this (only) in step 1b may break what we said earlier.
Perhaps it should (instead) be said earlier?
Eric: pausing this disucssion here, to shift it
to mailing list.
... Primary observation from here is from Detlev - focusing eval to a subset
of a page
Shadi: in the spirit of "addressing things head on", it might really be good under "scope of applicability" to have a section saying "you can still apply this guidance in other situations" and list them (such as Detlev's "part of a page", also "during development", etc.)
<MartijnHoutepen> +1
Eric: took outcome of earlier discussion (even
from last year) and put it here. Not sure if it should be here, or in different
form...
... added "factors related to likelihood of a sample being representative"
Peter: likes section, but remains concerned with
the lack of statistic rigor guidance here for how to determine the "likelihood
of a sample being representative"
... suggests we might again do more website testing under WCAG-EM with a focus
on how similar/different the results are, to inform how much further guidance
may be needed here for "representative sample" and "confidence"
Eric: concerned that there are so many ways in
literature for doing sampling, we don't want to pick one above others
... +1 to doing another test run again
Detlev: evaluators will need some guidance for
following processes (not just "put in URL of a page"). Menus, forms,
lightboxes, etc.
... only way to include such generated pages is to refer to the "base page"
and how to get to the "generated page"
... sets forth doing it the "default" way, and then also suggests looking at
some variants (e.g. input error)
... an obvious can of worms for very complex sites - all the potential
complexity, and need to draw the line somewhere, make a judgement call as to
how far to go.
Eric: this is one place in the methodology where we really take people by the hand. Originally just had a little text here; now we are doing much more leading people through "how you do this". Perhaps we should look at doing that amount of leading in other places.
Detlev: risk is that we are too descriptive - too focused on a particular type of website. Need to think about that too.
Peter: consider "at least" in number 2; make clear that going through the default branch is a minimum.
Eric: discussion about the prescriptive-ness of this... anyone think this is too much? That we should do more of that level elsewhere?
Peter: seems like the right direction but maybe not the right level of prescriptiveness
<shadi> ...a bit more elsewhere may be helpfull too but need to check
Kathy: good to have this level of detail in this section, but have to be careful. If you lead too much by the hand, it may not fit certain circumstances. So agrees - evaluate level of prescriptive-ness on a section-by-section basis
Vivienne: likes level of detail here. Finds with
clients that guidance around complete processes is necessary.
... thinks we will find other areas where more point-by-point instructions
will be very helpful.
Mike: thinks we keep running into issue of how to
define a representative sample. What is sufficient... A little reluctant to
make things too detailed, but this seems like a good place for this level of
detail.
... maybe have examples here as well.
Eric: (summarizing) not hearing people saying
they don't want this level of detail; should discuss on mailnig list whether we
want this much, and where else in document we want this level of detail (or at
least more detail than is there presently)
... will start discussions of each of these on the mailing list (after Dutch
dinner), and hopes we continue discussions there.
... next week, hopes to have a survey so we can close down some of these
discussions
<Mike_Elledge> Bravo!
Shadi: really great work Eric