13:57:18 <RRSAgent> RRSAgent has joined #eval
13:57:18 <RRSAgent> logging to http://www.w3.org/2013/09/05-eval-irc
13:57:20 <trackbot> RRSAgent, make logs world
13:57:20 <Zakim> Zakim has joined #eval
13:57:22 <trackbot> Zakim, this will be 3825
13:57:22 <Zakim> ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 3 minutes
13:57:23 <trackbot> Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
13:57:23 <trackbot> Date: 05 September 2013
13:58:15 <Kathy> Kathy has joined #eval
13:58:45 <shadi> zakim, this is eval
13:58:45 <Zakim> ok, shadi; that matches WAI_ERTWG(Eval TF)10:00AM
13:58:54 <Zakim> + +31.30.239.aaaa
13:58:55 <shadi> zakim, who is on the phone?
13:58:55 <Zakim> On the phone I see Mike, Liz, Shadi, +31.30.239.aaaa
13:58:59 <Zakim> + +1.978.261.aabb
13:59:12 <Kathy> zakim aabb is me
13:59:12 <Zakim> +MartijnHoutepen
13:59:16 <ericvelleman> Zakim, aaaa is me
13:59:16 <Zakim> +ericvelleman; got it
13:59:25 <Kathy> zakim, aabb is Kathy
13:59:25 <Zakim> +Kathy; got it
14:00:04 <Kathy> zakim, mute me
14:00:04 <Zakim> Kathy should now be muted
14:00:38 <korn> korn has joined #eval
14:01:17 <Kathy> ack me
14:01:29 <Kathy> zakim, mute me
14:01:29 <Zakim> Kathy should now be muted
14:01:41 <Zakim> +Detlev
14:01:43 <shadi> zakim, who is on the phone?
14:01:43 <Zakim> On the phone I see Mike, Liz, Shadi, ericvelleman, Kathy (muted), MartijnHoutepen, Detlev
14:02:29 <Detlev> zakim, mute me
14:02:29 <Zakim> Detlev should now be muted
14:03:13 <shadi> scribe: shadi
14:03:14 <MartijnHoutepen> Zakim, mute me
14:03:14 <Zakim> MartijnHoutepen should now be muted
14:03:40 <Zakim> + +1.650.506.aacc
14:03:42 <shadi> Eric: welcome to the call
14:03:50 <shadi> ...tried to incorporate all changes into the recent Editor Draft
14:04:03 <korn> Zakim, aacc has Peter_Korn
14:04:03 <Zakim> +Peter_Korn; got it
14:04:43 <shadi> scribe: Peter
14:04:43 <shadi> scribenick: korn
14:05:42 <Sarah_Swierenga> Sarah_Swierenga has joined #eval
14:05:46 <ericvelleman> http://www.w3.org/WAI/ER/methodology/
14:06:01 <shadi> agenda?
14:06:14 <korn> topic: new editor draft
14:06:20 <Zakim> +Sarah
14:06:26 <ericvelleman> http://www.w3.org/WAI/ER/methodology/
14:06:44 <ericvelleman> http://www.w3.org/WAI/ER/conformance/comments-20130226
14:08:21 <korn> Eric: key topics to discuss, likely on the mailing list in the coming week...
14:08:36 <korn> Eric: New conformance section, changes in 1b, how to choose a sample
14:09:03 <korn> Eric: in document, noted where the change comes from (e.g. disposition of comments), why the change, etc.
14:09:04 <Zakim> +Tim_Boland
14:10:54 <korn> topic: Discussion start about new section Conformance in the context of WCAG-EM
14:11:04 <ericvelleman> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#conformanceintro
14:11:11 <Vivienne> Vivienne has joined #eval
14:11:16 <korn> q
14:11:17 <korn> q+
14:12:11 <shadi> q+
14:12:12 <Zakim> +[IPcaller]
14:12:17 <Vivienne> zakim, IPcaller is me
14:12:17 <Zakim> +Vivienne; got it
14:13:09 <ericvelleman> q?
14:14:22 <korn> q-
14:14:46 <korn> Shadi: issue with the title of the section... "Conformance in the context of WCAG-EM"
14:15:17 <Vivienne> zakim, mute me
14:15:17 <Zakim> Vivienne should now be muted
14:15:26 <Zakim> -Mike
14:15:26 <korn> Shadi: may need to tweek the wording of the title.  Also comment AC81 (from Peter) and the use of the word "statements"
14:16:01 <korn> Shadi: also should be clear we aren't proposing a new/different conformance model here.
14:16:05 <Zakim> +Mike
14:18:06 <Detlev> Why not "Step 5.b: Provide an Accessibility Evaluation Statement"?
14:18:22 <shadi> q-
14:19:47 <korn> Fredrick: worried about using the word "conformance" itself here.  Could cause confusion.
14:19:48 <shadi> q+
14:20:11 <korn> Eric: uses the term "Accessibility Evaluation Conformance Statement", to try to be distinct.
14:20:23 <korn> q+
14:21:09 <korn> Fredrick: Prefer to not use "conformcne" here; so "Accessibility Evaluation Statement"
14:22:28 <korn> Peter (from earlier in the thread): purpose / goal of this new section is to make clear that WCAG conformance claims cannot be made, by their definition, to our output, as our output is intentionally a sampling.
14:22:39 <Vivienne> Sorry i was late in from a meeting .  Can someone fill me in on what section we're discussing?
14:23:03 <shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#conformanceintro
14:23:11 <korn> Peter (still from earlier): therefore we want to lay out the issues in this section, and set forth the idea that the output of WCAG-EM is a statement addressing the extent to which conformance was found.
14:23:48 <korn> q?
14:24:48 <korn> Shadi: likes what Tim is saying - avoiding the words "conformance" and "conformance claim" as much as possible.
14:25:26 <korn> Shadi: Q (to Peter perhaps): is it that we are trying to define the scope of WCAG-EM here?
14:26:14 <korn> Shadi: thinks that the second paragraph is more important; maybe flip the order.  Seems we are more describing the limitations & scope, rather than any conformance requirements.  Hence the disconnect of the title from the contents of the section.
14:26:18 <korn> ack s
14:26:25 <Detlev> q+
14:26:28 <shadi> scribe: shadi
14:27:13 <shadi> Peter: appreciate to try to avoid conformance but important to talk about it when we introduce the evaluation statements or such
14:27:20 <Detlev> q-
14:27:43 <shadi> ...because conforms is used inherently and needs to be addressed straight on
14:28:07 <shadi> ...need to address the many ways people will use this
14:28:51 <shadi> ...agree that title could be improved
14:29:05 <shadi> ...maybe "output from WCAG-EM"
14:29:15 <korn> scribe: korn
14:29:21 <shadi> ...but critical to have description of these aspects of the document
14:29:42 <korn> Eric: suggests we suspend further discussion of this section here, and shift it to the mailing list.
14:30:05 <korn> topic: Discussion start about changes to Step 1.b
14:30:24 <MartijnHoutepen> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#step1b
14:30:56 <MartijnHoutepen> Define the goal of the evaluation
14:30:58 <shadi> http://www.w3.org/TR/WCAG-EM/#step1b
14:31:47 <korn> Eric: changing 1b to be about "additional requirments", beyond what is set forth in WCAG-EM
14:32:13 <korn> q+
14:32:52 <ericvelleman> q?
14:32:58 <Detlev> q+
14:33:03 <shadi> scribe: shadi
14:33:43 <shadi> Peter: might be good to list examples of some of the things that are additional
14:33:46 <shadi> ...not sure if the term "requirements" is precise
14:34:00 <korn> q-
14:34:05 <Detlev> ack me
14:34:05 <shadi> ...maybe better "aspects" as they are more content than process aspects
14:34:09 <korn> scribe: korn
14:34:39 <korn> Detlev: might this also be a place to define an eval that is more limited (e.g. to explicitly NOT evaluate against some SCs)
14:34:56 <korn> Detlev: or to evaluation portions of a page (since WCAG applies to entire pages)
14:35:08 <Zakim> -Tim_Boland
14:35:12 <ericvelleman> q?
14:35:13 <korn> Detlev: thus you might skip "Page title", etc. since it wouldn't apply to a portion of a page.
14:35:29 <korn> +1 to this from Peter
14:35:41 <shadi> q+
14:35:45 <shadi> ack me
14:36:39 <korn> Shadi: paraphrasing Detlev: if you have a bunch of templates / code objects / pieces of content, and you want to evaluate those on their own, and make an eval statement about those, you would want to set that forth here in step 1b.
14:37:34 <korn> Eric: this would be the place to put such a thing, though maybe a bit different of an "addition" (being a subtraction) than we had at first throught.
14:37:42 <korn> /throught/thought
14:37:55 <korn> q+
14:38:16 <korn> Peter: perhaps "define any modifications to the evaluation process"
14:38:33 <korn> Peter: then in examples, we might have the one Detlev set forth
14:38:35 <shadi> q+
14:38:38 <korn> q-
14:38:41 <shadi> ack me
14:39:05 <korn> Shadi: need to look back at the section "Using this methodology" and "scope" and "applicability".  Those sections also list types of websites.
14:39:27 <korn> Shadi: we encourage the use of this methodology in different situations, but the primary focus is the website as a whole, as presented to user.
14:40:05 <korn> Shadi: lots of disucssion about incremental eval, development eval, regression eval.  But... adding this (only) in step 1b may break what we said earlier.  Perhaps it should (instead) be said earlier?
14:40:13 <Detlev> Zakim, mute me
14:40:13 <Zakim> Detlev should now be muted
14:40:14 <korn> Eric: pausing this disucssion here, to shift it to mailing list.
14:40:43 <shadi> q+
14:40:57 <korn> Eric: Primary observation from here is from Detlev - focusing eval to a subset of a page
14:42:13 <korn> Shadi: in the spirit of "addressing things head on", it might really be good under "scope of applicability" to have a section saying "you can still apply this guidance in other situations" and list them (such as Detlev's "part of a page", also "during development", etc.)
14:42:17 <MartijnHoutepen> +1
14:42:27 <korn> topic: Discussion start about changes to Step 3: Select a Representative Sample
14:43:12 <korn> Eric: took outcome of earlier discussion (even from last year) and put it here.  Not sure if it should be here, or in different form...
14:43:34 <korn> Eric: added "factors related to likelihood of a sample being representative"
14:43:53 <korn> q+
14:44:16 <shadi> ack me
14:45:31 <korn> Peter: likes section, but remains concerned with the lack of statistic rigor guidance here for how to determine the "likelihood of a sample being representative"
14:48:06 <korn> Peter: suggests we might again do more website testing under WCAG-EM with a focus on how similar/different the results are, to inform how much further guidance may be needed here for "representative sample" and "confidence"
14:48:34 <korn> Eric: concerned that there are so many ways in literature for doing sampling, we don't want to pick one above others
14:48:47 <korn> Eric: +1 to doing another test run again
14:49:14 <korn> topic: Discussion start about changes to Step 3.d: Include Complete Processes in the Sample
14:49:29 <Detlev> ack me
14:49:35 <korn> q-
14:50:08 <korn> q+
14:50:43 <korn> Detlev: evaluators will need some guidance for following processes (not just "put in URL of a page").  Menus, forms, lightboxes, etc.
14:51:11 <Zakim> -Sarah
14:51:12 <korn> Detlev: only way to include such generated pages is to refer to the "base page" and how to get to the "generated page"
14:51:51 <korn> Detlev: sets forth doing it the "default" way, and then also suggests looking at some variants (e.g. input error)
14:52:32 <korn> Detlev: an obvious can of worms for very complex sites - all the potential complexity, and need to draw the line somewhere, make a judgement call as to how far to go.
14:53:20 <korn> Eric: this is one place in the methodology where we really take people by the hand.  Originally just had a little text here; now we are doing much more leading people through "how you do this".  Perhaps we should look at doing that amount of leading in other palces.
14:53:25 <korn> /palces/places
14:53:40 <korn> Detlev: risk is that we are too descriptive - too focused on a particular type of website.  Need to think about that too.
14:54:33 <korn> Peter: consider "at least" in number 2; make clear that going through the default branch is a minimum.
14:54:50 <korn> Eric: discussion about the prescriptive-ness of this... anyone think this is too much?  That we should do more of that level elsewhere?
14:54:53 <korn> Q+
14:55:19 <shadi> scribe: shadi
14:55:51 <shadi> Peter: seems like the right direction but maybe not the right level of prescriptiveness
14:55:57 <Kathy> q+
14:55:58 <Vivienne> q+
14:56:02 <korn> q-
14:56:09 <korn> scribe: korn
14:56:11 <Vivienne> ack me
14:56:13 <Kathy> ack me
14:56:13 <shadi> ...a bit more elsewhere may be helpfull too but need to check
14:56:48 <Mike_Elledge> q+
14:56:51 <Kathy> zakim, mute me
14:56:51 <Zakim> Kathy should now be muted
14:56:53 <korn> Kathy: good to have this level of detail in this section, but have to be careful.  If you lead too much by the hand, it may not fit certain circumstances.  So agrees - evaluate level of prescriptive-ness on a section-by-section basis
14:56:55 <ericvelleman> q?
14:57:23 <korn> Vivienne: likes level of detail here.  Finds with clients that guidance around complete processes is necessary.
14:57:48 <korn> Vivienne: thinks we will find other areas where more point-by-point instructions will be very helpful.
14:57:57 <Vivienne> q-
14:58:03 <Vivienne> zakim, mute me
14:58:03 <Zakim> Vivienne should now be muted
14:58:31 <korn> Mike: thinks we keep running into issue of how to define a representative sample.  What is sufficient...  A little reluctant to make things too detailed, but this seems like a good place for this level of detail.
14:58:52 <korn> Mike: maybe have examples here as well.
14:59:41 <korn> Eric: (summarizing) not hearing people saying they don't want this level of detail; should discuss on mailnig list whether we want this much, and where else in document we want this level of detail (or at least more detail than is there presently)
15:00:34 <korn> Eric: will start discussions of each of these on the mailing list (after Dutch dinner), and hopes we continue discussions there.
15:00:59 <korn> Eric: next week, hopes to have a survey so we can close down some of these discussions
15:01:05 <korn> topic: Other issues
15:01:11 <Mike_Elledge> Bravo!
15:01:12 <korn> Shadi: really great work Eric
15:01:32 <Liz> bye
15:01:37 <Mike_Elledge> Ciao
15:01:43 <Vivienne> good night all
15:01:45 <Kathy> bye
15:01:46 <Detlev> bye!
15:01:49 <Zakim> -Mike
15:01:54 <Zakim> -Kathy
15:01:56 <Zakim> -Detlev
15:02:02 <Zakim> -Vivienne
15:02:05 <Zakim> -ericvelleman
15:02:06 <Zakim> -Shadi
15:02:08 <Zakim> -MartijnHoutepen
15:02:10 <Zakim> -Liz
15:02:13 <ericvelleman> ericvelleman has left #eval
15:03:55 <shadi> regrets: Moe
15:04:09 <shadi> trackbot, end meeting
15:04:09 <trackbot> Zakim, list attendees
15:04:09 <Zakim> As of this point the attendees have been Mike, Liz, Shadi, +31.30.239.aaaa, +1.978.261.aabb, MartijnHoutepen, ericvelleman, Kathy, Detlev, Peter_Korn, Sarah, Tim_Boland, Vivienne
15:04:17 <trackbot> RRSAgent, please draft minutes
15:04:17 <RRSAgent> I have made the request to generate http://www.w3.org/2013/09/05-eval-minutes.html trackbot
15:04:18 <trackbot> RRSAgent, bye
15:04:18 <RRSAgent> I see no action items