14:04:35 <RRSAgent> RRSAgent has joined #eval
14:04:36 <RRSAgent> logging to http://www.w3.org/2013/09/19-eval-irc
14:04:37 <trackbot> RRSAgent, make logs world
14:04:38 <Zakim> Zakim has joined #eval
14:04:40 <trackbot> Zakim, this will be 3825
14:04:40 <Zakim> ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start 4 minutes ago
14:04:41 <trackbot> Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
14:04:41 <trackbot> Date: 19 September 2013
14:05:36 <MartijnHoutepen> scribe: korn
14:05:40 <shadi> shadi has joined #eval
14:06:26 <korn> Topic: Welcome
14:06:59 <Sarah_Swierenga> Sarah_Swierenga has joined #eval
14:07:10 <MartijnHoutepen> Zakim, mute me
14:07:10 <Zakim> sorry, MartijnHoutepen, I don't know what conference this is
14:07:18 <MartijnHoutepen> Zakim, this is 3925
14:07:20 <Zakim> sorry, MartijnHoutepen, I do not see a conference named '3925' in progress or scheduled at this time
14:07:20 <korn> Topic: Walkthrough smaller changes in editor draft
14:07:22 <MartijnHoutepen> Zakim, this is 3825
14:07:22 <Zakim> ok, MartijnHoutepen; that matches WAI_ERTWG(Eval TF)10:00AM
14:07:22 <Zakim> +[IPcaller.a]
14:07:29 <Vivienne> zakim, IPcaller.a is me
14:07:29 <Zakim> +Vivienne; got it
14:07:30 <shadi> zakim, who is on the phone?
14:07:31 <Zakim> On the phone I see Kathy, +31.30.239.aaaa, +1.313.322.aabb, Peter_Korn, +31.30.239.aacc, [IPcaller], Vivienne
14:07:32 <Zakim> + +1.517.432.aadd
14:07:58 <MartijnHoutepen> Zakim, aacc is me
14:07:58 <Zakim> +MartijnHoutepen; got it
14:08:00 <ericvelleman> Zakim, aaaa is me
14:08:00 <Zakim> +ericvelleman; got it
14:08:11 <Sarah_Swierenga> aadd is me
14:08:22 <shadi> zakim, vivienne is shadi
14:08:22 <shadi> zakim, ipcaller is Vivienne
14:08:22 <Zakim> +shadi; got it
14:08:23 <Zakim> +Vivienne; got it
14:08:26 <MartijnHoutepen> Zakim, aadd is Sarah_Swierenga
14:08:26 <Zakim> +Sarah_Swierenga; got it
14:08:26 <shadi> zakim, who is on the phone?
14:08:27 <Zakim> On the phone I see Kathy, ericvelleman, +1.313.322.aabb, Peter_Korn, MartijnHoutepen, Vivienne, shadi, Sarah_Swierenga
14:08:57 <MartijnHoutepen> Zakim, aabb is Mike_Elledge
14:08:57 <Zakim> +Mike_Elledge; got it
14:09:05 <MartijnHoutepen> Zakim, mute me
14:09:05 <Zakim> MartijnHoutepen should now be muted
14:10:17 <ericvelleman> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903
14:10:20 <Zakim> + +1.617.480.aaee
14:10:22 <shadi> agenda?
14:10:40 <Vivienne> zakim, mute me
14:10:40 <Zakim> Vivienne should now be muted
14:11:47 <korn> Eric working on a new editor's draft, so woud like to skip over topics we discussed earlier.
14:12:48 <ericvelleman> q?
14:13:36 <Vivienne> q+
14:13:49 <Vivienne> zakim, ack me
14:13:49 <Zakim> unmuting Vivienne
14:13:50 <Zakim> I see no one on the speaker queue
14:15:50 <korn> Vivienne - suggest we say "this methodology describes the min. req. for one approach to evaluating a webiste"
14:16:06 <korn> q+
14:17:33 <shadi> peter: keep in mind that every part of this note is advisory
14:17:51 <shadi> ...but vivienne's wording highlights this well
14:18:00 <Mike_Elledge> q+
14:18:03 <korn> ack k
14:18:57 <korn> Mike: ~+1 to Peter.  EvalTF is a pretty "weighty" document.  Anything we can do to help folks understand that this is simply 1 way of doing this.  Also keep the document streamlined.
14:19:08 <korn> q+
14:19:12 <korn> ack M
14:19:32 <Mike_Elledge> +
14:20:00 <shadi> peter: could try progressive disclosure approach
14:20:16 <Kathy> +1
14:20:22 <shadi> ...first describe methodology at a higher level
14:20:22 <shadi> ...then drill deeper into specifics later on
14:20:51 <shadi> ...not sure what specific changes are needed for this
14:21:07 <korn> ack k
14:21:21 <korn> Kathy - a verbal +1 as well.
14:22:10 <korn> Kathy: adding a summary section at the top might be an approach to this.
14:22:16 <korn> q+
14:22:27 <Mike_Elledge> +1
14:22:39 <korn> Eric: another approach as this is throughout the docment is to put more into appendicies.
14:22:46 <korn> Kathy: does seem overwhelming.
14:23:33 <shadi> q+
14:23:45 <korn> ack k
14:24:04 <korn> Peter suggested expand/collapse widgets as an option - see WCAG2ICT doc.
14:24:22 <shadi> http://www.w3.org/WAI/ACT/deliverables#eval_guide
14:24:27 <korn> Peter: take a look at http://www.w3.org/WAI/GL/2013/WD-wcag2ict-20130905/accordion
14:25:27 <korn> Shadi: many folks we expect to use this document won't be the typical readers of a W3C spec.
14:26:37 <Mike_Elledge> q+
14:27:07 <Bim> Bim has joined #eval
14:27:22 <shadi> ack me
14:27:58 <korn> Shadi: W3C / WAI has funding to work on interactive guides for evaluators.  Could include WCAG-EM, as well as other educational materials for evaluation
14:28:36 <korn> Mike: discussion reminds him of how handy the old WCAG 1.0 "wallet card" was.
14:28:56 <shadi> http://www.w3.org/WAI/WCAG20/glance/
14:29:08 <shadi> called WCAG 2 at a Glance
14:30:41 <ericvelleman> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#step3f
14:30:42 <korn> Topic: (sub-topic) Step 3F
14:31:19 <korn> q+
14:31:23 <Kathy> q+
14:31:50 <korn> Eric: discuss optional/non-optionality of this step
14:32:48 <Sarah_Swierenga> q+
14:32:54 <korn> ack M
14:33:29 <MartijnHoutepen> q+
14:33:50 <korn> ack k
14:34:59 <korn> Peter: rather than the optional/non-optional-ality of this section, we should use this section to CAPTURE the fact that redundant pages were found.  E.g. a site with 3 unique pages and 500 all of the same template is a site on which the evaluation will have a high degree of confidence - what is found on one/two of the 500 would likely apply to all.
14:35:12 <korn> ... so we should retain the knowledge of the reduncancy, not just toss it out.
14:35:27 <korn> q-
14:35:43 <korn> Kathy: agrees we shouldn't take this out; reasons to have redundancy in the sample.  Esp. if the site has different developers, etc.
14:35:43 <korn> q+
14:35:57 <MoeKraft> MoeKraft has joined #eval
14:36:00 <korn> Kathy: so there are reasons to have redundancy in sample.
14:36:36 <korn> Sarah: reading the exact language, and what we are saying in 3f - if you happen to pick the same page as you did earlier, you would then select a different one (don't evaluate the precisely same page twice)
14:36:42 <Zakim> + +1.978.899.aaff
14:36:43 <Zakim> - +1.617.480.aaee
14:36:52 <korn> ... "... thta are identical with other web pages in the sample..."
14:36:58 <korn> /thta/that
14:36:59 <MoeKraft> zakim,aaff is Moe
14:36:59 <Zakim> +Moe; got it
14:37:20 <korn> Eric: this discussion illustrates that we need to work on the text, as people on the call are reading the text differently.
14:37:21 <MoeKraft> zakim, aaee was Moe
14:37:21 <Zakim> I don't understand 'aaee was Moe', MoeKraft
14:37:47 <korn> Eric: thinks we all agree if we mean the "same page" we should take it out.
14:38:47 <MartijnHoutepen> ack me
14:39:24 <korn> Peter: interpretation of "... that are identical with other web pages in the sample..." as meaning the precisely same page is not consistent with "Filter the sample to eliminate excessive redundancies." (as "excessive" suggests that sometimes looking at the same/identical page multiple times is OK, so long as you aren't doing it "excessively"
14:39:27 <korn> q-
14:39:49 <korn> Eric: we all agree that taking identical pages out is OK?
14:39:51 <korn> q+
14:39:59 <korn> ack S
14:41:00 <MartijnHoutepen> Zakim, mute me
14:41:00 <Zakim> MartijnHoutepen should now be muted
14:42:02 <korn> Peter: put "uniqueness" of items in the sample into step 3.e.  Then 3.f can be a discussion about "appearing to be similar", and about hetero-/homo-genaity
14:42:15 <Sarah_Swierenga> +1
14:42:22 <Vivienne> Yes, I agree with the other callers about this
14:42:27 <korn> Eric: thought's on Peter's suggestion?
14:42:35 <ericvelleman> +1
14:42:42 <MartijnHoutepen> +1
14:43:19 <korn> Topic: (sub-topic) Section 5 introduction
14:43:22 <ericvelleman> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#step5
14:47:15 <korn> Topic: (sub-topic) move now to discuss performance score
14:47:49 <korn> q+
14:48:18 <Kathy> q+
14:49:22 <shadi> peter: if we don't have a common way to score, there can't be comparability
14:49:42 <korn> ack k
14:49:47 <Vivienne> q+
14:49:47 <shadi> ...but again don't think should have performance score whatsoever
14:50:02 <korn> q-
14:50:46 <korn> Kathy: agrees with Peter - so much that can go into a score (not just WCAG 2.0 compliance, but severity of issue, impact on various different PwDs, needs) it get so very difficult.
14:50:57 <Kathy> http://jimthatcher.com/favelets/
14:51:22 <shadi> q+
14:51:25 <korn> ... maybe look at what Knowbility has done for their scoring, also Jim Thatcher's work.  So may want to lok at that, lots of different ideas there.
14:51:37 <Vivienne> zakim, ack me
14:51:37 <Zakim> I see shadi on the speaker queue
14:52:09 <korn> Vivienne: understand's what Peter is saying, his concerns.  But it's not just because people want it; in some situations it is required (for when you are comparing things).
14:52:10 <korn> q+
14:53:10 <korn> ... so need some way of recording things in a stastical manner.  How to compare the same website over time.  So has no problem with saying "scoring isn't part of the scope", but still providing some suggestions on this.  Because the question is going to get raised, etc.
14:53:48 <korn> ... we shouldn't just bury our heads in the sand and say "don't do it".  We need to deal with the issue.
14:54:03 <korn> Eric: we should find some way to give a score, even if we provide caveats around it.
14:54:24 <MoeKraft> q+
14:54:29 <shadi> ack me
14:54:38 <Vivienne> zakim, mute me
14:54:38 <Zakim> Vivienne should now be muted
14:54:55 <korn> Shadi: is going to be somewhat contradictory... (is really torn).  Agrees with lots of the arguments raised.  There are uses cases for scoring, use cases against.
14:55:21 <korn> ... we previous agreed that we wanted to provide scoring for comparing their own websites over time, not to compare different websites with each other.
14:55:54 <korn> ... so thinks it might be good that we don't provide a single approach, but rather several they can use.  We are far away from comparing different sites effectively.
14:56:33 <korn> ... concern is that people use this to compare against each other, in which case maybe we should dig our heads into the sand.
14:56:36 <ericvelleman> q?
14:58:31 <ericvelleman> We do not make it a step but make it a section about Performance scores, and say we will not make it. They can be helpful but are dubious. Then have appendix about resources of performance scores… Not part of WCAG-EM to make performance score.
14:59:09 <Zakim> -Kathy
14:59:22 <MartijnHoutepen> +1
14:59:27 <korn> Peter: perhaps a way forward: (a) we don't make this a formal step, (b) we talk about all of the challenges in performance scores - that they are NOT valid for inter-site comparisons but can be helpful for giving a quick look at improvements/regressions in a single site, and (c) have an appendix with resources for folks that want to use them ONLY for site improvement evaluation.
14:59:33 <Vivienne> sounds like it might work - depends on the wording
14:59:39 <Sarah_Swierenga> +1
15:00:06 <korn> Eric: will put this onto a survey; thinks this may be the best way forward.
15:00:09 <korn> q?
15:00:13 <korn> ack me
15:01:15 <korn> Moe: have no problem with this as a separate section, but want to echo some of the thoughts/ideas discussed.  May be that you have a single evaluator looking at multiple websites, using the same scoring methods to compare them (may also be for the same commissioner).
15:01:37 <korn> ... so comparing multiple websites could be OK.  But no problem in a separte section.
15:02:03 <korn> Topic: Further steps and agenda
15:02:28 <korn> Eric: working on a survey for final changes on the doc, & new editor draft.  You will first get the survey, then next week we can discuss outcomes from that survey.
15:03:35 <Sarah_Swierenga> bye everyone!
15:03:39 <Zakim> -Sarah_Swierenga
15:03:39 <Mike_Elledge> ciao!
15:03:43 <Zakim> -MartijnHoutepen
15:03:43 <ericvelleman> ericvelleman has left #eval
15:03:44 <Zakim> -Moe
15:03:45 <Vivienne> good night all
15:03:46 <Zakim> -Peter_Korn
15:03:46 <Zakim> -shadi
15:03:46 <Zakim> -Mike_Elledge
15:03:50 <Zakim> -ericvelleman
15:03:52 <Zakim> -Vivienne
15:03:54 <Zakim> WAI_ERTWG(Eval TF)10:00AM has ended
15:03:54 <Zakim> Attendees were Kathy, +31.30.239.aaaa, +1.313.322.aabb, Peter_Korn, +31.30.239.aacc, +1.517.432.aadd, MartijnHoutepen, ericvelleman, shadi, Vivienne, Sarah_Swierenga, Mike_Elledge,
15:03:54 <Zakim> ... +1.617.480.aaee, +1.978.899.aaff, Moe
15:04:19 <shadi> trackbot, end meeting
15:04:19 <trackbot> Zakim, list attendees
15:04:19 <Zakim> sorry, trackbot, I don't know what conference this is
15:04:27 <trackbot> RRSAgent, please draft minutes
15:04:27 <RRSAgent> I have made the request to generate http://www.w3.org/2013/09/19-eval-minutes.html trackbot
15:04:28 <trackbot> RRSAgent, bye
15:04:28 <RRSAgent> I see no action items