IRC log of eval on 2013-02-14

Timestamps are in UTC.

14:45:14 [RRSAgent]
RRSAgent has joined #eval
14:45:14 [RRSAgent]
logging to http://www.w3.org/2013/02/14-eval-irc
14:45:16 [trackbot]
RRSAgent, make logs world
14:45:16 [Zakim]
Zakim has joined #eval
14:45:18 [trackbot]
Zakim, this will be 3825
14:45:18 [Zakim]
ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 15 minutes
14:45:19 [trackbot]
Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
14:45:19 [trackbot]
Date: 14 February 2013
14:45:51 [shadi]
zakim, this is eval
14:45:51 [Zakim]
shadi, I see WAI_ERTWG(Eval TF)10:00AM in the schedule but not yet started. Perhaps you mean "this will be eval".
14:45:58 [MartijnHoutepen]
Zakim, what's on the agenda?
14:45:58 [Zakim]
I see nothing on the agenda
14:50:46 [Zakim]
WAI_ERTWG(Eval TF)10:00AM has now started
14:50:53 [Zakim]
+ +1.313.322.aaaa
14:51:51 [Mike_Elledge]
Mike_Elledge has joined #eval
14:56:11 [Zakim]
+[IPcaller]
14:56:16 [Liz]
Liz has joined #eval
14:56:19 [Vivienne]
zakim, IPcaller is me
14:56:19 [Zakim]
+Vivienne; got it
14:57:05 [Zakim]
+ +1.301.975.aabb
14:57:06 [Vivienne]
zakim, mute me
14:57:07 [Zakim]
Vivienne should now be muted
14:57:51 [Liz]
Zakim; aabb is Liz
14:58:09 [Zakim]
+Shadi
14:58:20 [Vivienne]
hi shadi
14:58:40 [Vivienne]
ack me
14:58:52 [Detlev]
Detlev has joined #eval
14:59:09 [Vivienne]
zakim, mute me
14:59:09 [Zakim]
Vivienne should now be muted
14:59:56 [Zakim]
+ +31.30.239.aacc
15:00:07 [MartijnHoutepen]
Zakim, aacc is me
15:00:08 [Zakim]
+MartijnHoutepen; got it
15:00:15 [MartijnHoutepen]
Zakim, mute me
15:00:15 [Zakim]
MartijnHoutepen should now be muted
15:00:34 [Zakim]
+ +49.404.318.aadd
15:00:54 [Detlev]
Zakim, aadd is Detlev
15:00:54 [Zakim]
+Detlev; got it
15:01:37 [Vivienne]
zakim, aabb is Liz
15:01:37 [Vivienne]
ack me
15:01:37 [Vivienne]
zakim, unmute me
15:01:37 [Zakim]
+Liz; got it
15:01:38 [Zakim]
Vivienne should no longer be muted
15:02:14 [Vivienne]
zakim, mute me
15:02:14 [Zakim]
Vivienne should now be muted
15:02:23 [shadi]
zakim, who is on the phone?
15:02:23 [Zakim]
On the phone I see +1.313.322.aaaa, Vivienne (muted), Liz, Shadi, MartijnHoutepen (muted), Detlev
15:02:33 [Sarah_Swierenga]
Sarah_Swierenga has joined #eval
15:02:55 [MartijnHoutepen]
ack me
15:03:04 [Zakim]
+ +1.517.432.aaee
15:03:07 [Zakim]
+ +1.510.334.aaff
15:03:09 [ericvelleman]
ericvelleman has joined #eval
15:03:10 [shadi]
zakim, aaaa is Mike
15:03:10 [Zakim]
+Mike; got it
15:03:25 [Kathy]
Kathy has joined #eval
15:03:30 [Vivienne]
zakim, unmute me
15:03:32 [Zakim]
Vivienne should no longer be muted
15:03:55 [Zakim]
+Katie_Haritos-Shea
15:04:16 [shadi]
zakim, who is on the phone?
15:04:16 [Zakim]
On the phone I see Mike, Vivienne, Liz, Shadi, MartijnHoutepen, Detlev, +1.517.432.aaee, +1.510.334.aaff, Katie_Haritos-Shea
15:04:23 [Zakim]
+ +31.30.695.aagg
15:04:47 [Zakim]
+ +1.978.443.aahh
15:04:54 [Kathy]
zakim, aahh is me
15:04:54 [Zakim]
+Kathy; got it
15:04:55 [shadi]
zakim, aaee is Sarah
15:04:56 [Zakim]
+Sarah; got it
15:05:02 [MartijnHoutepen]
Zakim, aagg is ericvelleman
15:05:02 [Zakim]
+ericvelleman; got it
15:05:10 [Detlev]
Zakim. mute me
15:05:14 [shadi]
zakim, aaff is Peter
15:05:14 [shadi]
zakim, who is on the phone?
15:05:14 [Zakim]
+Peter; got it
15:05:15 [Zakim]
On the phone I see Mike, Vivienne, Liz, Shadi, MartijnHoutepen, Detlev, Sarah, Peter, Katie_Haritos-Shea, ericvelleman, Kathy
15:05:19 [Detlev]
Zakim,mute me
15:05:19 [Zakim]
Detlev should now be muted
15:05:46 [MartijnHoutepen]
zakim, mute me
15:05:46 [Zakim]
MartijnHoutepen should now be muted
15:06:14 [shadi]
scribe: Sarah
15:06:22 [Kathy]
zakim, mute me
15:06:22 [Zakim]
Kathy should now be muted
15:07:14 [shadi]
agenda: http://lists.w3.org/Archives/Public/public-wai-evaltf/2013Feb/0016.html
15:08:33 [shadi]
https://www.w3.org/2002/09/wbs/48225/WCAG-EM20130208/results
15:08:41 [shadi]
https://www.w3.org/2002/09/wbs/35422/WCAGevalTaskForce/results
15:09:48 [shadi]
ack me
15:10:19 [MoeKraft]
MoeKraft has joined #eval
15:10:48 [Sarah_Swierenga]
shadi: first internal survey, we don't need to fill in both surveys
15:11:16 [Zakim]
+MoeKraft
15:11:55 [Sarah_Swierenga]
shadi: objection from Samuel - does not object to idea of adding a note; can be discussed after public review; editor note will be added to step 5.c
15:12:45 [Sarah_Swierenga]
shadi: text from current working draft will be used for the public comments draft
15:12:53 [shadi]
[Review Note: Eval TF is particularly looking for feedback on this section.]
15:13:02 [Sarah_Swierenga]
+1
15:13:08 [ericvelleman]
+1
15:13:13 [MartijnHoutepen]
+1
15:13:22 [Liz]
+1
15:13:26 [Vivienne]
_1
15:13:26 [MoeKraft]
+1
15:13:29 [Detlev]
plus one
15:13:32 [Kathy]
+1
15:13:43 [Vivienne]
+1
15:13:45 [Sarah_Swierenga]
katie: +1
15:13:46 [Mike_Elledge]
+1
15:14:15 [Sarah_Swierenga]
peter: +1
15:14:25 [shadi]
kathy: +1
15:14:47 [ericvelleman]
https://www.w3.org/2002/09/wbs/35422/WCAGevalTaskForce/results
15:15:08 [Sarah_Swierenga]
eric: most responses accept as is
15:15:24 [shadi]
[[I think there is still sometimes ambiguity in the term WCAG 2.0 Techniques, which sometimes is used to refer to any technique that meets a WCAG success criterion and sometimes to refer to the techniques published in the Techniques document. It is not necessary for this draft, but at some point please make a review and use consistent terminology to distinguish those two meanings. You may need to define those two terms in your document to distinguish them.]]
15:15:52 [Sarah_Swierenga]
eric: conformance for particular situations - not sure what we should do with this one; will be discussed in wcag group tonight
15:16:35 [Sarah_Swierenga]
shadi: lorraine asks for a review, so shadi and eric will look for obvious ambiguities and adjust wording for next draft
15:16:46 [shadi]
s/lorraine/loretta
15:16:56 [ericvelleman]
q?
15:16:57 [Sarah_Swierenga]
eric: proposed solution ok?
15:17:10 [Sarah_Swierenga]
+1
15:18:13 [shadi]
shadi: propose Eric and Shadi to take a quick pass at catching obvious ambiguity, then do a more thorough review of terminology before the next publication
15:18:16 [Sarah_Swierenga]
+1
15:18:21 [ericvelleman]
+1
15:18:22 [MartijnHoutepen]
+1
15:18:26 [Vivienne]
+1
15:18:28 [Detlev]
fine
15:18:29 [Mike_Elledge]
+1
15:18:38 [Liz]
+1
15:18:46 [MoeKraft]
+1
15:18:47 [Kathy]
+1
15:19:12 [ericvelleman]
q?
15:19:21 [korn]
korn has joined #eval
15:19:40 [Vivienne]
q+
15:19:53 [Vivienne]
ack me
15:20:00 [ericvelleman]
https://www.w3.org/2002/09/wbs/35422/WCAGevalTaskForce/results
15:20:09 [ericvelleman]
https://www.w3.org/2002/09/wbs/48225/WCAG-EM20130208/results
15:20:28 [Sarah_Swierenga]
vivienne: looking for samuel's actual comment
15:21:11 [Zakim]
+[Oracle]
15:21:22 [Zakim]
-Peter
15:21:25 [Kathy]
I am still working on an updated version
15:21:32 [korn]
Zakim, Oracle has Peter_Korn
15:21:32 [Zakim]
+Peter_Korn; got it
15:21:38 [Sarah_Swierenga]
eric: evaluation flow graphic
15:21:43 [Kathy]
q+
15:21:52 [ericvelleman]
q?
15:21:52 [Kathy]
ack me
15:22:48 [Sarah_Swierenga]
kathy: using waterfall diagram that Peter came up with, and taking colorful style from previous one. also, trying to show the interaction without having it be overwhelming
15:23:34 [Sarah_Swierenga]
shadi: we need to have final version by tuesday for approval process
15:24:10 [Sarah_Swierenga]
shadi: deadline for external comments is today
15:24:13 [Zakim]
-Katie_Haritos-Shea
15:24:30 [shadi]
[[Samuel's comments resent: http://lists.w3.org/Archives/Public/public-wai-evaltf/2013Feb/0022.html]]
15:24:31 [Sarah_Swierenga]
shadi: final version for public review is due tuesday
15:25:08 [Sarah_Swierenga]
kathy: will try to have revised diagram by saturday
15:26:09 [Detlev]
fine
15:26:12 [MartijnHoutepen]
+1
15:26:13 [Mike_Elledge]
+1
15:26:14 [Vivienne]
that sounds fine to me
15:26:15 [Sarah_Swierenga]
eric: proposal to use new diagram if everybody agrees. otherwise we will use the existing diagram if there are issues with the new one
15:26:16 [Sarah_Swierenga]
+1
15:26:18 [Kathy]
+1
15:26:21 [Liz]
+1
15:26:23 [Kathy]
zakim, mute me
15:26:23 [Zakim]
Kathy should now be muted
15:26:24 [MoeKraft]
+1
15:26:49 [ericvelleman]
+1
15:27:56 [ericvelleman]
q?
15:28:49 [Sarah_Swierenga]
shadi: please fill out the survey - need to have comments if there are concerns
15:30:57 [Sarah_Swierenga]
peter: still concerned about sampling template and not being able to test web apps in their entirety
15:31:39 [shadi]
q+
15:32:14 [Sarah_Swierenga]
peter: should there be something other than a total conformance claims for cases where you can't test everything, e.g. web applications and software? a conformance (or another word) claim (or another word), like 'to our knowledge'?
15:32:31 [shadi]
ack me
15:32:34 [ericvelleman]
q?
15:32:38 [Sarah_Swierenga]
peter: want this to be helpful for others who haven't worked on the website
15:33:18 [korn]
q+
15:33:33 [Zakim]
-Vivienne
15:33:42 [ericvelleman]
q?
15:33:47 [Sarah_Swierenga]
shadi: agrees that this a bigger issue than this methodology which is bound to WCAG - we are not chartered or scoped to address this issue. inherent aspect of wcag, that the methodology applies to the entire scope of the website
15:34:21 [Sarah_Swierenga]
shadi: we will not be able to address it in this draft, even though this is an important issue
15:34:30 [ericvelleman]
q?
15:34:53 [ericvelleman]
q+
15:35:11 [Sarah_Swierenga]
peter: want to avoid the language, 'alternative to a conformance claim'. this is not an alternative. really likes the phrase, 'evaluation findings.'
15:35:14 [MartijnHoutepen]
q- korn
15:36:25 [ericvelleman]
q-
15:36:32 [MartijnHoutepen]
q- eric
15:36:34 [Sarah_Swierenga]
peter: is not recommending that this issue hold up the public working draft
15:36:46 [Detlev]
q+
15:36:59 [Vivienne]
Sorry guys, I can't get back on the call
15:37:11 [ericvelleman]
q?
15:37:13 [Detlev]
ack me
15:37:21 [Sarah_Swierenga]
shadi: can you really do a conformance claim, maybe to the 'best of our knowledge' would be good to do in the next version of the working doc.
15:37:36 [shadi]
s/shadi/eric
15:37:41 [shadi]
zakim, mute me
15:37:41 [Zakim]
Shadi should now be muted
15:37:57 [shadi]
q+
15:38:33 [Vivienne]
zakim keeps tellingme my passcode is not valid!
15:38:45 [Sarah_Swierenga]
detlev: all or nothing evaluation methodology is difficult. evaluation findings is usually more the case. we don't place ourselves outside of the practical side of this
15:39:41 [shadi]
ack me
15:39:50 [Detlev]
zakim, mute me
15:39:50 [Zakim]
Detlev should now be muted
15:39:53 [Sarah_Swierenga]
detlev: using conformance claim, the risk is that people will downplay the risks, and impact the credibility of the claim
15:40:12 [korn]
q+
15:41:25 [Sarah_Swierenga]
shadi: two aspects - peter - you can't evaluate everything - scoping issue; detlev - tolerance aspect - websites will typically have some oversight or issue; maybe they need to be addressed slightly differently
15:42:49 [Sarah_Swierenga]
peter: developers probably can't make conformance claims. they have more knowledge, but all apps have bugs. wacg is structured around conformance claims, but nothing is ever perfect, so a conformance claim approach is problematic.
15:42:50 [Mike_Elledge]
q+
15:43:05 [korn]
q-
15:43:58 [Zakim]
+Katie_Haritos-Shea
15:43:58 [Sarah_Swierenga]
peter: this is not only scoping or complexity, but a mixture of both. could we put this as a action item - to explore 'evaluation finding' and challenges of making claims on dynamic, complex apps
15:44:34 [shadi]
ack me
15:44:42 [Detlev]
thumbs up for that issue!
15:44:47 [Sarah_Swierenga]
eric: will make this part of the "issue" list
15:45:36 [Detlev]
Zakim, mute me
15:45:36 [Zakim]
Detlev was already muted, Detlev
15:45:59 [korn]
Issue: Further explore "evaluation findings" as a concept, and language that describes the challenges of making a "conformance claim" for large sites, dynamic sites, web apps
15:45:59 [trackbot]
Created ISSUE-13 - Further explore "evaluation findings" as a concept, and language that describes the challenges of making a "conformance claim" for large sites, dynamic sites, web apps; please complete additional details at <http://www.w3.org/WAI/ER/2011/eval/track/issues/13/edit>.
15:46:33 [ericvelleman]
Thanks Peter
15:47:35 [Sarah_Swierenga]
mike: conformance claims seems to be more objective than they actually are; however, for all practical purposes, even if the findings are all transparent, someone is going to compare them. they will try to create a score or metric to compare the two tools. so, if we don't establish the metric for conformance claim, someone else will.
15:48:11 [ericvelleman]
q?
15:48:17 [MartijnHoutepen]
q- mik
15:48:20 [korn]
q+
15:49:20 [Sarah_Swierenga]
peter: we should also be aware of this this will be used. language is important and if we suggest that this is 'the number' or
15:49:57 [korn]
q-
15:49:58 [shadi]
q+
15:50:02 [Sarah_Swierenga]
peter: or 'yes, it passes' can be risky, since it usually isn't true
15:50:02 [shadi]
ack me
15:50:14 [Detlev]
q+
15:50:19 [korn]
q+
15:50:44 [Sarah_Swierenga]
shadi: i don't disagree with the discussion, but does this belong in wcag, or in this methodology? needs to be discussed in the wcat working group.
15:51:02 [Detlev]
ack me
15:51:47 [shadi]
q+
15:52:39 [Sarah_Swierenga]
detlev: the methodology should draw a clear line between the 'check' and what it really does, which is show where the problems in the site are. more realistic and more usable.
15:53:04 [Detlev]
Zakim, mute me
15:53:04 [Zakim]
Detlev should now be muted
15:53:32 [Sarah_Swierenga]
peter: we are all members of wcag or eval tf, so anything has to be approved by the wcag working group anyway
15:54:14 [shadi]
ack me
15:54:17 [korn]
q-
15:55:14 [Zakim]
-Katie_Haritos-Shea
15:55:29 [Sarah_Swierenga]
shadi: it is our responsibility to bring back findings to the working group. so far in our draft, we have that you do a 'conformance check with reasonable confidence.'
15:56:11 [Detlev]
I have been going on about that repeatedly but it did not find consinsus of course
15:56:39 [shadi]
sometimes consensus building takes its time, Detlev :)
15:56:48 [Detlev]
sure :-)
15:57:36 [Sarah_Swierenga]
eric: watch for the revised diagram and get comments back to the group quickly
16:00:15 [Zakim]
-Sarah
16:02:13 [Mike_Elledge]
Thanks, all. Happy Valentine's Day!
16:02:24 [Detlev]
bye
16:02:25 [MartijnHoutepen]
bye
16:02:27 [Liz]
Bye
16:02:28 [Zakim]
-[Oracle]
16:02:30 [Zakim]
-Mike
16:02:32 [Zakim]
-MoeKraft
16:02:32 [Zakim]
-Kathy
16:02:33 [Zakim]
-ericvelleman
16:02:35 [Zakim]
-MartijnHoutepen
16:02:35 [ericvelleman]
ericvelleman has left #eval
16:02:37 [MartijnHoutepen]
MartijnHoutepen has left #eval
16:02:37 [Zakim]
-Liz
16:02:38 [shadi]
trackbot, end meeting
16:02:38 [trackbot]
Zakim, list attendees
16:02:38 [Zakim]
As of this point the attendees have been +1.313.322.aaaa, Vivienne, +1.301.975.aabb, Shadi, +31.30.239.aacc, MartijnHoutepen, +49.404.318.aadd, Detlev, Liz, +1.517.432.aaee,
16:02:38 [Zakim]
... +1.510.334.aaff, Mike, Katie_Haritos-Shea, +31.30.695.aagg, +1.978.443.aahh, Kathy, Sarah, ericvelleman, Peter, MoeKraft, Peter_Korn
16:02:43 [Zakim]
-Detlev
16:02:46 [trackbot]
RRSAgent, please draft minutes
16:02:46 [RRSAgent]
I have made the request to generate http://www.w3.org/2013/02/14-eval-minutes.html trackbot