IRC log of eval on 2012-01-19

Timestamps are in UTC.

14:49:59 [RRSAgent]
RRSAgent has joined #eval
14:49:59 [RRSAgent]
logging to http://www.w3.org/2012/01/19-eval-irc
14:50:01 [trackbot]
RRSAgent, make logs world
14:50:01 [Zakim]
Zakim has joined #eval
14:50:03 [trackbot]
Zakim, this will be 3825
14:50:03 [Zakim]
ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 10 minutes
14:50:04 [trackbot]
Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
14:50:04 [trackbot]
Date: 19 January 2012
14:55:45 [shadi]
zakim, call shadi-617
14:55:45 [Zakim]
ok, shadi; the call is being made
14:55:47 [Zakim]
WAI_ERTWG(Eval TF)10:00AM has now started
14:55:48 [Zakim]
+Shadi
14:55:51 [shadi]
chair: Eric
14:56:17 [Liz]
Liz has joined #eval
14:56:24 [shadi]
regrets: Samuel, Emmanuelle
14:57:03 [Zakim]
+Liz
14:57:04 [shadi]
regrets: Samuel, Emmanuelle, Martijn
14:57:46 [Elle]
Elle has joined #eval
14:58:17 [Detlev]
Detlev has joined #eval
14:58:21 [vivienne]
vivienne has joined #eval
14:59:38 [Kathy]
Kathy has joined #eval
14:59:56 [ericvelleman]
ericvelleman has joined #eval
14:59:59 [Zakim]
+Kathy
15:00:26 [Zakim]
+ +1.415.692.aaaa
15:00:29 [Zakim]
+Detlev
15:00:35 [Detlev]
zakim, aaaa is me
15:00:35 [Zakim]
+Detlev; got it
15:00:39 [shadi]
http://www.w3.org/WAI/ER/2011/eval/minutes.html
15:01:22 [Zakim]
+??P21
15:01:32 [vivienne]
zakim, +??P21 is me
15:01:33 [Zakim]
sorry, vivienne, I do not recognize a party named '+??P21'
15:01:38 [Zakim]
+ericvelleman
15:02:06 [vivienne]
zakim, mute me
15:02:06 [Zakim]
sorry, vivienne, I do not know which phone connection belongs to you
15:02:46 [vivienne]
zakim, ??P21 is me
15:02:46 [Zakim]
+vivienne; got it
15:02:54 [vivienne]
zakim, mute me
15:02:54 [Zakim]
vivienne should now be muted
15:03:00 [Detlev]
zakim, mute me
15:03:00 [Zakim]
Detlev should now be muted
15:03:05 [shadi]
zakizakim, who is on the phone?
15:03:14 [shadi]
zakim, who is on the phone?
15:03:14 [Zakim]
On the phone I see Shadi, Liz, Kathy, Detlev.a, Detlev (muted), vivienne (muted), ericvelleman
15:03:35 [Detlev]
zakim, unmute me
15:03:45 [Zakim]
Detlev should no longer be muted
15:04:03 [shadi]
zakim, detlev.a is really Elle
15:04:11 [Detlev]
zakim, mute me
15:04:13 [Zakim]
+??P7
15:04:19 [Zakim]
+Elle; got it
15:04:27 [Zakim]
Detlev should now be muted
15:04:30 [shadi]
zakim, ??p7 is Kerstin
15:04:41 [kerstin]
kerstin has joined #eval
15:04:49 [Zakim]
+Kerstin; got it
15:04:55 [Zakim]
+Sarah
15:05:24 [Zakim]
+ +1.248.342.aabb
15:05:42 [shadi]
zakim, aabb is Mike
15:05:42 [Zakim]
+Mike; got it
15:06:24 [SarahSwierenga]
SarahSwierenga has joined #eval
15:06:26 [shadi]
scribe: kerstin
15:06:56 [Kathy]
zakim, mute me
15:06:56 [Zakim]
Kathy should now be muted
15:07:11 [shadi]
zakim, mute me
15:07:12 [Zakim]
Shadi should now be muted
15:07:19 [kerstin]
zakim, mute me
15:07:19 [Zakim]
Kerstin should now be muted
15:07:28 [shadi]
agenda+ Specific discussion on section 5.4 Barrier Recognition
15:07:34 [agarrison]
agarrison has joined #eval
15:07:40 [shadi]
agenda+ Specific discussion on section 5.5 Error Margin

15:07:54 [shadi]
agenda+ Starting the evaluation section. Kick-off of the discussion.
15:08:01 [shadi]
zakim, take up agendum 1
15:08:01 [Zakim]
agendum 1. "Specific discussion on section 5.4 Barrier Recognition" taken up [from shadi]
15:08:13 [kerstin]
Eric resumes the discussion so far in the last weeks about 5.4
15:08:34 [ericvelleman]
http://www.w3.org/WAI/ER/conformance/ED-methodology-20120119.html
15:09:06 [shadi]
[[ reminder that the latest Editor Draft is always available from http://www.w3.org/WAI/ER/methodology/ ]]
15:09:13 [shadi]
ack me
15:09:28 [kerstin]
Eric will send a mail with a list of changes later
15:09:51 [Zakim]
+??P39
15:09:56 [shadi]
zakim, mute me
15:09:56 [Zakim]
Shadi should now be muted
15:10:06 [shadi]
zakim, ??p39 is agarrison
15:10:06 [Zakim]
+agarrison; got it
15:10:48 [lwatson]
lwatson has joined #eval
15:11:21 [shadi]
http://www.w3.org/WAI/ER/conformance/ED-methodology-20120119#barrierrecognition
15:11:52 [kerstin]
Eric: not a lot of discussion about barrier recognition, more questions
15:12:37 [Detlev]
q+
15:12:43 [Detlev]
zakim, unmute me
15:12:43 [Zakim]
Detlev should no longer be muted
15:12:47 [shadi]
ack d
15:13:07 [Zakim]
+??P42
15:13:20 [agarrison]
q+
15:13:42 [kerstin]
Detlev: many of the barriers might not be obvious, so it's necessary to evaluate the whole page to be sure
15:14:18 [kerstin]
Detlev: looking for differences is important, not only looking for the same
15:14:21 [lwatson]
zakim, ??p42 is lwatson
15:14:22 [Zakim]
+lwatson; got it
15:14:32 [kerstin]
Detlev, this was very fast
15:14:32 [Detlev]
zakim, mute me
15:14:32 [Zakim]
Detlev should now be muted
15:14:34 [ericvelleman]
q?
15:14:54 [Detlev]
agree
15:15:19 [shadi]
q+
15:15:24 [shadi]
ack ag
15:15:40 [kerstin]
agarrison: barrier recognition needs better definition
15:16:11 [Detlev]
q+
15:16:44 [ericvelleman]
q?
15:16:47 [shadi]
ack me
15:16:52 [kerstin]
eric: will try to describe barrier recognition in a better way
15:17:15 [agarrison]
I would actually remove the term barrier recognition
15:17:23 [Detlev]
ack me
15:17:23 [kerstin]
Shadi: Difference between stop condition and barrier recognition
15:17:23 [shadi]
zakim, mute me
15:17:26 [Zakim]
Shadi should now be muted
15:17:27 [Kathy]
q+
15:17:28 [ericvelleman]
q?
15:18:23 [kerstin]
Detlev: I meant probably something else, distinction between severe problems, systematic erros and not severe problems
15:18:49 [kerstin]
Detlev: question which errors could be tolerated?
15:19:07 [agarrison]
Surely this is the difference between A, AA and AAA
15:19:17 [kerstin]
Eric: change of terminology for 5.5
15:19:24 [Kathy]
ack me
15:19:32 [ericvelleman]
q?
15:19:37 [Detlev]
zakim, mute me
15:19:37 [Zakim]
Detlev should now be muted
15:19:44 [shadi]
[[ wonder if "incidental" is better than "accidental" ]]
15:20:25 [kerstin]
Kathy: Discussing error margin, no good definition until now
15:21:29 [kerstin]
Eric: issue will come back in the reporting
15:21:42 [Detlev]
My understanding is that you pick a sample and run every page against *all* SC
15:21:52 [ericvelleman]
q?
15:21:53 [Kathy]
mute me
15:22:20 [shadi]
zakim, mute kathy
15:22:20 [Zakim]
Kathy should now be muted
15:22:29 [kerstin]
Eric: first setup for when to stop evaluation this week
15:23:16 [kerstin]
Who is speaking? for scribing
15:23:20 [shadi]
[[ suggestions from Mike "types of errors" or "error reporting" ]]
15:23:45 [kerstin]
Mike: documenting different type of errors
15:23:59 [shadi]
s/Who is speaking? for scribing/
15:24:25 [shadi]
zakim, take up next
15:24:25 [Zakim]
agendum 2. "Specific discussion on section 5.5 Error Margin
" taken up [from shadi]
15:24:28 [vivienne]
q?
15:24:38 [vivienne]
q?
15:24:40 [shadi]
http://www.w3.org/WAI/ER/conformance/ED-methodology-20120119#errormargin
15:24:48 [kerstin]
Eric: Do we accept error margin? example: one missing alt or Do we don't accept?
15:24:54 [vivienne]
ack me
15:25:04 [vivienne]
q+
15:25:11 [ericvelleman]
q?
15:25:19 [shadi]
ack v
15:26:30 [Zakim]
+Mike.a
15:26:32 [Tim]
Tim has joined #eval
15:26:47 [Kathy]
q+
15:26:48 [Zakim]
-Mike
15:26:56 [kerstin]
Vivienne: testing against SCs, give the client possibility to fix them, first evaluation a lot pages will fail, after fixing there is a chance for conformance
15:27:01 [Detlev]
q+
15:27:11 [ericvelleman]
q?
15:27:16 [Kathy]
ack me
15:27:21 [vivienne]
zakim, mute me
15:27:21 [Zakim]
vivienne should now be muted
15:27:25 [kerstin]
Vivienne: if errors will not be fixed -> o compliance
15:28:02 [SarahSwierenga]
q+
15:28:05 [agarrison]
q+
15:28:30 [Elle]
I agree with Vivienne, if errors are identified, they must be accounted in the overall conformance (or lack thereof)
15:28:38 [kerstin]
Kathy: we can use a lot of tools, check a lot of website, it wll never be complete. Suggestion: Conformance Claim just for pages we had checked
15:28:40 [Zakim]
+Tim_Boland
15:29:01 [lwatson]
+1 to Kathy
15:29:07 [Detlev]
ack me
15:29:10 [Kathy]
mute me
15:29:10 [ericvelleman]
q?
15:30:18 [kerstin]
Detlev: one way dealing with might be seperating strict conformance and graded accessibility results
15:30:54 [kerstin]
Detlev: full conformance will never be happened, need a realistic approach
15:31:13 [Detlev]
mute me
15:31:16 [kerstin]
Sarah: agree with Kathy
15:31:25 [shadi]
zakim, mute detlev
15:31:25 [Zakim]
Detlev should now be muted
15:31:35 [lwatson]
q+
15:31:40 [shadi]
ack sar
15:32:33 [kerstin]
Sarah: suspects that people will claim conformance, and the public will point out problems, purpose of methodology is to identify what is needed for conformance
15:32:44 [ericvelleman]
q?
15:32:44 [Detlev]
agree
15:33:20 [shadi]
ack ag
15:33:46 [kerstin]
sorry, I don't understand alistair
15:34:40 [kerstin]
agarrison: importance of same results
15:35:06 [kerstin]
agarrison: comparison is important
15:35:25 [Detlev]
q+
15:36:08 [kerstin]
Eric: in the netherlands we check minimum 20 pages, error margin is 5 %
15:36:54 [Kathy]
q+
15:36:56 [kerstin]
Eric: to get the logo 10% error margin is allowed, but not for severe barriers, but not always measering in percentages
15:37:00 [shadi]
ack lw
15:37:54 [Zakim]
-Elle
15:38:27 [kerstin]
Leonie: realistic amount of pages in sample is important
15:38:54 [Detlev]
ack me
15:38:55 [kerstin]
Leonie: acceptable is when out of 20 pages 2 fail
15:39:01 [lwatson]
ack me
15:39:13 [kerstin]
Detlev: error margin is important
15:39:15 [shadi]
regrets+ Vincent
15:39:43 [Mike_Elledge]
Mike_Elledge has joined #eval
15:39:52 [kerstin]
Detlev: critical barriers like keybaord traps should not pass
15:40:22 [kerstin]
Detlev: what are acceptable errors
15:41:27 [agarrison]
Who decides what are acceptable errors - would this lead to a replicable evaluation?
15:41:59 [kerstin]
Detlev: in our appproach we have 95%, then is full conformance. more realistic are grades
15:42:02 [Kathy]
ack me
15:42:08 [ericvelleman]
q?
15:43:02 [kerstin]
Kathy: minor things should pass. How is error margin calculated in the NL
15:43:14 [kerstin]
Eric: error margin depends on the elements
15:43:39 [agarrison]
q+
15:43:48 [kerstin]
Eric: if one of 10 has a problem (eg. videos) it will pass
15:44:02 [Elle]
apologies, the call signal was bad and I'm only able to stay within IRC (thanks for the scribing!)
15:44:10 [kerstin]
Kathy: we also have to talk about frequency
15:44:15 [Kathy]
mute me
15:44:23 [Kathy]
zakim, mute me
15:44:23 [Zakim]
Kathy should now be muted
15:44:48 [vivienne]
sounds good to me
15:44:49 [Detlev]
not quite
15:45:05 [Detlev]
ack me
15:45:08 [kerstin]
Eric: Suggestion: there could be minor errors and if so, then we don't give a full conformance
15:45:10 [ericvelleman]
q?
15:46:53 [kerstin]
Detlev: if there is no error margin possible its from a pratical view not usefull
15:47:25 [lwatson]
q+
15:47:32 [shadi]
ack ag
15:47:44 [Mike_Elledge]
+1
15:48:19 [kerstin]
agarrison: points out the replicability of the test
15:49:30 [kerstin]
agarrison: difficult to say, that if just 2 pages it pass
15:49:53 [kerstin]
agarrison: when tested 20 pages, it's important to go further
15:50:10 [shadi]
ack lw
15:50:47 [kerstin]
sorry, haven't understood
15:50:48 [ericvelleman]
q?
15:51:02 [shadi]
q+ to propose dual approach
15:51:39 [Detlev]
q+
15:52:05 [shadi]
ack me
15:52:06 [Zakim]
shadi, you wanted to propose dual approach
15:52:07 [kerstin]
Leonie and agarrison: discussing the issue that probably interpretations of SCs are not the same. which will be a problem concerning replicacability
15:52:12 [lwatson]
ack me
15:52:18 [kerstin]
Hope this scribe was correctly
15:52:22 [ericvelleman]
q?
15:54:46 [kerstin]
Shadi: suggestion: you only claim conform if you conform, possibility. aggregation and documenting what passes and what fails, documenting as process (percentages) for motivating
15:54:52 [shadi]
zakim, mute me
15:54:52 [Zakim]
Shadi should now be muted
15:55:21 [shadi]
ack me
15:56:43 [kerstin]
Shadi: aggregation way could mean different levels, until now not found a really good aggregation method
15:56:43 [Detlev]
q-
15:57:07 [kerstin]
Shadi: full conformance as primary measure, aggregation as addo
15:57:13 [kerstin]
addon
15:57:15 [shadi]
zakim, mute me
15:57:15 [Zakim]
Shadi should now be muted
15:57:50 [kerstin]
+1
15:58:46 [kerstin]
Eric: Starting soon with evaluation, important discussions about reliability, validity - in the next call
15:58:54 [vivienne]
ack me
15:58:56 [Detlev]
bye
15:58:56 [Kathy]
thanks, bye
15:59:02 [kerstin]
Eric: closes the call
15:59:03 [vivienne]
vivienne has left #eval
15:59:03 [shadi]
ack me
15:59:08 [Zakim]
-Tim_Boland
15:59:10 [Zakim]
-lwatson
15:59:13 [kerstin]
kerstin, unmute me
15:59:18 [Zakim]
-Sarah
15:59:28 [Zakim]
-agarrison
15:59:32 [Zakim]
-Detlev
15:59:36 [Zakim]
-Kathy
15:59:42 [Zakim]
-Mike.a
15:59:50 [Zakim]
-Liz
15:59:52 [Zakim]
-ericvelleman
15:59:54 [Zakim]
-vivienne
15:59:57 [kerstin]
Finish? or do I have to do something?
16:00:06 [Zakim]
-Shadi
16:00:27 [kerstin]
oki
16:00:31 [kerstin]
bye
16:01:19 [shadi]
zakim, who is on the phone?
16:01:19 [Zakim]
On the phone I see Kerstin
16:01:26 [shadi]
zakim, drop kerstin
16:01:26 [Zakim]
Kerstin is being disconnected
16:01:27 [Zakim]
WAI_ERTWG(Eval TF)10:00AM has ended
16:01:28 [Zakim]
Attendees were Shadi, Liz, Kathy, +1.415.692.aaaa, Detlev, ericvelleman, vivienne, Elle, Kerstin, Sarah, +1.248.342.aabb, Mike, agarrison, lwatson, Tim_Boland
16:01:33 [shadi]
trackbot, end meeting
16:01:33 [trackbot]
Zakim, list attendees
16:01:33 [Zakim]
sorry, trackbot, I don't know what conference this is
16:01:36 [trackbot]
RRSAgent, please draft minutes
16:01:36 [RRSAgent]
I have made the request to generate http://www.w3.org/2012/01/19-eval-minutes.html trackbot
16:01:37 [trackbot]
RRSAgent, bye
16:01:37 [RRSAgent]
I see no action items