IRC log of eval on 2012-02-23

Timestamps are in UTC.

14:46:16 [RRSAgent]
RRSAgent has joined #eval
14:46:16 [RRSAgent]
logging to
14:46:18 [trackbot]
RRSAgent, make logs world
14:46:18 [Zakim]
Zakim has joined #eval
14:46:20 [trackbot]
Zakim, this will be 3825
14:46:20 [Zakim]
ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 14 minutes
14:46:21 [trackbot]
Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
14:46:21 [trackbot]
Date: 23 February 2012
14:49:49 [Detlev]
Detlev has joined #eval
14:50:36 [MartijnHoutepen]
MartijnHoutepen has joined #eval
14:52:18 [Zakim]
WAI_ERTWG(Eval TF)10:00AM has now started
14:52:25 [Zakim]
14:53:04 [shadi]
regrets: Alistair
14:53:25 [Kathy]
Kathy has joined #eval
14:53:44 [vivienne]
vivienne has joined #eval
14:54:25 [Zakim]
14:54:31 [vivienne]
IPcaller is me
14:54:47 [vivienne]
zakim, mute me
14:54:47 [Zakim]
sorry, vivienne, I do not know which phone connection belongs to you
14:55:09 [vivienne]
+IPcaller is me
14:55:46 [vivienne]
zakim, ipcaller is me
14:55:46 [Zakim]
+vivienne; got it
14:55:52 [vivienne]
zakim, mute me
14:55:53 [Zakim]
vivienne should now be muted
14:55:59 [Zakim]
14:56:46 [shadi]
agenda+ Is there more than pass and fail?
14:56:50 [shadi]
agenda+ Three different samples is that possible?
14:56:59 [shadi]
agenda+ Evaluation clause 5
14:57:03 [Kathy]
zakim, mute me
14:57:03 [Zakim]
Kathy should now be muted
14:57:12 [Liz]
Liz has joined #eval
14:57:53 [Zakim]
14:58:32 [Zakim]
14:58:41 [Zakim]
14:59:04 [Kathy]
Hi Shadi
14:59:12 [shadi]
zakim, who is on the phone?
14:59:12 [Zakim]
On the phone I see MartijnHoutepen, vivienne (muted), Kathy (muted), Liz, Don, Shadi
14:59:58 [Zakim]
15:01:13 [Zakim]
15:01:51 [Detlev]
Zakim, mute me
15:02:02 [Zakim]
Detlev should now be muted
15:02:31 [vivienne]
I wont' be able to make it next week as I'll be out of town
15:02:41 [MartijnHoutepen]
zakim, mute me
15:02:41 [Zakim]
MartijnHoutepen should now be muted
15:02:43 [ericvelleman]
ericvelleman has joined #eval
15:02:54 [Elle]
Elle has joined #eval
15:02:56 [MartijnHoutepen]
I won't be there as well
15:03:02 [Richard]
Richard has joined #eval
15:03:04 [shadi]
chair: Eric
15:03:17 [MartijnHoutepen]
15:03:43 [shadi]
agenda+ Next Meeting
15:03:54 [shadi]
15:04:07 [Zakim]
15:04:17 [Detlev]
Shadi could you repeat
15:04:21 [shadi]
zakim, ipcaller is Richard
15:04:36 [Zakim]
+Richard; got it
15:04:40 [shadi]
scribe: Richard
15:04:42 [Richard]
15:04:44 [Zakim]
15:04:54 [shadi]
15:06:06 [Detlev]
Look at the last mail form Tim Borland: the latest W3C WAI ATAG2.0 draft success criteria satisfaction options for conformance are: yes, no, not applicable
15:06:09 [shadi]
zakim, take up agendum 1
15:06:14 [shadi]
15:06:34 [Zakim]
agendum 1. "Is there more than pass and fail?" taken up [from shadi]
15:06:37 [Zakim]
15:06:37 [Richard]
Eric: new version with change log
15:07:03 [shadi]
zakim, mute me
15:07:18 [shadi]
Topic: Changes to the Methodology draft
15:07:31 [Zakim]
15:07:39 [Zakim]
Shadi should now be muted
15:07:41 [Detlev]
15:07:48 [shadi]
zakim, who is on the phone
15:07:50 [shadi]
zakim, who is on the phone?
15:07:53 [Zakim]
15:07:59 [Richard]
Eric: Changes from Kerstin, added some things Richard reconnect
15:08:12 [Zakim]
15:08:14 [Zakim]
I don't understand 'who is on the phone', shadi
15:08:17 [Zakim]
On the phone I see MartijnHoutepen (muted), vivienne (muted), Kathy (muted), Liz, Shadi (muted), Detlev (muted), Elle, Eric_Velleman, Don, [IPcaller]
15:08:31 [shadi]
ack me
15:08:34 [Richard]
Richard -is on the phone he had to reconnest
15:08:46 [shadi]
zakim, ipcaller is Richard
15:08:55 [Zakim]
+Richard; got it
15:08:57 [shadi]
zakim, mute me
15:09:07 [Zakim]
Shadi should now be muted
15:09:16 [Richard]
Eric: Pass and fail discussion - do we want N/A as in WCAG 1
15:09:37 [shadi]
ack me
15:09:43 [Detlev]
Accordign to Tim, the latest W3C WAI ATAG2.0 draft success criteria satisfaction options for conformance are: yes, no, not applicable
15:09:50 [ericvelleman]
15:09:54 [Mike_Elledge]
Mike_Elledge has joined #eval
15:09:58 [Detlev]
ack me
15:10:50 [Detlev]
15:11:00 [Zakim]
15:11:01 [Richard]
Shadi: W3C though N/A is ambigous. Conetnet that does not need feartures conforms by virtue of not needing features
15:11:25 [vivienne]
15:12:04 [Richard]
Shadi: If people feel strongly we could define N/A if so. Would need to make N/A almost, but not exactly a Yes
15:12:16 [Mike_Elledge]
15:12:27 [shadi]
ack det
15:14:40 [Richard]
Detlev: Please see comment on irc earlier. I think N/A should be included because it is already acceptable to W3C and it makes sense to our end users
15:14:42 [Detlev]
Zakim, mute me
15:14:42 [Zakim]
Detlev should now be muted
15:15:13 [vivienne]
zakim, ack me
15:15:13 [Zakim]
unmuting vivienne
15:15:15 [Zakim]
I see Mike_Elledge on the speaker queue
15:15:32 [Richard]
Shadi: Wonders why W3C WCAG2 decided against N/A. We need to check
15:16:05 [shadi]
s/Wonders why/Reminds that
15:16:36 [Richard]
Vivien: Giving too many passes suggests they are nearer than they are, but N/A allows for them to better understand the real state of their website
15:16:54 [shadi]
15:16:59 [vivienne]
zakim, mute me
15:16:59 [Zakim]
vivienne should now be muted
15:17:20 [Richard]
Eric: Does Shadi know of any discussions?
15:17:36 [Elle]
I agree with Vivenne regarding N/A and false positive results - my apologies, all, but I've been called into another meeting - I will follow up and participate via email
15:17:42 [Zakim]
15:18:09 [Richard]
Shadi: I will look more. I don't think we should stall. I would say attempt is as a draft and ask for input
15:18:10 [shadi]
15:18:46 [Richard]
Mike:I think N/A very useful - but needs careful definition
15:19:23 [Richard]
Eric: I propose we discuss this over the coming week and see what we come up with
15:19:35 [shadi]
zakim, take up next
15:19:35 [Zakim]
I see a speaker queue remaining and respectfully decline to close this agendum, shadi
15:19:40 [shadi]
15:19:45 [shadi]
ack mike
15:19:48 [shadi]
zakim, take up next
15:19:48 [Zakim]
agendum 2. "Three different samples is that possible?" taken up [from shadi]
15:20:24 [Zakim]
15:20:44 [Detlev]
15:20:51 [Detlev]
ack me
15:20:51 [vivienne]
15:20:55 [shadi]
zakim, ipcaller is Kerstin
15:20:55 [Zakim]
+Kerstin; got it
15:21:00 [shadi]
15:21:01 [Richard]
Eric: Item 3 - 3 different samples. Some others think the three approach is a good idea. Does anyone else think it good or bad?
15:21:04 [Zakim]
15:21:07 [ericvelleman]
15:21:32 [Kerstin]
Kerstin has joined #eval
15:21:48 [Kerstin]
zakim, mute me
15:21:48 [Zakim]
Kerstin should now be muted
15:21:55 [Zakim]
15:22:04 [Kerstin]
sorry for delay
15:22:21 [Richard]
Detlev: Possible misunderstanding of diference between sampling a web page and statistical sampling of more general things such as populations. Web pages have a common structure and limited in basic technology.
15:22:35 [vivienne]
zakim, ack me
15:22:35 [Zakim]
unmuting vivienne
15:22:36 [Zakim]
I see shadi on the speaker queue
15:22:48 [Detlev]
Zakim, mute me
15:22:48 [Zakim]
Detlev should now be muted
15:23:55 [Richard]
Viviene: I agree with most of what Detlev says. We are auditing the pages - but we are using three ways =of chosing teh pages we test. It is all about the choosing. Random alone would be useless
15:24:21 [vivienne]
zakim, mute me
15:24:21 [Zakim]
vivienne should now be muted
15:24:21 [shadi]
ack me
15:24:23 [Richard]
Viviene: Possible misunderstanding
15:24:25 [ericvelleman]
15:24:43 [Richard]
Eric: We seem to agree that the three sample types are good
15:26:33 [ericvelleman]
15:26:49 [shadi]
zakim, mute me
15:26:49 [Zakim]
Shadi should now be muted
15:26:58 [Richard]
Shadi: I think it is a matter of explaining it better. Thinks that Mike thinks we suggest using random to fill up number - this is not the case so our text needs clarifying
15:28:00 [vivienne]
I think I prefer pages
15:28:03 [shadi]
15:28:08 [Richard]
Eric: Resources- Confusion between resources and pages. Sometimes we call a page a resource, other times we say a page contains resources. Any thoughts?
15:28:12 [Kathy]
15:28:31 [Detlev]
I#d use pages, states of pages, elements
15:28:32 [shadi]
ack me
15:28:39 [vivienne]
I think a page can include some resources
15:28:52 [shadi]
[[A structured sequence of resources (of any type), for example an RDF sequence]]
15:29:06 [Richard]
Eric. I can clarify when I write page or resource.
15:29:18 [ericvelleman]
15:29:33 [Kathy]
ack me
15:29:35 [Mike_Elledge]
15:29:45 [MartijnHoutepen]
+1 for pages
15:29:49 [Richard]
Shadi: Eiter we use the two words as interchangeable or we need to retink
15:29:52 [vivienne]
Can we also clarify whether we refer to "URL" or "URI"?
15:29:57 [Tim]
Tim has joined #eval
15:30:13 [Mike_Elledge]
+1 for page elements
15:31:07 [Richard]
Viviens: Pages are more understandable to our users. Eric has suggested "elements" as bits within a page . Prefer Pages and elements. If we use "resource" it needs careful definition
15:31:35 [Richard]
Eric: I will make some changes to see how it work. We can put resources back if we need to
15:31:54 [Detlev]
That was Kathy who said that, not "Viviens"
15:32:00 [Richard]
Shadi:We can put a small clarification of web-page
15:32:03 [Kathy]
zakim - mute me
15:32:07 [ericvelleman]
15:32:12 [shadi]
15:32:36 [shadi]
q- mike
15:32:41 [shadi]
zakim, take up next
15:32:41 [Zakim]
agendum 3. "Evaluation clause 5" taken up [from shadi]
15:32:45 [Richard]
Eric: Clause 5 - Evaluation - three levels
15:34:27 [Detlev]
15:34:37 [Richard]
Eric: Cut into 1) Order, 2) Criterea - I added stuff here with a seperate section about stop critera - this is for discussion.
15:35:21 [Richard]
Eric: we have to allow for when a series of pages present nothing new - ie we can go somehwre else
15:35:33 [Kathy]
15:36:15 [Richard]
Eric:What do you think of stop criterea? 4.3 ?
15:36:18 [Detlev]
ack me
15:36:20 [ericvelleman]
15:36:24 [Zakim]
15:38:03 [Richard]
Detlev: Not sure I understand what they are. From aconformance point of view just 1 fail might be OK - but it is important to check if the error is truly accross the whole site or just those in one section
15:38:10 [Kerstin]
15:38:47 [Detlev]
Zakim, mute me
15:38:47 [Zakim]
Detlev should now be muted
15:38:49 [Richard]
Detlev: This would make it esier for corrections if user knows which areas need attention and which do not
15:38:57 [vivienne]
15:39:36 [Kathy]
ack me
15:39:40 [Richard]
Eric: Compare Global and regional errors- perhaps part of this discussion
15:40:11 [ericvelleman]
15:41:33 [Richard]
Kathy: We mention point of severity. If an error prevent complete use then everything fails : after evaluating just one page by silverlight it was clear that the whole thing was not accessible - so it was pointless doing four more pages
15:41:52 [Detlev]
I'd be more comfortable having fewer pages checked fully...
15:42:19 [Richard]
Eric: We have to decide if it is global or regional.
15:42:38 [Kerstin]
zakim, unmute me
15:42:38 [Zakim]
Kerstin should no longer be muted
15:42:47 [Richard]
Kathy: If it becomes absolutely clear that it can't be evaluated then I stop
15:42:50 [Kathy]
zakim, mute me
15:42:55 [Zakim]
Kathy should now be muted
15:44:17 [Richard]
Kerstin: I don't like "scores". We can check individual items. When I have cheked 3 or 4 tables I make a list and give a note and tell client he needs to check all other instances.
15:44:44 [Kerstin]
zakim, mute me
15:44:44 [Zakim]
Kerstin should now be muted
15:45:01 [vivienne]
ack me
15:45:03 [Kerstin]
I don't believe in "scores" ;-)
15:45:10 [Richard]
Eric: It must be possible to say "this is always wrong". - but impact of some regional areas can be serious
15:45:48 [Kerstin]
15:46:57 [Detlev]
Re-checking is much quicker than the first check!
15:47:20 [Kerstin]
zakim, unmute me
15:47:20 [Zakim]
Kerstin should no longer be muted
15:47:21 [Richard]
Viviene: I just copy comment each time so it is not a lot of hard work. But I agree that we can tell user that many pages have same error and expect user to check all other occurancies. But if you use a score and want accurate statisticsthen you have to check and comment on every page
15:47:23 [vivienne]
zakim, mute me
15:47:23 [Zakim]
vivienne should now be muted
15:48:10 [Kerstin]
zakim, mute me
15:48:10 [Zakim]
Kerstin should now be muted
15:48:27 [Richard]
Kerstin: For me Pass/Fail is enough. But for objectivity perhaps we need more
15:48:40 [Detlev]
15:48:49 [Kathy]
15:48:49 [vivienne]
ack me
15:49:10 [Richard]
Eric: Many of you do evaluations for real. Some of you take all pages individually
15:49:36 [Kerstin]
don't know why I'm on the speaker queue, don't want to say something
15:49:48 [Detlev]
15:49:51 [vivienne]
zakim, mute me
15:49:51 [Zakim]
vivienne should now be muted
15:49:54 [Detlev]
15:50:01 [Kathy]
ack me
15:50:03 [Richard]
Viviene: Yes I score every page. I give score points to each error and can total at the end
15:50:19 [Richard]
Kathy: I also look at all teh check points
15:50:34 [Richard]
15:50:47 [vivienne]
I do the same as Kathy when I am working for clients
15:50:55 [Kathy]
zakim, mute me
15:50:55 [Zakim]
Kathy should now be muted
15:50:57 [Kerstin]
zakim, unmute me
15:50:57 [Zakim]
Kerstin should no longer be muted
15:51:10 [shadi]
q- ker
15:51:15 [shadi]
ack richard
15:52:28 [Detlev]
15:53:24 [Kathy]
15:53:31 [Kerstin]
zakim, mute me
15:53:31 [Zakim]
Kerstin should now be muted
15:53:42 [Detlev]
ack me
15:54:43 [vivienne]
15:55:35 [Richard]
Richard:I check is sequence. Run special checks over a range of pages; For example check that the keyboard works on a sequence of pages and enavle the completeion of tasks etc. Then do the sanme without CSS ro see structure etc andthe without ime=ges to seeif it works wu=itout images (there are sutable aklt tags
15:55:44 [shadi]
15:55:57 [Kerstin]
we must describe procedure for be sure that the test is reliable
15:56:28 [Richard]
Eric: Detlev is amore a statistical thing, what we and richard do is more generic.
15:56:44 [Richard]
Detlev: Either way we still fill up the list of scs
15:56:58 [Kerstin]
I'm on the generic side of the evaluators ;-)
15:57:05 [Kathy]
ack me
15:57:10 [Richard]
Detlev: Good idea to check each element , SC
15:57:20 [Detlev]
Zakim, mute me
15:57:20 [Zakim]
Detlev should now be muted
15:57:43 [Kerstin]
15:58:15 [Richard]
Kathy: How I di it depends upon teh type of site. For a task based site s=do as Richard by following tasks using different technologies, I also agree with Detlev taht we must have a checklist
15:58:21 [vivienne]
ack me
15:58:28 [Kathy]
zakim, mute me
15:58:28 [Zakim]
Kathy should now be muted
15:59:47 [vivienne]
zakim, mute me
15:59:54 [Zakim]
vivienne should now be muted
15:59:54 [Richard]
Viviene:Diferent types of page need different methods. You need to actially use the technoogy - such as screen reader or keyboard etc. You still follow structure to make sure you don't miss anything
16:00:14 [Richard]
Eric: We need to cover - Stop Criterea,
16:00:57 [vivienne]
Eric, are we able to share the draft with others to get their viewpoint?
16:01:05 [Richard]
IEric: Please walk throgh the changes I made and discuss also on scoring. It is crucial to get this clear - thanks
16:01:14 [MartijnHoutepen]
ack me
16:01:14 [Kerstin]
zakim, unmute me
16:01:20 [shadi]
ack me
16:01:50 [Zakim]
Kerstin should no longer be muted
16:02:00 [Kerstin]
q- ker
16:02:06 [Richard]
Shadi: Some of us a CSUN but you should do weell withiout us :)
16:02:21 [vivienne]
bye now
16:02:24 [vivienne]
16:02:26 [Richard]
16:02:43 [ericvelleman]
ericvelleman has left #eval
16:02:54 [Richard]
16:03:30 [Zakim]
16:03:36 [Zakim]
16:03:49 [Zakim]
16:03:52 [Zakim]
16:03:56 [Zakim]
16:03:58 [Zakim]
16:04:00 [Zakim]
16:04:07 [Zakim]
16:04:10 [Zakim]
16:04:14 [Zakim]
16:04:18 [Zakim]
16:09:09 [Zakim]
16:09:12 [Zakim]
WAI_ERTWG(Eval TF)10:00AM has ended
16:09:13 [Zakim]
Attendees were MartijnHoutepen, vivienne, Kathy, Liz, Don, Shadi, Detlev, Elle, Richard, Eric_Velleman, Mike, Kerstin, Tim_Boland
16:12:32 [shadi]
trackbot, end meeting
16:12:32 [trackbot]
Zakim, list attendees
16:12:33 [Zakim]
sorry, trackbot, I don't know what conference this is
16:12:40 [trackbot]
RRSAgent, please draft minutes
16:12:40 [RRSAgent]
I have made the request to generate trackbot
16:12:41 [trackbot]
RRSAgent, bye
16:12:41 [RRSAgent]
I see no action items