IRC log of eval on 2011-12-15

Timestamps are in UTC.

14:48:34 [RRSAgent]
RRSAgent has joined #eval
14:48:34 [RRSAgent]
logging to http://www.w3.org/2011/12/15-eval-irc
14:48:36 [trackbot]
RRSAgent, make logs world
14:48:36 [Zakim]
Zakim has joined #eval
14:48:38 [trackbot]
Zakim, this will be 3825
14:48:38 [Zakim]
ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 12 minutes
14:48:39 [trackbot]
Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
14:48:39 [trackbot]
Date: 15 December 2011
14:49:04 [shadi]
chair: Eric
14:53:03 [shadi]
agenda: http://lists.w3.org/Archives/Public/public-wai-evaltf/2011Dec/0035.html
14:53:07 [Detlev]
Detlev has joined #eval
14:53:51 [shadi]
regrets: Amy, Kostas, Denis, Liz
14:54:16 [shadi]
agenda+ Section 4
14:54:33 [shadi]
agenda+ Publication Planning
14:54:43 [shadi]
agenda+ Availability for f2f
14:55:16 [Zakim]
WAI_ERTWG(Eval TF)10:00AM has now started
14:55:23 [Zakim]
+houtepen
14:55:36 [Zakim]
+Eric
14:55:37 [Zakim]
-houtepen
14:55:39 [Zakim]
+houtepen
14:56:57 [Zakim]
+Shadi
14:57:57 [Zakim]
+[IPcaller]
14:58:15 [shadi]
zakim, ipcaller is Kerstin
14:58:15 [Zakim]
+Kerstin; got it
14:58:33 [kerstin]
kerstin has joined #eval
14:58:49 [shadi]
zakim, mute me
14:58:49 [Zakim]
Shadi should now be muted
14:59:07 [kerstin]
zakim, mute me
14:59:07 [Zakim]
Kerstin should now be muted
14:59:17 [Zakim]
+[IPcaller]
14:59:20 [Kathy]
Kathy has joined #eval
14:59:26 [houtepen]
zakim, mute me
14:59:26 [Zakim]
houtepen should now be muted
14:59:32 [shadi]
zakim, unmute me
14:59:33 [Zakim]
Shadi should no longer be muted
14:59:40 [Mike_Elledge]
Mike_Elledge has joined #eval
14:59:48 [shadi]
zakim, ipcaller is Vivienne
14:59:49 [Zakim]
+kathy
14:59:49 [Zakim]
+Vivienne; got it
14:59:58 [SarahSwierenga]
SarahSwierenga has joined #eval
15:00:24 [Zakim]
+ +49.404.318.aaaa
15:00:42 [vivienne]
vivienne has joined #eval
15:00:50 [Zakim]
+Sarah
15:00:59 [Detlev]
Zakim, aaaa is me
15:00:59 [Zakim]
+Detlev; got it
15:01:08 [vivienne]
zakim, mute me
15:01:08 [Zakim]
Vivienne should now be muted
15:01:22 [Zakim]
+Mike
15:01:35 [Detlev]
Zakim, mute me
15:01:36 [Zakim]
Detlev should now be muted
15:02:07 [kerstin]
no, i have to leave earlier
15:02:20 [Detlev]
I can do it
15:02:21 [kerstin]
sorry
15:02:39 [Detlev]
fine
15:02:49 [shadi]
scribe: Detlev
15:02:50 [Detlev]
zakim, unmute me
15:02:52 [Zakim]
Detlev should no longer be muted
15:03:06 [shadi]
zakim, take up agendum 1
15:03:10 [Zakim]
agendum 1. "Section 4" taken up [from shadi]
15:03:16 [Kathy]
zakim, mute me
15:03:20 [Zakim]
kathy should now be muted
15:03:35 [vivienne]
zakim, unmute me
15:03:35 [Zakim]
Vivienne should no longer be muted
15:03:51 [shadi]
zakim, who is on the phone?
15:03:51 [Zakim]
On the phone I see houtepen (muted), Eric, Shadi, Kerstin (muted), Vivienne, kathy (muted), Detlev, Sarah, Mike
15:03:59 [vivienne]
zakim, mute me
15:03:59 [Zakim]
Vivienne should now be muted
15:04:18 [shadi]
zakim, mute me
15:04:18 [Zakim]
Shadi should now be muted
15:04:45 [richard]
richard has joined #eval
15:05:12 [ericvelleman]
ericvelleman has joined #eval
15:05:19 [shadi]
zakim, who is on the phone?
15:05:24 [Zakim]
On the phone I see houtepen (muted), Eric, Shadi (muted), Kerstin (muted), Vivienne (muted), kathy (muted), Detlev, Sarah, Mike
15:06:08 [Detlev]
EV mentions discussion on sampling, wanted to send newer version of methodology
15:06:35 [Zakim]
+[IPcaller]
15:06:55 [shadi]
zakim, ipcaller is Richard
15:06:55 [Zakim]
+Richard; got it
15:07:41 [Detlev]
EV: lots of discussions on scope, will be worked into draft
15:08:08 [Detlev]
EV: sampling: 1 core resources, 2 Complete processes, 3 Random sampling
15:08:45 [Detlev]
EV: Question: what are the minimum requirements for sampling?
15:08:51 [ericvelleman_]
ericvelleman_ has joined #eval
15:08:55 [ericvelleman_]
q?
15:09:18 [Kathy]
q+
15:09:24 [Detlev]
q+
15:09:27 [Kathy]
zakim, unmute me
15:09:27 [Zakim]
kathy should no longer be muted
15:09:28 [Mike_Elledge]
looked good to me
15:09:36 [shadi]
ack kathy
15:10:22 [Detlev]
Kathy: liked task-oriented sampling approach, identify core tasks by differnet types of users. Random sampling also OK, mor as validation
15:10:48 [SarahSwierenga]
q+
15:10:49 [vivienne]
q+
15:11:16 [Detlev]
EV: Barriers and amoount of pages in sample an issue
15:11:30 [ericvelleman_]
q?
15:11:50 [Detlev]
Kathy: Regarding processes: if you run automated scan, that indicates pages with more technical issues
15:11:56 [Kathy]
zakim, mute me
15:11:56 [Zakim]
kathy should now be muted
15:11:59 [shadi]
ack det
15:13:36 [shadi]
ack me
15:13:44 [Detlev]
DF: raises issue of dropping issues in evaluation when they have been identified once, they may be played down too much
15:14:54 [shadi]
zakim, mute me
15:14:54 [Zakim]
Shadi should now be muted
15:15:05 [ericvelleman_]
q?
15:15:10 [Mike_Elledge]
q+
15:15:29 [shadi]
ack sarah
15:15:44 [Detlev]
Sarah: asks clients about the different templates used to assist in page sampling; start with core pages then extend
15:16:25 [Detlev]
Sarah: concern about the random sample is the number of pages ending up in the sample
15:17:05 [Detlev]
EV: agree to add templates as important element
15:17:16 [vivienne]
ack me
15:18:21 [Detlev]
Vivienne: Important to go through WCAG checkpoints to select pages to whcih as many checkpoints as possible apply
15:19:10 [Detlev]
Vivienne: also uses random sample, but mainly uses combination of pages manually selected. Complete processes must not be missed
15:19:22 [vivienne]
zakim, mute me
15:19:22 [Zakim]
Vivienne should now be muted
15:21:10 [houtepen]
q+
15:21:23 [Detlev]
ME: Frequency of issue appearing - raises question of consistency. Templates help to check consistency of coding in website. Problem with limited sample is that developers may claim a structured approach but that may not be true and should be checked, ideally.
15:21:27 [ericvelleman_]
q?
15:21:36 [houtepen]
ack me
15:21:40 [shadi]
ack mike
15:22:14 [Detlev]
MH: In partial evaluations, can we use the same sampling rules?
15:22:32 [houtepen]
zakim, mute me
15:22:32 [Zakim]
houtepen should now be muted
15:22:42 [houtepen]
zakim
15:22:47 [Detlev]
EV: Needs discussion. Focus is full evaluations, but there may be claims for partial conformance.
15:22:48 [houtepen]
zakim, unmute me
15:22:48 [Zakim]
houtepen should no longer be muted
15:23:34 [Detlev]
MH: Maybe - full sample but parial conformance claim
15:23:44 [Detlev]
EV: confirms focus on full evaluation
15:24:07 [houtepen]
zakim, mute me
15:24:07 [Zakim]
houtepen should now be muted
15:24:25 [ericvelleman_]
q?
15:24:30 [vivienne]
q+
15:24:36 [vivienne]
ack me
15:24:40 [Detlev]
EV: any comment on the issue of trust between tester and site owner?
15:24:48 [richard]
q+
15:24:53 [SarahSwierenga]
q+
15:25:49 [Detlev]
Vivienne: Does not see this tension as a big problem - parties need to develop trust, tester wants to be helpful for project owner, gets more out of it if communication is open and trustful
15:26:23 [Detlev]
EV: Input from developer regarding setting the page sample OK?
15:27:13 [vivienne]
zakim, mute me
15:27:13 [Zakim]
Vivienne should now be muted
15:27:16 [Detlev]
Vivienne: owner should not dictate sample, but communicate to assist tester in sampling, explain scenarios of use, identify cirtical pages and processes
15:28:00 [Tim]
Tim has joined #eval
15:28:34 [Detlev]
RW: When identifying a barrier, evaluator cannot tell site owner exactly how to fix a barrier, the developer must address the issue - a number of occurences might not be needed
15:28:42 [shadi]
ack richard
15:28:46 [shadi]
ack sarah
15:29:28 [shadi]
regrets+ Vincent
15:30:06 [Detlev]
Sarah: You cannot dictate trust but llook at pages and paths to check whether design and a11y is used consistently, check several parts from top to bottom
15:30:52 [ericvelleman_]
q?
15:31:21 [Detlev]
Sarah: not so much complete process but user task-based approach, looking for paths and features that tell what WCAG items are relevant
15:31:26 [Zakim]
+Tim_Boland
15:31:41 [agarrison]
agarrison has joined #eval
15:31:45 [Detlev]
Sarah: Scenarios and tasks included
15:31:48 [ericvelleman_]
q?
15:33:25 [Detlev]
Detlev: difference between developer focus where issues are identified once, and user perspective where quantify of problems also counts
15:33:28 [Zakim]
+[IPcaller]
15:33:30 [agarrison]
zakim, ipcaller is agarrison
15:33:30 [Zakim]
+agarrison; got it
15:33:56 [Detlev]
EV: raises question of automatic, semi-auutom. and manual sampling
15:34:08 [vivienne]
q+
15:34:18 [Detlev]
EV: can you have a full evaluation with an automatic sample?
15:34:19 [Kathy]
manual sample is important
15:34:32 [vivienne]
zakim, unmute me
15:34:32 [Zakim]
Vivienne should no longer be muted
15:34:35 [ericvelleman_]
q?
15:34:42 [SarahSwierenga]
Manual sample is very important!
15:35:00 [Tim]
automatic plus manual
15:35:26 [Detlev]
Vivienne: Seems clear that relying on automated tools is not sufficient, issue is choice of methods to select pages that are then checked manually
15:35:35 [vivienne]
zakim, mute me
15:35:36 [Zakim]
Vivienne should now be muted
15:35:57 [vivienne]
q+
15:36:03 [Detlev]
EV: Raises issue of number of pages to sample
15:36:28 [vivienne]
zakim, unmute me
15:36:28 [Zakim]
Vivienne should no longer be muted
15:36:40 [shadi]
ack vivienne
15:37:17 [vivienne]
zakim, mute me
15:37:17 [Zakim]
Vivienne should now be muted
15:37:18 [Detlev]
Vivienne: Australian Government has decided for national transition strategy that 10% of the page total should be checked - on large sites this is probably not feasible...
15:37:20 [SarahSwierenga]
q+
15:38:30 [shadi]
ack sarah
15:38:34 [Detlev]
Sarah: Above 20 or 25 pages will not make sense, takes a significant amount of time, below 20 should be OK - customers will not want larger samples, costs get prohibitive
15:38:44 [richard]
q+
15:38:56 [vivienne]
I agree with Sarah. Even 20 pages would take a significant amount of time
15:39:48 [Detlev]
Sarah: Discount usability procedure is an example of a weay of getting a pretty good idea of the state, there are diminishing returns with more users / pages
15:40:12 [Detlev]
q+
15:41:29 [Detlev]
RW: With a site like Amazon you got 1000s of pages but templates mean tzhey are mostly similar pages; on other more diverse sites you need a larger sample. A ruigit perecntage won't work
15:41:44 [shadi]
ack richard
15:42:38 [Kathy]
q+
15:42:51 [ericvelleman_]
q?
15:42:53 [SarahSwierenga]
Responding to Richard - depends on what you call the 'site.' You seemed to be describing a portal-type design.
15:42:55 [shadi]
ack Detlev
15:43:01 [Kathy]
zakim, unmute me
15:43:01 [Zakim]
kathy should no longer be muted
15:43:07 [Detlev]
DF: is it fair to selct pages with more issues?
15:43:12 [shadi]
ack Kathy
15:43:37 [agarrison]
q+
15:43:55 [kerstin]
sorry, I have to leave
15:44:10 [Detlev]
Kathy: If client has tight budget, should areas pages identified that have key a11y issues
15:44:12 [Zakim]
-Kerstin
15:44:14 [Tim]
q+
15:44:50 [houtepen]
q+
15:44:51 [Detlev]
EV: will be difficult to make part of the methodology text...
15:45:13 [ericvelleman_]
q?
15:45:19 [Kathy]
zakim, mute me
15:45:19 [Zakim]
kathy should now be muted
15:45:23 [Detlev]
Kathy: Value of taking selectively pages
15:46:08 [Detlev]
AG: Run a tool over pages to identify pages that where relevant for each checkpoint, makimng up the sample
15:46:13 [shadi]
ack agarrison
15:46:29 [Detlev]
EV: Good approach, Alistair?
15:46:47 [vivienne]
What kind of tool would you use to identify this Alistair?
15:46:47 [Detlev]
AG: (inaudible)
15:47:10 [agarrison]
What we used to do was run a tool over the site to identify pages which would be most relevant to each checkpoint - maybe identifying 5 - 10 per checkpoint. All identified pages went into the sample.
15:47:13 [ericvelleman_]
q?
15:47:37 [shadi]
ack tim
15:47:37 [agarrison]
This was a good approach for WCAG 1.0 - as pages were in effect delivered to the evaluator - however, WCAG 2.0 is a bigger beast.
15:48:26 [houtepen]
ack me
15:48:28 [Detlev]
TB: Site analytics could suggest most visited pages to enter the sample. There is no one sampling method working across the board; therefore the evaluator should document the sampling method used
15:48:40 [shadi]
q+
15:49:07 [shadi]
q-
15:49:16 [Detlev]
MH: Choose some pages with multimedia if there are many in the site
15:49:38 [Detlev]
EV: should pages be added manually (q to Aliastair)
15:49:42 [houtepen]
zakim, mute me
15:49:42 [Zakim]
houtepen should now be muted
15:50:27 [Detlev]
AG: good point to select pages that are frequently used - link that to pages to which many checkpoints apply
15:51:00 [shadi]
ack me
15:51:02 [Detlev]
AG: Helps finding the few pages that have special elements like Video
15:51:21 [shadi]
zakim, mute me
15:51:21 [Zakim]
Shadi should now be muted
15:52:01 [agarrison]
http://code.google.com/p/googlesitemapgenerator/ have a look at this for selecting pages which are frequently viewed
15:52:05 [Detlev]
EV: Proposes discussing number of pages in mailing list, continue discussion with AG about automatic selection of pages
15:52:06 [shadi]
zakim, take up next
15:52:06 [Zakim]
agendum 2. "Publication Planning" taken up [from shadi]
15:52:20 [shadi]
ack me
15:52:28 [Detlev]
EV: Is this version fit to be sent to WCAG WG as draft?
15:53:04 [Detlev]
Shadi: no meetings next weeks, use draft and study at the fireplace ;-)
15:53:41 [Detlev]
Shadi: wonders whether current draft will give sufficient impression of the shape of methodology - may be not yet...
15:54:06 [Detlev]
Shadi: use time to make it ready for publication
15:54:06 [shadi]
zakim, mute me
15:54:06 [Zakim]
Shadi should now be muted
15:54:39 [shadi]
zakim, take up next
15:54:39 [Zakim]
agendum 3. "Availability for f2f" taken up [from shadi]
15:54:41 [vivienne]
It might make a difference how much Christmas cheer we have consumed before that reading!
15:54:46 [shadi]
ack me
15:54:50 [Detlev]
EV: will work in comments and hopes to publish update hopefully before Xmas
15:55:15 [ericvelleman_]
Happily we have Sinterklaas in the Netherlands, so we already had our presents
15:55:46 [Detlev]
Shadi: Availability for face-to-face meetings (survey) - some will be at CSUN, other opportunities in Europe
15:56:02 [agarrison]
Should try England for meeting - seems to be in the atlantic between Europe and America ;-)
15:56:21 [Detlev]
TPAC is a W3C meeting with other groups
15:56:48 [vivienne]
Alistair, a little far from Western Australia.
15:56:57 [Detlev]
Shadi: face-to-face good for detailed work, but expensive
15:57:29 [Detlev]
Shadi: more ideas for meetings at conferences are welcome, but may be difficult to get to
15:58:03 [Detlev]
EV: Shadi might send list of meetings where he will be
15:58:15 [Detlev]
Shadi: will attend most of them (in the survey)
15:58:29 [shadi]
zakim, mute me
15:58:29 [Zakim]
Shadi should now be muted
15:58:43 [Detlev]
EV: any other issues? No. Closes meeting.
15:58:59 [shadi]
ack me
15:59:00 [vivienne]
okay, bye for now. A very blessed Chrismtas and New Year to you all.
15:59:06 [Kathy]
Thanks! Merry Christmas to you too... bye
15:59:12 [Mike_Elledge]
Merry Xmas and Happy Holidays, all!
15:59:13 [Detlev]
EV: Merry Xmas to all, read Methodology, meet again on 6. January
15:59:19 [houtepen]
zakim, unmute me
15:59:19 [Zakim]
houtepen should no longer be muted
15:59:20 [Zakim]
-Mike
15:59:23 [Zakim]
-Tim_Boland
15:59:24 [houtepen]
houtepen has left #eval
15:59:24 [richard]
Merry Xmas
15:59:25 [Zakim]
-agarrison
15:59:26 [Zakim]
-kathy
15:59:26 [Zakim]
-Sarah
15:59:28 [Zakim]
-Shadi
15:59:28 [vivienne]
- vivienne
15:59:32 [vivienne]
vivienne has left #eval
15:59:42 [Zakim]
-houtepen
15:59:58 [Zakim]
-Detlev
16:00:09 [Zakim]
-Vivienne
16:00:28 [Zakim]
-Eric
16:00:37 [Zakim]
-Richard
16:00:39 [Zakim]
WAI_ERTWG(Eval TF)10:00AM has ended
16:00:41 [Zakim]
Attendees were houtepen, Eric, Shadi, Kerstin, kathy, Vivienne, +49.404.318.aaaa, Sarah, Detlev, Mike, Richard, Tim_Boland, agarrison
16:00:53 [shadi]
trackbot, end meeting
16:00:54 [trackbot]
Zakim, list attendees
16:00:56 [trackbot]
RRSAgent, please draft minutes
16:00:56 [RRSAgent]
I have made the request to generate http://www.w3.org/2011/12/15-eval-minutes.html trackbot
16:00:58 [trackbot]
RRSAgent, bye
16:00:58 [RRSAgent]
I see no action items