14:48:34 <RRSAgent> RRSAgent has joined #eval
14:48:34 <RRSAgent> logging to http://www.w3.org/2011/12/15-eval-irc
14:48:36 <trackbot> RRSAgent, make logs world
14:48:36 <Zakim> Zakim has joined #eval
14:48:38 <trackbot> Zakim, this will be 3825
14:48:38 <Zakim> ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 12 minutes
14:48:39 <trackbot> Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
14:48:39 <trackbot> Date: 15 December 2011
14:49:04 <shadi> chair: Eric
14:53:03 <shadi> agenda: http://lists.w3.org/Archives/Public/public-wai-evaltf/2011Dec/0035.html
14:53:07 <Detlev> Detlev has joined #eval
14:53:51 <shadi> regrets: Amy, Kostas, Denis, Liz
14:54:16 <shadi> agenda+ Section 4
14:54:33 <shadi> agenda+ Publication Planning
14:54:43 <shadi> agenda+ Availability for f2f
14:55:16 <Zakim> WAI_ERTWG(Eval TF)10:00AM has now started
14:55:23 <Zakim> +houtepen
14:55:36 <Zakim> +Eric
14:55:37 <Zakim> -houtepen
14:55:39 <Zakim> +houtepen
14:56:57 <Zakim> +Shadi
14:57:57 <Zakim> +[IPcaller]
14:58:15 <shadi> zakim, ipcaller is Kerstin
14:58:15 <Zakim> +Kerstin; got it
14:58:33 <kerstin> kerstin has joined #eval
14:58:49 <shadi> zakim, mute me
14:58:49 <Zakim> Shadi should now be muted
14:59:07 <kerstin> zakim, mute me
14:59:07 <Zakim> Kerstin should now be muted
14:59:17 <Zakim> +[IPcaller]
14:59:20 <Kathy> Kathy has joined #eval
14:59:26 <houtepen> zakim, mute me
14:59:26 <Zakim> houtepen should now be muted
14:59:32 <shadi> zakim, unmute me
14:59:33 <Zakim> Shadi should no longer be muted
14:59:40 <Mike_Elledge> Mike_Elledge has joined #eval
14:59:48 <shadi> zakim, ipcaller is Vivienne
14:59:49 <Zakim> +kathy
14:59:49 <Zakim> +Vivienne; got it
14:59:58 <SarahSwierenga> SarahSwierenga has joined #eval
15:00:24 <Zakim> + +49.404.318.aaaa
15:00:42 <vivienne> vivienne has joined #eval
15:00:50 <Zakim> +Sarah
15:00:59 <Detlev> Zakim, aaaa is me
15:00:59 <Zakim> +Detlev; got it
15:01:08 <vivienne> zakim, mute me
15:01:08 <Zakim> Vivienne should now be muted
15:01:22 <Zakim> +Mike
15:01:35 <Detlev> Zakim, mute me
15:01:36 <Zakim> Detlev should now be muted
15:02:07 <kerstin> no, i have to leave earlier
15:02:20 <Detlev> I can do it
15:02:21 <kerstin> sorry
15:02:39 <Detlev> fine
15:02:49 <shadi> scribe: Detlev
15:02:50 <Detlev> zakim, unmute me
15:02:52 <Zakim> Detlev should no longer be muted
15:03:06 <shadi> zakim, take up agendum 1
15:03:10 <Zakim> agendum 1. "Section 4" taken up [from shadi]
15:03:16 <Kathy> zakim, mute me
15:03:20 <Zakim> kathy should now be muted
15:03:35 <vivienne> zakim, unmute me
15:03:35 <Zakim> Vivienne should no longer be muted
15:03:51 <shadi> zakim, who is on the phone?
15:03:51 <Zakim> On the phone I see houtepen (muted), Eric, Shadi, Kerstin (muted), Vivienne, kathy (muted), Detlev, Sarah, Mike
15:03:59 <vivienne> zakim, mute me
15:03:59 <Zakim> Vivienne should now be muted
15:04:18 <shadi> zakim, mute me
15:04:18 <Zakim> Shadi should now be muted
15:04:45 <richard> richard has joined #eval
15:05:12 <ericvelleman> ericvelleman has joined #eval
15:05:19 <shadi> zakim, who is on the phone?
15:05:24 <Zakim> On the phone I see houtepen (muted), Eric, Shadi (muted), Kerstin (muted), Vivienne (muted), kathy (muted), Detlev, Sarah, Mike
15:06:08 <Detlev> EV mentions discussion on sampling, wanted to send newer version of methodology
15:06:35 <Zakim> +[IPcaller]
15:06:55 <shadi> zakim, ipcaller is Richard
15:06:55 <Zakim> +Richard; got it
15:07:41 <Detlev> EV: lots of discussions on scope, will be worked into draft
15:08:08 <Detlev> EV: sampling: 1 core resources, 2 Complete processes, 3 Random sampling
15:08:45 <Detlev> EV: Question: what are the minimum requirements for sampling?
15:08:51 <ericvelleman_> ericvelleman_ has joined #eval
15:08:55 <ericvelleman_> q?
15:09:18 <Kathy> q+
15:09:24 <Detlev> q+
15:09:27 <Kathy> zakim, unmute me
15:09:27 <Zakim> kathy should no longer be muted
15:09:28 <Mike_Elledge> looked good to me
15:09:36 <shadi> ack kathy
15:10:22 <Detlev> Kathy: liked task-oriented sampling approach, identify core tasks by differnet types of users. Random sampling also OK, mor as validation
15:10:48 <SarahSwierenga> q+
15:10:49 <vivienne> q+
15:11:16 <Detlev> EV: Barriers and amoount of pages in sample an issue
15:11:30 <ericvelleman_> q?
15:11:50 <Detlev> Kathy: Regarding processes: if you run automated scan, that indicates pages with more technical issues
15:11:56 <Kathy> zakim, mute me
15:11:56 <Zakim> kathy should now be muted
15:11:59 <shadi> ack det
15:13:36 <shadi> ack me
15:13:44 <Detlev> DF: raises issue of dropping issues in evaluation when they have been identified once, they may be played down too much
15:14:54 <shadi> zakim, mute me
15:14:54 <Zakim> Shadi should now be muted
15:15:05 <ericvelleman_> q?
15:15:10 <Mike_Elledge> q+
15:15:29 <shadi> ack sarah
15:15:44 <Detlev> Sarah: asks clients about the different templates used to assist in page sampling; start with core pages then extend
15:16:25 <Detlev> Sarah: concern about the random sample is the number of pages ending up in the sample
15:17:05 <Detlev> EV: agree to add templates as important element
15:17:16 <vivienne> ack me
15:18:21 <Detlev> Vivienne: Important to go through WCAG checkpoints to select pages to whcih as many checkpoints as possible apply
15:19:10 <Detlev> Vivienne: also uses random sample, but mainly uses combination of pages manually selected. Complete processes must not be missed
15:19:22 <vivienne> zakim, mute me
15:19:22 <Zakim> Vivienne should now be muted
15:21:10 <houtepen> q+
15:21:23 <Detlev> ME: Frequency of issue appearing - raises question of consistency. Templates help to check consistency of coding in website. Problem with limited sample is that developers may claim a structured approach but that may not be true and should be checked, ideally.
15:21:27 <ericvelleman_> q?
15:21:36 <houtepen> ack me
15:21:40 <shadi> ack mike
15:22:14 <Detlev> MH: In partial evaluations, can we use the same sampling rules?
15:22:32 <houtepen> zakim, mute me
15:22:32 <Zakim> houtepen should now be muted
15:22:42 <houtepen> zakim 
15:22:47 <Detlev> EV: Needs discussion. Focus is full evaluations, but there may be claims for partial conformance.
15:22:48 <houtepen> zakim, unmute me
15:22:48 <Zakim> houtepen should no longer be muted
15:23:34 <Detlev> MH: Maybe - full sample but parial conformance claim
15:23:44 <Detlev> EV: confirms focus on full evaluation
15:24:07 <houtepen> zakim, mute me
15:24:07 <Zakim> houtepen should now be muted
15:24:25 <ericvelleman_> q?
15:24:30 <vivienne> q+
15:24:36 <vivienne> ack me
15:24:40 <Detlev> EV: any comment on the issue of trust between tester and site owner?
15:24:48 <richard> q+
15:24:53 <SarahSwierenga> q+
15:25:49 <Detlev> Vivienne: Does not see this tension as a big problem - parties need to develop trust, tester wants to be helpful for project owner, gets more out of it if communication is open and trustful
15:26:23 <Detlev> EV: Input from developer regarding setting the page sample OK?
15:27:13 <vivienne> zakim, mute me
15:27:13 <Zakim> Vivienne should now be muted
15:27:16 <Detlev> Vivienne: owner should not dictate sample, but communicate to assist tester in sampling, explain scenarios of use, identify cirtical pages and processes
15:28:00 <Tim> Tim has joined #eval
15:28:34 <Detlev> RW: When identifying a barrier, evaluator cannot tell site owner exactly how to fix a barrier, the developer must address the issue - a number of occurences might not be needed
15:28:42 <shadi> ack richard
15:28:46 <shadi> ack sarah
15:29:28 <shadi> regrets+ Vincent
15:30:06 <Detlev> Sarah: You cannot dictate trust but llook at pages and paths to check whether design and a11y is used consistently, check several parts from top to bottom
15:30:52 <ericvelleman_> q?
15:31:21 <Detlev> Sarah: not so much complete process but user task-based approach, looking for paths and features that tell what WCAG items are relevant 
15:31:26 <Zakim> +Tim_Boland
15:31:41 <agarrison> agarrison has joined #eval
15:31:45 <Detlev> Sarah: Scenarios and tasks included
15:31:48 <ericvelleman_> q?
15:33:25 <Detlev> Detlev: difference between developer focus where issues are identified once, and user perspective where quantify of problems also counts
15:33:28 <Zakim> +[IPcaller]
15:33:30 <agarrison> zakim, ipcaller is agarrison
15:33:30 <Zakim> +agarrison; got it
15:33:56 <Detlev> EV: raises question of automatic, semi-auutom. and manual sampling
15:34:08 <vivienne> q+
15:34:18 <Detlev> EV: can you have a full evaluation with an automatic sample?
15:34:19 <Kathy> manual sample is important
15:34:32 <vivienne> zakim, unmute me
15:34:32 <Zakim> Vivienne should no longer be muted
15:34:35 <ericvelleman_> q?
15:34:42 <SarahSwierenga> Manual sample is very important! 
15:35:00 <Tim> automatic plus manual
15:35:26 <Detlev> Vivienne: Seems clear that relying on automated tools is not sufficient, issue is choice of methods to select pages that are then checked manually
15:35:35 <vivienne> zakim, mute me
15:35:36 <Zakim> Vivienne should now be muted
15:35:57 <vivienne> q+
15:36:03 <Detlev> EV: Raises issue of number of pages to sample
15:36:28 <vivienne> zakim, unmute me
15:36:28 <Zakim> Vivienne should no longer be muted
15:36:40 <shadi> ack vivienne
15:37:17 <vivienne> zakim, mute me
15:37:17 <Zakim> Vivienne should now be muted
15:37:18 <Detlev> Vivienne: Australian Government has decided for national transition strategy that 10% of the page total should be checked - on large sites this is probably not feasible...
15:37:20 <SarahSwierenga> q+
15:38:30 <shadi> ack sarah
15:38:34 <Detlev> Sarah: Above 20 or 25 pages will not make sense, takes a significant amount of time, below 20 should be OK - customers will not want larger samples, costs get prohibitive
15:38:44 <richard> q+
15:38:56 <vivienne> I agree with Sarah.  Even 20 pages would take a significant amount of time
15:39:48 <Detlev> Sarah: Discount usability procedure is an example of a weay of getting a pretty good idea of the state, there are diminishing returns with more users / pages
15:40:12 <Detlev> q+
15:41:29 <Detlev> RW: With a site like Amazon you got 1000s of pages but templates mean tzhey are mostly similar pages; on other more diverse sites you need a larger sample. A ruigit perecntage won't work
15:41:44 <shadi> ack richard 
15:42:38 <Kathy> q+
15:42:51 <ericvelleman_> q?
15:42:53 <SarahSwierenga> Responding to Richard - depends on what you call the 'site.' You seemed to be describing a portal-type design.
15:42:55 <shadi> ack Detlev 
15:43:01 <Kathy> zakim, unmute me
15:43:01 <Zakim> kathy should no longer be muted
15:43:07 <Detlev> DF: is it fair to selct pages with more issues?
15:43:12 <shadi> ack Kathy 
15:43:37 <agarrison> q+
15:43:55 <kerstin> sorry, I have to leave
15:44:10 <Detlev> Kathy: If client has tight budget, should areas pages identified that have key a11y issues
15:44:12 <Zakim> -Kerstin
15:44:14 <Tim> q+
15:44:50 <houtepen> q+
15:44:51 <Detlev> EV: will be difficult to make part of the methodology text...
15:45:13 <ericvelleman_> q?
15:45:19 <Kathy> zakim, mute me
15:45:19 <Zakim> kathy should now be muted
15:45:23 <Detlev> Kathy: Value of taking selectively pages
15:46:08 <Detlev> AG: Run a tool over pages to identify pages that where relevant for each checkpoint, makimng up the sample
15:46:13 <shadi> ack agarrison 
15:46:29 <Detlev> EV: Good approach, Alistair?
15:46:47 <vivienne> What kind of tool would you use to identify this Alistair?
15:46:47 <Detlev> AG: (inaudible)
15:47:10 <agarrison> What we used to do was run a tool over the site to identify pages which would be most relevant to each checkpoint - maybe identifying 5 - 10 per checkpoint. All identified pages went into the sample.
15:47:13 <ericvelleman_> q?
15:47:37 <shadi> ack tim
15:47:37 <agarrison> This was a good approach for WCAG 1.0 - as pages were in effect delivered to the evaluator - however, WCAG 2.0 is a bigger beast.
15:48:26 <houtepen> ack me
15:48:28 <Detlev> TB: Site analytics could suggest most visited pages to enter the sample. There is no one sampling method working across the board; therefore the evaluator should document the sampling method used
15:48:40 <shadi> q+
15:49:07 <shadi> q-
15:49:16 <Detlev> MH: Choose some pages with multimedia if there are many in the site
15:49:38 <Detlev> EV: should pages be added manually (q to Aliastair)
15:49:42 <houtepen> zakim, mute me
15:49:42 <Zakim> houtepen should now be muted
15:50:27 <Detlev> AG: good point to select pages that are frequently used - link that to pages to which many checkpoints apply
15:51:00 <shadi> ack me
15:51:02 <Detlev> AG: Helps finding the few pages that have special elements like Video
15:51:21 <shadi> zakim, mute me
15:51:21 <Zakim> Shadi should now be muted
15:52:01 <agarrison> http://code.google.com/p/googlesitemapgenerator/ have a look at this for selecting pages which are frequently viewed
15:52:05 <Detlev> EV: Proposes discussing number of pages in mailing list, continue discussion with AG about automatic selection of pages
15:52:06 <shadi> zakim, take up next
15:52:06 <Zakim> agendum 2. "Publication Planning" taken up [from shadi]
15:52:20 <shadi> ack me
15:52:28 <Detlev> EV: Is this version fit to be sent to WCAG WG as draft?
15:53:04 <Detlev> Shadi: no meetings next weeks, use draft and study at the fireplace ;-)
15:53:41 <Detlev> Shadi: wonders whether current draft will give sufficient impression of the shape of methodology - may be not yet...
15:54:06 <Detlev> Shadi: use time to make it ready for publication
15:54:06 <shadi> zakim, mute me
15:54:06 <Zakim> Shadi should now be muted
15:54:39 <shadi> zakim, take up next
15:54:39 <Zakim> agendum 3. "Availability for f2f" taken up [from shadi]
15:54:41 <vivienne> It might make a difference how much Christmas cheer we have consumed before that reading!
15:54:46 <shadi> ack me
15:54:50 <Detlev> EV: will work in comments and hopes to publish update hopefully before Xmas
15:55:15 <ericvelleman_> Happily we have Sinterklaas in the Netherlands, so we already had our presents
15:55:46 <Detlev> Shadi: Availability for face-to-face meetings (survey)  - some will be at CSUN, other opportunities in Europe
15:56:02 <agarrison> Should try England for meeting - seems to be in the atlantic between Europe and America ;-)
15:56:21 <Detlev> TPAC is a W3C meeting with other groups
15:56:48 <vivienne> Alistair, a little far from Western Australia.
15:56:57 <Detlev> Shadi: face-to-face good for detailed work, but expensive
15:57:29 <Detlev> Shadi: more ideas for meetings at conferences are welcome, but may be difficult to get to
15:58:03 <Detlev> EV: Shadi might send list of meetings where he will be
15:58:15 <Detlev> Shadi: will attend most of them (in the survey)
15:58:29 <shadi> zakim, mute me
15:58:29 <Zakim> Shadi should now be muted
15:58:43 <Detlev> EV: any other issues? No. Closes meeting.
15:58:59 <shadi> ack me
15:59:00 <vivienne> okay, bye for now.  A very blessed Chrismtas and New Year to you all.
15:59:06 <Kathy> Thanks!  Merry Christmas to you too... bye
15:59:12 <Mike_Elledge> Merry Xmas and Happy Holidays, all!
15:59:13 <Detlev> EV: Merry Xmas to all, read Methodology, meet again on 6. January
15:59:19 <houtepen> zakim, unmute me
15:59:19 <Zakim> houtepen should no longer be muted
15:59:20 <Zakim> -Mike
15:59:23 <Zakim> -Tim_Boland
15:59:24 <houtepen> houtepen has left #eval
15:59:24 <richard> Merry Xmas
15:59:25 <Zakim> -agarrison
15:59:26 <Zakim> -kathy
15:59:26 <Zakim> -Sarah
15:59:28 <Zakim> -Shadi
15:59:28 <vivienne> - vivienne
15:59:32 <vivienne> vivienne has left #eval
15:59:42 <Zakim> -houtepen
15:59:58 <Zakim> -Detlev
16:00:09 <Zakim> -Vivienne
16:00:28 <Zakim> -Eric
16:00:37 <Zakim> -Richard
16:00:39 <Zakim> WAI_ERTWG(Eval TF)10:00AM has ended
16:00:41 <Zakim> Attendees were houtepen, Eric, Shadi, Kerstin, kathy, Vivienne, +49.404.318.aaaa, Sarah, Detlev, Mike, Richard, Tim_Boland, agarrison
16:00:53 <shadi> trackbot, end meeting
16:00:54 <trackbot> Zakim, list attendees
16:00:56 <trackbot> RRSAgent, please draft minutes
16:00:56 <RRSAgent> I have made the request to generate http://www.w3.org/2011/12/15-eval-minutes.html trackbot
16:00:58 <trackbot> RRSAgent, bye
16:00:58 <RRSAgent> I see no action items