14:57:04 <RRSAgent> RRSAgent has joined #eval
14:57:04 <RRSAgent> logging to http://www.w3.org/2013/03/07-eval-irc
14:57:06 <trackbot> RRSAgent, make logs world
14:57:06 <Zakim> Zakim has joined #eval
14:57:08 <trackbot> Zakim, this will be 3825
14:57:09 <trackbot> Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
14:57:09 <trackbot> Date: 07 March 2013
14:57:10 <Zakim> ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 3 minutes
14:57:17 <ericvelleman> ericvelleman has joined #eval
15:01:01 <Sarah_Swierenga> Sarah_Swierenga has joined #eval
15:01:58 <shadi> zakim, this is eval
15:01:58 <Zakim> ok, shadi; that matches WAI_ERTWG(Eval TF)10:00AM
15:02:00 <shadi> zakim, who is on the phone?
15:02:00 <Zakim> On the phone I see ??P0, +1.510.334.aaaa, +1.517.432.aabb
15:02:15 <Zakim> +Shadi
15:02:28 <Vivienne> zakim, P0 is me
15:02:28 <Zakim> sorry, Vivienne, I do not recognize a party named 'P0'
15:02:34 <Zakim> + +31.30.239.aacc
15:02:36 <Vivienne> zakim, ??P0 is me
15:02:36 <Zakim> +Vivienne; got it
15:02:48 <MartijnHoutepen> Zakim, aacc is me
15:02:48 <Zakim> +MartijnHoutepen; got it
15:03:19 <Sarah_Swierenga> aabb is me
15:03:43 <Sarah_Swierenga> zakim, aabb is me
15:03:43 <Zakim> +Sarah_Swierenga; got it
15:04:01 <shadi> zakim, this is eval
15:04:01 <Zakim> shadi, this was already WAI_ERTWG(Eval TF)10:00AM
15:04:02 <Zakim> ok, shadi; that matches WAI_ERTWG(Eval TF)10:00AM
15:04:04 <Zakim> + +31.30.239.aadd
15:04:05 <shadi> zakim, who is on the phone?
15:04:05 <Zakim> On the phone I see Vivienne, +1.510.334.aaaa, Sarah_Swierenga, Shadi, MartijnHoutepen, +31.30.239.aadd
15:04:23 <ericvelleman> Zakim, aadd is me
15:04:23 <Zakim> +ericvelleman; got it
15:04:26 <shadi> zakim, aaaa is Peter
15:04:27 <Zakim> +Peter; got it
15:07:16 <MartijnHoutepen> zakim, mute me
15:07:16 <Zakim> MartijnHoutepen should now be muted
15:07:23 <shadi> regrets: Kathy, Tim, Liz, Richard, Kostas
15:07:27 <shadi> scribe: shadi
15:07:34 <shadi> zakim, mute me
15:07:34 <Zakim> Shadi should now be muted
15:08:45 <shadi> EV: updated Overview document by EOWG http://www.w3.org/WAI/eval/conformance
15:09:08 <shadi> ...updated Working Draft out for review
15:09:14 <Vivienne> zakim, mute me
15:09:14 <Zakim> Vivienne should now be muted
15:09:25 <shadi> ...now expecting comments on it
15:09:28 <shadi> ack me
15:10:25 <shadi> http://lists.w3.org/Archives/Public/public-wai-evaltf/2013Feb/0009.html
15:12:15 <shadi> PK: prior comments from ERT WG and WCAG WG that we said we'll address after publication
15:12:23 <shadi> ...need to put these in our list too
15:13:53 <MoeKraft> MoeKraft has joined #eval
15:13:58 <shadi> EV: will review previous disposition of comments
15:14:05 <shadi> ...and create a new one
15:14:10 <Zakim> +MoeKraft
15:14:53 <MoeKraft> q+
15:15:29 <shadi> MK: deadline for comments is not very long
15:15:35 <shadi> EV: can extend if needed
15:15:37 <shadi> ack me
15:15:47 <MoeKraft> q-
15:17:04 <shadi> SAZ: can extend if needed, but also want to have new draft before the summer breaks
15:17:16 <ericvelleman> q?
15:17:18 <shadi> EV: can keep the deadline for now and extend in the last week
15:17:33 <shadi> Topic: Testing WCAG-EM
15:17:58 <shadi> EV: proposal for Eval TF participants to check a website using WCAG-EM
15:18:05 <shadi> ...and compare the results we get
15:18:20 <Vivienne> q+
15:18:30 <Vivienne> ack me
15:18:34 <shadi> PK: like the idea
15:19:04 <shadi> VC: good test for our process
15:19:07 <shadi> ...but tricky which website we pick
15:19:14 <shadi> ...good to use the same website
15:19:27 <shadi> ...need to consider what features it presents
15:19:35 <ericvelleman> q?
15:19:41 <MoeKraft> q+
15:19:41 <shadi> ...should cover as many WCAG 2 requirements as possible
15:19:43 <Sarah_Swierenga> +1
15:19:44 <MartijnHoutepen> +1
15:19:45 <shadi> q+
15:19:46 <Vivienne> zakim, mute me
15:19:46 <Zakim> Vivienne should now be muted
15:19:47 <MoeKraft> +1
15:19:48 <Vivienne> +1
15:19:58 <shadi> EV: people like the idea?
15:20:16 <shadi> MK: wonder if we could reach out to the WCAG community for someone to volunteer a website
15:20:22 <shadi> ack m
15:20:28 <ericvelleman> q?
15:20:37 <Vivienne> q+
15:20:48 <Vivienne> zakim, unmute me
15:20:48 <Zakim> Vivienne should no longer be muted
15:21:04 <shadi> PK: ideally would be testing two sets of things
15:21:06 <shadi> ...the level of how we are all following the same guidance
15:21:19 <shadi> ...but also looking at what the challenge areas are in the guidance
15:21:30 <shadi> ...for example web applications could be a challenge area
15:21:58 <shadi> ...could be different then how effective the guidance is for people to follow equally
15:22:00 <Sarah_Swierenga> q+
15:22:19 <shadi> EV: may mean that we look at more than one website
15:22:21 <shadi> ack me
15:22:29 <ericvelleman> q?
15:25:04 <shadi> SAZ: Kathy had the idea of splitting volunteers into groups
15:25:09 <shadi> ...then we can address coverage of different types of websites
15:25:27 <shadi> ...but also have comparable notes
15:26:02 <ericvelleman> q?
15:26:15 <shadi> PK: could also use fictional websites
15:27:15 <shadi> SAZ: we have the Before and After Demo (BAD) but quite difficult to develop such sites
15:27:25 <shadi> ...might be better to use existing ones
15:27:36 <shadi> VC: may have some candidate sites in mind
15:28:14 <korn> korn has joined #eval
15:28:15 <Vivienne> zakim, mute me
15:28:15 <Zakim> Vivienne should now be muted
15:28:25 <shadi> SAZ: please send to me and Eric before contacting them
15:28:32 <Zakim> +[Oracle]
15:28:37 <shadi> ...need to determine volunteers first
15:28:40 <Zakim> -Peter
15:28:55 <korn> Zakim, Oracle has Peter_Korn
15:28:55 <Zakim> +Peter_Korn; got it
15:28:56 <korn> q+
15:29:34 <shadi> SW: could have someone do the sampling then others evaluate these samples
15:29:41 <shadi> q- viv
15:29:46 <shadi> q- sarag
15:29:50 <shadi> q- sara
15:29:54 <MartijnHoutepen> q+
15:30:06 <shadi> q+
15:30:18 <shadi> ...otherwise not comparable results
15:30:36 <shadi> PK: part of what we are testing is the sampling methodology
15:30:54 <ericvelleman> q+
15:30:58 <shadi> ...need to include the sampling as part of the testing
15:31:10 <shadi> ...if we get diverging results then the methodology is not ready for prime time
15:31:31 <shadi> ...could have one group follow this approach
15:31:44 <shadi> ...or be able to factor out the sampling aspect
15:31:56 <MartijnHoutepen> ack me
15:32:01 <shadi> ...but need to also test how the sampling guidance works
15:32:01 <shadi> q-
15:32:05 <shadi> q- k
15:32:21 <shadi> MH: could also evaluate our evaluations after each step
15:32:22 <ericvelleman> q-
15:32:34 <shadi> ...like compare notes after each step
15:33:07 <shadi> EV: had similar idea in a project in The Netherlands
15:33:27 <shadi> ...first see what samples people come up with
15:33:38 <shadi> ...then all evaluate the same sample
15:33:54 <Sarah_Swierenga> q+
15:33:57 <shadi> ...but could be extended to different scenarios
15:33:58 <MartijnHoutepen> zakim, mute me
15:33:59 <Zakim> MartijnHoutepen should now be muted
15:33:59 <ericvelleman> q?
15:34:03 <shadi> ack sarah
15:34:23 <shadi> SW: agree with Peter, like the idea that everyone has the same core of web pages
15:34:35 <shadi> ...but need to test the same pages as well
15:35:01 <shadi> s/the same pages as well/the sampling procedure as well
15:38:40 <shadi> [Proposal: ~3 teams of ~3-4 people to evaluate a website per team in two phases; phase would combine steps 1-3 of WCAG-EM (then compare notes); phase 2 would combine steps 4-5 of WCAG-EM (then compare notes)]
15:39:15 <shadi> PK: do we need teams?
15:39:21 <shadi> ...need to compare notes
15:39:36 <shadi> ...but if they compare notes in between it mixes results
15:40:06 <shadi> EV: so not comparison between the phases?
15:42:31 <shadi> PK: comparing notes part way through could muddy the results
15:44:52 <shadi> ...why have people talk if we are trying to test the methodology in its entirety?
15:45:08 <shadi> q+
15:45:39 <shadi> EV: should get to the same result with different samples
15:45:41 <shadi> ack me
15:48:05 <MartijnHoutepen> I will volunteer
15:48:09 <Vivienne> I'm happy to and I can get some of my staff to help
15:48:09 <Sarah_Swierenga> +1
15:48:27 <MoeKraft> +1 I would like to be involved and/or have someone from my team assist in this effort.
15:48:34 <shadi> SAZ: the more data points we have the more we know what we need to improve
15:48:44 <MoeKraft> I will be away however from 3/28-4/11
15:49:04 <shadi> EV: who would volunteer?
15:50:09 <shadi> SAZ: need to ask absent people as well
15:51:26 <ericvelleman> q?
15:53:36 <Vivienne> q+
15:53:42 <Vivienne> ack me
15:53:52 <shadi> EV: ask for public volunteers?
15:53:59 <shadi> PK: could do 2-step
15:54:19 <shadi> ...inviting the public at this stage would impact confidentiality
15:54:21 <shadi> q+
15:54:28 <Vivienne> q+
15:54:34 <shadi> EV: already asking people to test the methodology
15:54:56 <shadi> ack viv
15:55:06 <shadi> VC: better if we don't for this first test
15:55:28 <shadi> ...because we know more about how this methodology was intended
15:55:40 <shadi> ...would be testing too many parameters if we open it up
15:55:45 <Vivienne> zakim, mute me
15:55:45 <Zakim> Vivienne should now be muted
15:56:01 <shadi> ack me
15:57:23 <shadi> SAZ: agree with Vivienne, need to do our homework first
15:58:27 <ericvelleman> <https://www.w3.org/2002/09/wbs/48225/evaltfq9/>
15:58:35 <shadi> Topic: Survey 09
15:59:01 <shadi> EV: your input on what we need to discuss in next steps
15:59:02 <shadi> ...already good suggestions there
15:59:25 <shadi> ...but please add others you think are missing
15:59:33 <shadi> ...or that need additional attention
15:59:59 <MartijnHoutepen> ack me
16:00:01 <Vivienne> good night all
16:00:04 <Zakim> -[Oracle]
16:00:08 <Zakim> -Sarah_Swierenga
16:00:09 <Zakim> -MoeKraft
16:00:09 <Zakim> -Shadi
16:00:10 <Sarah_Swierenga> bye
16:00:10 <Zakim> -Vivienne
16:00:10 <Zakim> -MartijnHoutepen
16:00:13 <ericvelleman> ericvelleman has left #eval
16:00:18 <Zakim> -ericvelleman
16:00:19 <Zakim> WAI_ERTWG(Eval TF)10:00AM has ended
16:00:19 <Zakim> Attendees were +1.510.334.aaaa, +1.517.432.aabb, Shadi, +31.30.239.aacc, Vivienne, MartijnHoutepen, Sarah_Swierenga, +31.30.239.aadd, ericvelleman, Peter, MoeKraft, Peter_Korn
16:00:26 <shadi> trackbot, end meeting
16:00:26 <trackbot> Zakim, list attendees
16:00:27 <Zakim> sorry, trackbot, I don't know what conference this is
16:00:34 <trackbot> RRSAgent, please draft minutes
16:00:34 <RRSAgent> I have made the request to generate http://www.w3.org/2013/03/07-eval-minutes.html trackbot
16:00:35 <trackbot> RRSAgent, bye
16:00:35 <RRSAgent> I see no action items