14:54:18 <RRSAgent> RRSAgent has joined #eval
14:54:21 <RRSAgent> logging to http://www.w3.org/2014/01/30-eval-irc
14:54:21 <trackbot> RRSAgent, make logs world
14:54:22 <trackbot> Zakim, this will be 3825
14:54:22 <Zakim> ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 6 minutes
14:54:23 <trackbot> Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
14:54:23 <trackbot> Date: 30 January 2014
14:55:18 <shadi> zakim, this is eval
14:55:18 <Zakim> shadi, I see WAI_ERTWG(Eval TF)10:00AM in the schedule but not yet started.  Perhaps you mean "this will be eval".
14:58:15 <Zakim> WAI_ERTWG(Eval TF)10:00AM has now started
14:58:22 <Zakim> +Shadi
14:58:50 <EricVelleman> EricVelleman has joined #eval
15:00:36 <Zakim> +EricVelleman
15:02:18 <MoeKraft> MoeKraft has joined #eval
15:02:36 <shadi> http://www.w3.org/blog/news/archives/3618
15:02:37 <Zakim> +MoeKraft
15:03:14 <richard> richard has joined #eval
15:03:21 <Zakim> + +1.410.649.aaaa
15:04:04 <shadi> regrets: MaryJo, Gavin
15:04:09 <Zakim> +[IPcaller]
15:04:14 <Sarah_Swierenga> Sarah_Swierenga has joined #eval
15:04:20 <shadi> zakim, aaaa is Tim
15:04:22 <Zakim> +Tim; got it
15:04:45 <shadi> zakim, ipcaller is richard
15:04:45 <Zakim> +richard; got it
15:04:48 <Zakim> +Sarah_Swierenga
15:05:39 <shadi> regrets: MaryJo, Gavin, Kathy
15:06:11 <shadi> scribe: shadi
15:06:19 <shadi> zakim, mute me
15:06:19 <Zakim> Shadi should now be muted
15:06:40 <shadi> agenda: http://lists.w3.org/Archives/Public/public-wai-evaltf/2014Jan/0023.html
15:07:07 <shadi> agenda+ Update on WCAG-EM publication
15:07:16 <shadi> zakim, clear agenda
15:07:16 <Zakim> agenda cleared
15:07:18 <shadi> agenda+ Update on WCAG-EM publication
15:07:39 <shadi> agenda+ Conceptual issues in WCAG-EM
15:07:42 <shadi> agenda+ Test Run
15:07:44 <shadi> ack me
15:07:56 <shadi> agenda+ Face-to-Face Meeting
15:08:02 <shadi> zakim, take up first item
15:08:02 <Zakim> I don't understand 'take up first item', shadi
15:08:08 <shadi> zakim, take up agendum 1
15:08:08 <Zakim> agendum 1. "Update on WCAG-EM publication" taken up [from shadi]
15:08:24 <shadi> http://www.w3.org/blog/news/archives/3618
15:08:36 <shadi> http://www.w3.org/TR/WCAG-EM/
15:09:40 <shadi> SAZ: WAI IG announcement coming up
15:09:44 <MoeKraft> q+
15:09:57 <shadi> ...feel free to circulate that around
15:10:11 <shadi> ...also to re-tweet the WAI twitter message
15:10:13 <Zakim> +[IPcaller]
15:10:17 <Vivienne> zakim, IPcaller is me
15:10:17 <Zakim> +Vivienne; got it
15:10:27 <EricVelleman> q?
15:11:42 <shadi> ack moe
15:12:03 <shadi> MK: link the WAI IG announcement fromn blog post
15:12:20 <shadi> SAZ: the W3C News item comes first, then IG message
15:12:34 <shadi> ...but IG message will be sent to Eval TF list
15:12:46 <shadi> ...feel free to circulate that when it arrives
15:12:54 <shadi> TB: next steps?
15:13:01 <shadi> EV: want to do a test run
15:13:39 <shadi> ...commenting period until 28 February
15:13:42 <shadi> ...can also talk about some of the conceptual issues
15:14:14 <shadi> zakim, take up next
15:14:14 <Zakim> agendum 2. "Conceptual issues in WCAG-EM" taken up [from shadi]
15:14:40 <shadi> EV: which areas in the current document need more discussion?
15:14:53 <shadi> q+
15:14:53 <Vivienne> q+
15:14:58 <shadi> ack me
15:15:41 <Vivienne> ack me
15:15:46 <shadi> SAZ: areas highlighted in the document with "Review Note"
15:16:07 <shadi> ...sample selection, scoring, and documentation
15:16:23 <Vivienne> zakim, mute me
15:16:23 <Zakim> Vivienne should now be muted
15:16:34 <shadi> VC: need to refine some of the sections
15:16:42 <shadi> ...read through them in detail
15:17:11 <shadi> EV: combined with the test run to ready very carefully in practice
15:17:37 <shadi> ...also to compare with people's own current procedures
15:17:40 <shadi> ...which sections work and which don't
15:17:52 <shadi> ...and why not, if certain sections don't work
15:18:45 <shadi> SAZ: need this to be an action?
15:18:54 <shadi> ...to read it by next week?
15:19:06 <shadi> EV: can put it as a request on the mailing list
15:19:37 <shadi> zakim, take up next
15:19:37 <Zakim> agendum 3. "Test Run" taken up [from shadi]
15:19:52 <shadi> EV: goal is to see how people use the methodology
15:20:05 <shadi> ...do people come to the same evaluation result?
15:20:21 <shadi> ...important time to specifically test this
15:20:40 <shadi> ...do an evaluation and compare the outcomes
15:21:04 <shadi> ...need to decide what we want to measure, which website, and how to capture the data
15:21:19 <shadi> ...and also find people willing to do a full review
15:21:39 <shadi> ...probably take a full-day worth of work for this
15:21:51 <Vivienne> q+
15:21:56 <Vivienne> zakim, ack me
15:21:56 <Zakim> unmuting Vivienne
15:21:57 <Zakim> I see no one on the speaker queue
15:21:58 <shadi> ack viv
15:22:17 <richard> zakim, q+
15:22:17 <Zakim> I see richard on the speaker queue
15:23:08 <shadi> VC: have been following WCAG-EM with my team
15:23:20 <shadi> ...works very well but not sure what you are asking for
15:23:39 <shadi> ...would need to compare it to something else
15:23:45 <shadi> TB: would need to compare it to another methodology
15:23:52 <shadi> ...do a comparative study
15:23:55 <Vivienne> zakim, mute me
15:23:55 <Zakim> Vivienne should now be muted
15:24:25 <shadi> RW: need to run WCAG-EM and our own methodologie
15:24:45 <shadi> ...and provide feedback on how it worked in practice
15:25:04 <shadi> ...usually needs a couple of runs until you get a hold of it
15:25:14 <shadi> ...need a panel to do that
15:25:21 <Zakim> -richard
15:25:44 <Zakim> +[IPcaller]
15:26:06 <shadi> EV: could we ask people to use WCAG-EM on the same website
15:26:20 <shadi> ...and compare what people did in each case
15:26:36 <richard> zakim, ipcaller is richard
15:26:36 <Zakim> +richard; got it
15:26:38 <Sarah_Swierenga> +1 I agree that we should use the same website.
15:26:49 <Vivienne> We still have our university library's approval to use them as an example for this comparison
15:26:50 <shadi> ...like which sample did they pick, how they got the results, etc
15:27:10 <richard> +1 for same website
15:27:26 <Vivienne> +1
15:27:32 <MoeKraft> +1
15:27:46 <MoeKraft> q+
15:28:05 <shadi> q- rich
15:28:15 <shadi> ack moe
15:28:45 <shadi> MK: would be helpful to have a survey to help organize the feedback
15:29:18 <shadi> ...liked previous organization of results
15:29:37 <shadi> RW: can do that for the new data as well
15:30:03 <shadi> EV: document has flexibility in it so might not be as comparable
15:30:07 <richard> q+
15:30:13 <shadi> ...but what data do we want to collect?
15:30:39 <shadi> ...think we need to pre-decide the website but also the scope
15:30:50 <richard> Accurate scope is essential
15:30:59 <shadi> ...that would set the perimeter
15:31:07 <richard> q-
15:31:16 <Zakim> -Tim
15:31:37 <MoeKraft> ------------------------------------------------------ 1. Introduction Please remember that we are testing the methodology, not WCAG or your personal ability to evaluate websites. As a member of the Eval task-force you fit into our profile by having the required expertise (http://www.w3.org/TR/WCAG-EM/#expertise) so please feel free to comment as much as possible.   Please follow the text in the Methodology as close as possible.   2. Step 1: Define the Ev[CUT]
15:31:42 <shadi> MK: previous results reminded that we are testing WCAG-EM not WCAG itself
15:32:34 <Sarah_Swierenga> Sorry, I need to hop off of this call. I have a few work issues I need to deal with right now. Have a good week. Sarah
15:32:45 <Zakim> -Sarah_Swierenga
15:32:48 <Vivienne> sounds good
15:32:51 <MoeKraft> +1
15:33:04 <Vivienne> +1
15:33:10 <shadi> http://lists.w3.org/Archives/Public/public-wai-evaltf/2013May/0027.html
15:33:17 <richard> +1 we must issue step 1
15:34:49 <richard> q+
15:35:41 <shadi> ack
15:35:42 <shadi> ack rich
15:36:08 <shadi> RW: need to get confirmation that it is understandable and reliable
15:37:50 <richard> Q+
15:38:39 <shadi> ack rich
15:38:58 <shadi> RW: testing if the website complies with WCAG 2.0
15:39:59 <shadi> ...looking for fails most of the time
15:40:48 <shadi> ...counting failures better shows the issues
15:41:18 <shadi> ...54 problems is more powerful than 17 of 34 Success Criteria met
15:43:20 <shadi> EV: two evaluators could have different interpretation of the Success Criteira
15:45:07 <shadi> SAZ: true, that happens
15:45:28 <Vivienne> q+
15:45:36 <Vivienne> zakim, ack me
15:45:36 <Zakim> unmuting Vivienne
15:45:37 <shadi> ...but we'd be discussing WCAG 2.0 rather than WCAG-EM
15:45:38 <Zakim> I see no one on the speaker queue
15:46:01 <Vivienne> q+
15:47:12 <shadi> EV: want to understand if differences are on Success Criteria level versus sampling
15:47:22 <shadi> ...this would give feedback on WCAG-EM
15:47:39 <shadi> ack viv
15:47:41 <shadi> q+
15:48:03 <shadi> q-
15:49:58 <shadi> SAZ: think listing in Step 5.a would produce sufficient data
15:51:29 <Vivienne> q+
15:51:38 <Vivienne> zakim, ack me
15:51:38 <Zakim> I see no one on the speaker queue
15:52:26 <shadi> VC: evaluating a full website is probably more than a day
15:52:35 <shadi> ...about 40 hours
15:53:51 <shadi> EV: already shows some differences in approaches
15:54:33 <shadi> VC: we use teams as well
15:54:54 <MoeKraft> +1
15:55:02 <shadi> q+
15:55:13 <shadi> ack me
15:55:46 <richard> Could do one
15:55:58 <EricVelleman> we could also do one
15:55:59 <MoeKraft> possibly with help from my team
15:56:00 <Vivienne> I could but not until after March 31
15:56:39 <shadi> zakim, take up next
15:56:39 <Zakim> agendum 4. "Face-to-Face Meeting" taken up [from shadi]
15:57:22 <MoeKraft> q+
15:58:01 <shadi> SAZ: looking more likely but not yet fully confirmed
15:59:11 <shadi> MK: MaryJo will need to know the dates pretty soon
15:59:29 <EricVelleman> Shadi: We are looking at monday and tuesday
15:59:42 <EricVelleman> Tuesday WCAG is meeting
16:00:06 <EricVelleman> Shadi: We will hear within next week more news
16:00:54 <Vivienne> thanks all - good night
16:01:01 <Zakim> -MoeKraft
16:01:07 <Zakim> -Vivienne
16:01:08 <Zakim> -richard
16:06:12 <Zakim> -EricVelleman
16:06:14 <Zakim> -Shadi
16:06:14 <Zakim> WAI_ERTWG(Eval TF)10:00AM has ended
16:06:14 <Zakim> Attendees were Shadi, EricVelleman, MoeKraft, +1.410.649.aaaa, Tim, richard, Sarah_Swierenga, Vivienne
16:06:19 <EricVelleman> EricVelleman has left #eval
16:06:45 <shadi> trackbot, end meeting
16:06:45 <trackbot> Zakim, list attendees
16:06:45 <Zakim> sorry, trackbot, I don't know what conference this is
16:06:53 <trackbot> RRSAgent, please draft minutes
16:06:53 <RRSAgent> I have made the request to generate http://www.w3.org/2014/01/30-eval-minutes.html trackbot
16:06:54 <trackbot> RRSAgent, bye
16:06:54 <RRSAgent> I see no action items