14:46:51 <RRSAgent> RRSAgent has joined #eval
14:46:51 <RRSAgent> logging to http://www.w3.org/2014/03/27-eval-irc
14:46:53 <trackbot> RRSAgent, make logs world
14:46:53 <Zakim> Zakim has joined #eval
14:46:55 <trackbot> Zakim, this will be 3825
14:46:55 <Zakim> ok, trackbot; I see WAI_ERTWG(Eval TF)11:00AM scheduled to start in 14 minutes
14:46:56 <trackbot> Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
14:46:56 <trackbot> Date: 27 March 2014
14:54:58 <shadi> zakim, call shadi-617
14:54:58 <Zakim> ok, shadi; the call is being made
14:54:59 <Zakim> WAI_ERTWG(Eval TF)11:00AM has now started
14:55:00 <Zakim> +Shadi
14:57:57 <Liz> Liz has joined #eval
14:58:35 <Zakim> +Liz
14:59:29 <shadi> agenda+ Call for Participation: WCAG-EM Test Run
15:00:05 <Zakim> +Kathy_Wahlbin
15:00:08 <shadi> agenda+ Removing version number from title
15:00:17 <Detlev> Detlev has joined #eval
15:00:21 <alistair> alistair has joined #eval
15:00:36 <alistair> Going to be a few minutes late
15:01:12 <shadi> agenda+ Merging "Review Teams" and "Required Expertise"
15:01:58 <shadi> agenda+ Clarifying "Define Evaluation Methods to be Used"
15:02:36 <shadi> agenda+ Continued discussion on "Scoring"
15:02:37 <Detlev> I get "this pass code is not valid" msg...
15:02:44 <shadi> zakim, code?
15:02:44 <Zakim> the conference code is 3825 (tel:+1.617.761.6200 sip:zakim@voip.w3.org), shadi
15:03:03 <Detlev> That's what I am keing in...
15:03:19 <Detlev> zakim, mute me
15:03:20 <Zakim> sorry, Detlev, I do not know which phone connection belongs to you
15:04:00 <Detlev> tzhe phone is on mute... I had this problem a few tiems but usually it was Ok on re-dial - tried that laready today - will try again
15:04:58 <shadi> regrets: Moe
15:05:15 <shadi> regrets: Moe, Eric, Martijn
15:05:49 <Sarah_Swierenga> Sarah_Swierenga has joined #eval
15:06:23 <Zakim> +Detlev
15:06:25 <Zakim> +Sarah_Swierenga
15:07:27 <Zakim> +[IPcaller]
15:07:27 <alistair> zakim, ipcaller is alistair
15:07:28 <Detlev> ack me
15:07:28 <Zakim> +alistair; got it
15:07:44 <Detlev> Zakim, mute me
15:07:44 <Zakim> Detlev should now be muted
15:08:33 <Sarah_Swierenga> Scribe: Sarah Swierenga
15:08:36 <shadi> scribenick: Sarah_Swierenga
15:08:41 <shadi> agenda?
15:09:24 <shadi> http://lists.w3.org/Archives/Public/public-wai-evaltf/2014Mar/0080.html
15:09:39 <shadi> Topic: Debreif from Face-to-Face meeting
15:09:52 <Detlev> who took part?
15:10:12 <shadi> http://www.w3.org/2014/03/17-eval-minutes
15:10:23 <shadi> Shadi_AZ, MaryJo_M, Eric_V, Katie_HS, Kathy_W, Tim_B, David_M, Michael_C (partially), Mike_E (partially), Judy_B (partially), Shawn_H (partially), Klaus_M (partially)
15:10:47 <shadi> http://www.w3.org/WAI/ER/conformance/comments-20140130
15:14:33 <Sarah_Swierenga> during the face-to-face mtg , we went through all substantial comments received so far
15:15:19 <Sarah_Swierenga> We relayed to the WCAG working group that there is a need for point to point evaluation of individual success criteria.
15:15:59 <Sarah_Swierenga> We also discussed the issue of scoring, but we need to have more discussion about that
15:16:12 <shadi> zakim, take up next
15:16:12 <Zakim> agendum 1. "Call for Participation: WCAG-EM Test Run" taken up [from shadi]
15:16:40 <shadi> http://lists.w3.org/Archives/Public/public-wai-evaltf/2014Mar/0083.html
15:17:07 <Sarah_Swierenga> april 10 is the preferred deadline for the internal website test run
15:17:45 <Sarah_Swierenga> we need to gather experiential feedback on the EM from the test run to finalize the document
15:18:03 <Sarah_Swierenga> q+
15:18:11 <shadi> ack sarah
15:18:16 <Detlev> q+
15:18:17 <Liz> Liz volunteered
15:18:45 <Liz> q+
15:19:07 <Sarah_Swierenga> Sarah: UARC is almost done with the review, and will be filling in the survey early next week.
15:19:07 <Detlev> ack me
15:19:12 <shadi> ack me
15:19:22 <Sarah_Swierenga> Detlev: no audio
15:19:24 <Detlev> ack me
15:19:43 <shadi> ack liz
15:19:45 <Sarah_Swierenga> Detlev: questions about external eval
15:20:43 <Sarah_Swierenga> Liz: looking at survey that Eric sent. He asked several questions about timing. I am willing to try the methodology, but am questionning the need for very precise timings.
15:21:17 <Sarah_Swierenga> Shadi: Precise minutes/seconds on timing, if possible, will be helpful.
15:21:46 <Sarah_Swierenga> Liz: I will be adding comments about the EM, and will try to get some timings.
15:22:12 <shadi> s/Precise minutes/does not need to be precise
15:22:34 <Sarah_Swierenga> Shadi: Correction: timings do not need to be precise, but general timings are helpful
15:23:04 <Sarah_Swierenga> Shadi: we want to have varying level of expertise using the EM
15:23:49 <Sarah_Swierenga> Shadi: We want to understanding general timings for various experience levels, to see what we can do to minmize the time involved to do the review.
15:24:33 <Sarah_Swierenga> Shadi: We want to get as many internal and external surveys completed as possible. Two surveys are up and running.
15:24:54 <Detlev> http://lists.w3.org/Archives/Public/public-wai-evaltf/2014Mar/0082.html
15:26:14 <Sarah_Swierenga> Detlev: External/open survey - it is a good idea to communicate that we want lots of feedback - even if they don't complete the whole survey. Concerned that we have 0 responses.
15:26:54 <Sarah_Swierenga> Detlev: The open survey url doesn't seem to have been distributed widely. Can we do more to get the news out?
15:27:39 <Sarah_Swierenga> Shadi: Eric and I recruited heavily at CSUN.
15:29:02 <Sarah_Swierenga> Shadi: We don't typically ask for this level of volunteer work for these kinds of projects. Also wondering if we will get the right people if we broadcast too widely.
15:29:32 <Sarah_Swierenga> Detlev: This is a public doc and a EM that we expect the public to use when it's published.
15:30:06 <Tim> Tim has joined #eval
15:30:29 <Sarah_Swierenga> Detlev: We should ask people to try it out, and get their feel for how the EM works. We should send it out broadly to get as much participation as possible.
15:31:00 <martijnhoutepen> martijnhoutepen has joined #eval
15:31:42 <Sarah_Swierenga> Shadi: We have published the docuement to get public comment, and have asked people to do a test run (which is more unusual for a W3C working group). Is there a quality vs quantity concern?
15:32:24 <Sarah_Swierenga> Shadi: We should certainly reach out to people who currently do evaluations to try out the EM.
15:33:03 <Sarah_Swierenga> Shadi: Does the group know of other people to reach out to for the test run?
15:33:05 <Detlev> q+
15:33:50 <Sarah_Swierenga> Shadi: time remaining is working against us now, too. We would likely need to extend the timeline.
15:34:14 <shadi> ack det
15:35:30 <Sarah_Swierenga> Detlev: We still need to get the word out. This is a small group, so we need an outside view. Awareness is key, and it is a shame not to get that broader feedback. We need to work harder to get more participation - even a handful would be helpful.
15:36:14 <Sarah_Swierenga> Detlev: Recommends around 10, but we need to ask many to get a few to actually conduct the evaluation.
15:37:11 <Sarah_Swierenga> Shadi: If each of us recruits 1-2 people to participate, we could get the feedback we need.
15:37:38 <Sarah_Swierenga> Detlev: is willing to post to WebAim list. is that ok?
15:38:29 <Sarah_Swierenga> Detlev: Disappointed that we haven't seen broader requests for participation in the test run.
15:38:34 <Detlev> I won't
15:39:39 <Sarah_Swierenga> Shadi: You can send to the WebAim list, but we really want people who are qualified to do the evaluation, and who can take the time to do the evaluation, so that we can get valuable, targeted feedback.
15:39:56 <Detlev> Ok, fine.
15:40:19 <Sarah_Swierenga> Shadi: we have the public comments and survey out there right now. Shadi and Eric will explore this further.
15:40:25 <shadi> zakim, take up next
15:40:25 <Zakim> agendum 2. "Removing version number from title" taken up [from shadi]
15:41:41 <Sarah_Swierenga> Shadi: Do we need a version number in the document.
15:41:42 <Zakim> +??P1
15:41:44 <Sarah_Swierenga> q+
15:42:17 <Sarah_Swierenga> shadi: every pub is dated
15:42:38 <shadi> ack sarah
15:42:56 <shadi> zakim, ??p1 is Tim
15:42:56 <Zakim> +Tim; got it
15:43:35 <Sarah_Swierenga> Sarah: having 1.0 in the title might be confusing since wcag is 2.0.
15:44:15 <shadi> http://www.w3.org/TR/WCAG-EM/
15:44:16 <Sarah_Swierenga> Shadi: every pub in TR space has a date, and revision dates. "this version" and "latest version"
15:44:54 <shadi> zakim, take up next
15:44:54 <Zakim> agendum 3. "Merging "Review Teams" and "Required Expertise"" taken up [from shadi]
15:44:59 <Sarah_Swierenga> +1 for no version number in the title
15:46:25 <Sarah_Swierenga> Shadi: concern about the review teams. the purpose of the review teams is that the evaluator needs a combination of skills and expertise, which one person may or may not have. A team approach with the combined expertise is the key.
15:47:23 <shadi> zakim, take up next
15:47:23 <Zakim> agendum 4. "Clarifying "Define Evaluation Methods to be Used"" taken up [from shadi]
15:47:25 <Sarah_Swierenga> Shadi: recommendation was to have the mulitple person review teams in the Required Expertise section. Think about this.
15:49:14 <Sarah_Swierenga> Shadi: we will also need to discuss this section in a survey and weekly call. this section needs to be refined again. Also discussed with the WCAG Working Group that we need to think about recommened techniques, but specific techniques should be clarified in the W3C Working Group.
15:50:07 <Sarah_Swierenga> Shadi: Green light that we reference the WG's techniques, as they clarify them more.
15:50:51 <shadi> zakim, take up next
15:50:51 <Zakim> agendum 5. "Continued discussion on "Scoring"" taken up [from shadi]
15:54:14 <Detlev> zakim. mute me
15:54:27 <Detlev> Zakim, mute me
15:54:28 <Zakim> Detlev should now be muted
15:54:32 <Detlev> can do
15:54:38 <Detlev> scribe: Detlev
15:54:40 <shadi> scribe: Detlev
15:54:57 <Detlev> ack me
15:55:46 <Sarah_Swierenga_> Sarah_Swierenga_ has joined #eval
15:55:58 <Sarah_Swierenga_> Sarah: I think I'm back in
15:56:14 <Tim> Tim has joined #eval
15:56:24 <Detlev> Art CSUN there was an engaged discussion with the WCAG WG on scoring, varied views of peope reg. scoring: needed for accounting fo rminor issues, afford comparison, etc, but others having valid concerns
15:57:08 <shadi> scribenick: Sarah_Swierenga
15:57:10 <Sarah_Swierenga_> Shadi: We will need to come back to this item.
15:57:25 <Sarah_Swierenga_> Tim: Lots of discussion in his CSUN session - and concerns
15:58:12 <Sarah_Swierenga_> Shadi: Comment that got traction was that since any approach to scoring will have issues associated with it, that W3C should not sanction any specific scoring system. Rather provide do
15:58:37 <Sarah_Swierenga_> Shadi: provide do's and dont's
15:59:59 <Sarah_Swierenga_> Shadi: Providing a variety of scoring methods, but let people know they can use their own. Or, we can recommend what they doing with their particular scoring method.
16:00:47 <Sarah_Swierenga_> Shadi: Please perform the test run and find others to do it as well.
16:00:48 <Detlev> bye
16:00:52 <Liz> bye
16:00:53 <Zakim> -alistair
16:00:53 <Zakim> -Tim
16:00:54 <Zakim> -Shadi
16:00:55 <Zakim> -Sarah_Swierenga
16:00:55 <Zakim> -Detlev
16:00:56 <Zakim> -Kathy_Wahlbin
16:01:01 <Zakim> -Liz
16:01:02 <Zakim> WAI_ERTWG(Eval TF)11:00AM has ended
16:01:02 <Zakim> Attendees were Shadi, Liz, Kathy_Wahlbin, Detlev, Sarah_Swierenga, alistair, Tim
17:04:54 <shadi> trackbot, end meeting
17:04:54 <trackbot> Zakim, list attendees
17:04:54 <Zakim> sorry, trackbot, I don't know what conference this is
17:05:02 <trackbot> RRSAgent, please draft minutes
17:05:02 <RRSAgent> I have made the request to generate http://www.w3.org/2014/03/27-eval-minutes.html trackbot
17:05:03 <trackbot> RRSAgent, bye
17:05:03 <RRSAgent> I see no action items