See also: IRC log
<shadi> http://www.w3.org/2014/03/17-eval-minutes
<shadi> Shadi_AZ, MaryJo_M, Eric_V, Katie_HS, Kathy_W, Tim_B, David_M, Michael_C (partially), Mike_E (partially), Judy_B (partially), Shawn_H (partially), Klaus_M (partially)
<shadi> http://www.w3.org/WAI/ER/conformance/comments-20140130
during the face-to-face mtg , we went through all substantial comments received so far
We relayed to the WCAG working group that there is a need for point to point evaluation of individual success criteria.
We also discussed the issue of scoring, but we need to have more discussion about that
<shadi> http://lists.w3.org/Archives/Public/public-wai-evaltf/2014Mar/0083.html
april 10 is the preferred deadline for the internal website test run
we need to gather experiential feedback on the EM from the test run to finalize the document
<Liz> Liz volunteered
Sarah: UARC is almost done with the review, and will be filling in the survey early next week.
Detlev: no audio
... questions about external eval
Liz: looking at survey that Eric sent. He asked several questions about timing. I am willing to try the methodology, but am questionning the need for very precise timings.
Shadi: does not need to be precise/seconds on timing, if possible, will be helpful.
Liz: I will be adding comments about the EM, and will try to get some timings.
Shadi: Correction: timings do not need to be
precise, but general timings are helpful
... we want to have varying level of expertise using the EM
... We want to understanding general timings for various experience levels, to
see what we can do to minmize the time involved to do the review.
... We want to get as many internal and external surveys completed as
possible. Two surveys are up and running.
<Detlev> http://lists.w3.org/Archives/Public/public-wai-evaltf/2014Mar/0082.html
Detlev: External/open survey - it is a good idea
to communicate that we want lots of feedback - even if they don't complete the
whole survey. Concerned that we have 0 responses.
... The open survey url doesn't seem to have been distributed widely. Can we
do more to get the news out?
Shadi: Eric and I recruited heavily at CSUN.
... We don't typically ask for this level of volunteer work for these kinds of
projects. Also wondering if we will get the right people if we broadcast too
widely.
Detlev: This is a public doc and a EM that we
expect the public to use when it's published.
... We should ask people to try it out, and get their feel for how the EM
works. We should send it out broadly to get as much participation as
possible.
Shadi: We have published the docuement to get
public comment, and have asked people to do a test run (which is more unusual
for a W3C working group). Is there a quality vs quantity concern?
... We should certainly reach out to people who currently do evaluations to
try out the EM.
... Does the group know of other people to reach out to for the test run?
... time remaining is working against us now, too. We would likely need to
extend the timeline.
Detlev: We still need to get the word out. This
is a small group, so we need an outside view. Awareness is key, and it is a
shame not to get that broader feedback. We need to work harder to get more
participation - even a handful would be helpful.
... Recommends around 10, but we need to ask many to get a few to actually
conduct the evaluation.
Shadi: If each of us recruits 1-2 people to participate, we could get the feedback we need.
Detlev: is willing to post to WebAim list. is
that ok?
... Disappointed that we haven't seen broader requests for participation in
the test run.
<Detlev> I won't
Shadi: You can send to the WebAim list, but we really want people who are qualified to do the evaluation, and who can take the time to do the evaluation, so that we can get valuable, targeted feedback.
<Detlev> Ok, fine.
Shadi: we have the public comments and survey out there right now. Shadi and Eric will explore this further.
Shadi: Do we need a version number in the
document.
... every pub is dated
Sarah: having 1.0 in the title might be confusing since wcag is 2.0.
<shadi> http://www.w3.org/TR/WCAG-EM/
Shadi: every pub in TR space has a date, and revision dates. "this version" and "latest version"
+1 for no version number in the title
Shadi: concern about the review teams. the purpose of the review teams is that the evaluator needs a combination of skills and expertise, which one person may or may not have. A team approach with the combined expertise is the key.
Shadi: recommendation was to have the mulitple
person review teams in the Required Expertise section. Think about this.
... we will also need to discuss this section in a survey and weekly call.
this section needs to be refined again. Also discussed with the WCAG Working
Group that we need to think about recommened techniques, but specific
techniques should be clarified in the W3C Working Group.
... Green light that we reference the WG's techniques, as they clarify them
more.
Art CSUN there was an engaged discussion with the WCAG WG on scoring, varied views of peope reg. scoring: needed for accounting fo rminor issues, afford comparison, etc, but others having valid concerns
<Sarah_Swierenga_> Shadi: We will need to come back to this item.
<Sarah_Swierenga_> Tim: Lots of discussion in his CSUN session - and concerns
<Sarah_Swierenga_> Shadi: Comment that got traction was that since any approach to scoring will have issues associated with it, that W3C should not sanction any specific scoring system. Rather provide do
<Sarah_Swierenga_> Shadi: provide do's and dont's
<Sarah_Swierenga_> Shadi: Providing a variety of scoring methods, but let people know they can use their own. Or, we can recommend what they doing with their particular scoring method.
<Sarah_Swierenga_> Shadi: Please perform the test run and find others to do it as well.