13:45:55 <RRSAgent> RRSAgent has joined #eval
13:45:55 <RRSAgent> logging to http://www.w3.org/2014/05/08-eval-irc
13:45:57 <trackbot> RRSAgent, make logs world
13:45:57 <Zakim> Zakim has joined #eval
13:45:59 <trackbot> Zakim, this will be 3825
13:45:59 <Zakim> ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 15 minutes
13:46:00 <trackbot> Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
13:46:00 <trackbot> Date: 08 May 2014
13:46:10 <martijnhoutepen> zakim, what is on the agenda?
13:46:10 <Zakim> I see nothing on the agenda
13:46:52 <martijnhoutepen> agenda+ Welcome
13:47:12 <martijnhoutepen> agenda+ Proposed rewrite of Step 1.d (evaluation methods)
13:47:35 <martijnhoutepen> agenda+ Proposed rewrite of Step 5.d
13:48:13 <martijnhoutepen> agenda+ Discussion of results of test-run survey
13:48:16 <martijnhoutepen> zakim, what is on the agenda?
13:48:16 <Zakim> I see 4 items remaining on the agenda:
13:48:17 <Zakim> 1. Welcome [from martijnhoutepen]
13:48:17 <Zakim> 2. Proposed rewrite of Step 1.d (evaluation methods) [from martijnhoutepen]
13:48:17 <Zakim> 3. Proposed rewrite of Step 5.d [from martijnhoutepen]
13:48:17 <Zakim> 4. Discussion of results of test-run survey [from martijnhoutepen]
13:52:58 <Mike_Elledge> Mike_Elledge has joined #eval
13:53:31 <Zakim> WAI_ERTWG(Eval TF)10:00AM has now started
13:53:38 <Zakim> +Mike_Elledge
13:56:58 <Kathy> Kathy has joined #eval
13:59:14 <Zakim> +Kathy_Wahlbin
13:59:35 <Vivienne> Vivienne has joined #eval
13:59:50 <ericvelleman> ericvelleman has joined #eval
14:00:22 <Zakim> +[IPcaller]
14:00:28 <Vivienne> zakim, IPcaller is me
14:00:28 <Zakim> +Vivienne; got it
14:00:42 <Zakim> +??P3
14:02:15 <Zakim> +MartijnHoutepen
14:03:43 <martijnhoutepen> scribenick: Mike_Elledge
14:03:50 <Vivienne> zakim, mute me
14:03:50 <Zakim> Vivienne should now be muted
14:03:59 <martijnhoutepen> zakim, who is on the phone
14:03:59 <Zakim> I don't understand 'who is on the phone', martijnhoutepen
14:04:06 <martijnhoutepen> Zakim, take up next
14:04:06 <Zakim> agendum 1. "Welcome" taken up [from martijnhoutepen]
14:04:15 <martijnhoutepen> Zakim, who is on the phone?
14:04:15 <Zakim> On the phone I see Mike_Elledge, Kathy_Wahlbin, Vivienne (muted), ??P3, MartijnHoutepen
14:04:42 <martijnhoutepen> Zakim, ??P3 is probably ericvelleman
14:04:42 <Zakim> +ericvelleman?; got it
14:05:07 <Mike_Elledge> Welcome everyone. No call last week, only 2 people there. Three surveys out. Eval questionaire #14, 15, 16.
14:05:35 <Detlev> Detlev has joined #eval
14:05:50 <martijnhoutepen> #2. Proposed rewrite of Step 1.d (evaluation methods)
14:05:50 <martijnhoutepen> Link: https://www.w3.org/2002/09/wbs/48225/evaltfq15/
14:05:56 <Mike_Elledge> Idea to keep open until next Tuesday, then discuss next meeting. Can discuss 5D today.
14:06:15 <Liz> Liz has joined #eval
14:06:20 <Zakim> +Detlev
14:06:22 <Mike_Elledge> #15 rewrite on 1D, evaluation methods to be used. Change current title or rewrite the section.
14:06:31 <ericvelleman> https://www.w3.org/2002/09/wbs/48225/evaltfq15/results
14:06:35 <martijnhoutepen> Zakim, take up next
14:06:35 <Zakim> agendum 2. "Proposed rewrite of Step 1.d (evaluation methods)" taken up [from martijnhoutepen]
14:07:18 <Vivienne> Sorry I haven't been too active with travelling, will get the surveys done this week
14:07:28 <Mike_Elledge> Results: Seven answers, so far. Can try to discuss on here.
14:07:49 <Zakim> +??P15
14:08:12 <martijnhoutepen> Zakim, ??P15 is Liz
14:08:12 <Zakim> +Liz; got it
14:09:09 <Mike_Elledge> First change to title. Two--accepted change  4--keep 1: do not accept for following reasons. Don't understand what we are asking evaluator to say?
14:11:05 <ericvelleman> q?
14:11:33 <Mike_Elledge> Detlev: Like the new title. don't need to enumerate all procedures using, concentrate on techniques for testing particular areas that owner has new method
14:12:59 <Mike_Elledge> Detlev: Change the title to "Define Evaluation Procedures for particulal success criteria.
14:13:07 <martijnhoutepen> zakim, mute me
14:13:07 <Zakim> MartijnHoutepen should now be muted
14:13:31 <Mike_Elledge> EV: Like two reasons you gave. Retweak the re-write for comment.
14:13:40 <Mike_Elledge> Ev: Comments?
14:13:59 <Mike_Elledge> EV: Think it would be more clear.
14:14:02 <Vivienne> +1
14:14:04 <martijnhoutepen> +1
14:14:06 <Mike_Elledge> K: Fine.
14:14:18 <Liz> change to new title
14:14:26 <Mike_Elledge> L: Fine.
14:14:31 <Mike_Elledge> M: Fine.
14:14:33 <ericvelleman> +1
14:15:02 <Mike_Elledge> EV: Will change title to Detlev's suggestion.
14:15:46 <Mike_Elledge> Det: Make them go well with text. Keep it short.
14:16:22 <Mike_Elledge> EV: Important to add D's two points. Will add to the text.
14:16:46 <Mike_Elledge> D: Easier to draft those things when they're in the text.
14:17:09 <Mike_Elledge> Ev: Add to comments field or list?
14:17:11 <Vivienne> +1
14:17:13 <Mike_Elledge> +1
14:17:14 <ericvelleman> +1
14:17:18 <Liz> +1
14:17:24 <Mike_Elledge> K; Yep
14:17:24 <martijnhoutepen> +1 for adding two points from detlevs comment
14:18:20 <Mike_Elledge> D: ? Would object. Need for more enumeration. Site owner has to describe what is needed.
14:18:44 <Vivienne> Mike, I think Detlev said Alistair would object
14:19:05 <Mike_Elledge> EV: LIke Richard's comment. That evaluator might have own way fo doing things.
14:19:15 <Mike_Elledge> M: Thanks Vivienne.
14:19:24 <Mike_Elledge> EV: Will send around for further comment.
14:19:38 <Mike_Elledge> EV: Can you change your decision?
14:19:52 <Detlev> I think "Describe Evaluation Procedures for particular Success Criteria" is better vs.  "Define Evaluation Procedures for particular Success Criteria"
14:19:57 <Detlev> Richards suggestion
14:20:00 <Mike_Elledge> EV: Would be nice, not have to do another survey.
14:20:14 <Mike_Elledge> EV: Anything else on this one?
14:20:17 <martijnhoutepen> Zakim, take up next
14:20:17 <Zakim> agendum 3. "Proposed rewrite of Step 5.d" taken up [from martijnhoutepen]
14:20:24 <ericvelleman> https://www.w3.org/2002/09/wbs/48225/evaltfq16
14:20:45 <Mike_Elledge> EV: Rewrite of 1C.
14:20:52 <martijnhoutepen> topic: rewrite of 1c
14:20:55 <Mike_Elledge> Six answers so far. Really difficult one.
14:21:39 <Mike_Elledge> Two questions: Current text and Guidance for baseline.
14:22:09 <Detlev> q+
14:22:17 <Mike_Elledge> Ev: Current text: 1 Add qeustions: 4
14:22:49 <Mike_Elledge> D: Discussing second question.
14:23:16 <Mike_Elledge> EV: Stick this first. Survey #16 first.
14:23:25 <Mike_Elledge> EV: Can come back to #15.
14:24:35 <Mike_Elledge> D: Pointless to list all adaptive technologies, browsers, etc. "When a particular choice of user agents and assistive technologies is specified by the site owner to achieve conformance and these technologies are known to the evaluator, they should be listed here."
14:24:45 <Mike_Elledge> Same as what Richard said.
14:25:31 <Mike_Elledge> EV: Like D's proposal. See this as restricted lsit.
14:26:13 <Mike_Elledge> D: Similar to what Michael said. Make it clear that you do this step when reviewing a particular site that is tuned to specific browsers and tech.
14:26:45 <Mike_Elledge> D: Main thiing to avoid confusion that don't have to list every conceivable combination of browsers and tech.
14:26:49 <Mike_Elledge> EV: Good Change?
14:26:50 <Liz> Yes
14:26:56 <ericvelleman> +1
14:27:05 <Vivienne> yes, the change is good - I just recorded my answers for this question
14:27:12 <martijnhoutepen> +1
14:27:12 <Mike_Elledge> K: Feeling a little lost. Have to look at closer.
14:27:15 <Mike_Elledge> +1
14:27:28 <Mike_Elledge> EV: Will come up with rewrite with this proposal.
14:27:31 <Tim> Tim has joined #eval
14:27:57 <Mike_Elledge> EV: Main thing is that we discussed a list of questions. Conclude that instead of list cover it with this change?
14:27:59 <Mike_Elledge> +!
14:28:02 <Mike_Elledge> +1
14:28:03 <Detlev> +1
14:28:07 <Vivienne> +1
14:28:08 <Liz> +1
14:28:10 <ericvelleman> +1
14:28:18 <martijnhoutepen> +1
14:28:23 <Tim> +1
14:28:26 <Mike_Elledge> Never too quick!
14:28:58 <Mike_Elledge> Ev: BAck to 15. Second question.
14:29:15 <ericvelleman> https://www.w3.org/2002/09/wbs/48225/evaltfq15/results
14:29:35 <Mike_Elledge> Rewrite of 1d.
14:29:54 <Mike_Elledge> EV: will rewrite and add D's two points.
14:30:41 <Mike_Elledge> Ev: Do not accept by Richard. I am not happy with "This step is optional because WCAG 2.0 conformance can be determined without the use of documented evaluation procedures". I think that, when reporting an evaluation, it is important to explain how it was done in order to give the recipient confidence that the work was carried out properly. I can live with this step being optional but do not like the implication that it is unecessary.
14:30:52 <Mike_Elledge> EV: Don't want to make it seem as unnecessary.
14:31:43 <Mike_Elledge> EV: D: Talks about other techniques but not custom. Just a technicality.
14:32:42 <Mike_Elledge> D: Pointless to list all adaptive technologies, browsers, etc. "When a particular choice of user agents and assistive technologies is specified by the site owner to achieve conformance and these technologies are known to the evaluator, they should be listed here."
14:33:03 <Mike_Elledge> D: A good practice to refer to documented procedures.
14:33:26 <Mike_Elledge> EV: would look into the text. Would add a lot of bulk.
14:33:41 <martijnhoutepen> q+
14:33:50 <Mike_Elledge> D: Not really clear what is meant.
14:33:52 <martijnhoutepen> ack D
14:34:21 <Mike_Elledge> EV: Trying to make clearer. Not sure what we meant.
14:34:54 <Mike_Elledge> EV: Now looks like have to do all the work all the time.
14:35:15 <Mike_Elledge> D: Similar to earlier question where specify particular circumstances.
14:35:29 <Mike_Elledge> D: 1D
14:35:34 <Mike_Elledge> D: Maybe can be merged.
14:35:53 <Mike_Elledge> EV: Will come up with another text proposal.
14:36:05 <Mike_Elledge> EV: and send around.
14:36:40 <martijnhoutepen> ack me
14:36:48 <Mike_Elledge> EV: Covered 1 D and 1C.
14:37:19 <ericvelleman> https://www.w3.org/2002/09/wbs/48225/evaltfq17/results
14:37:45 <Mike_Elledge> EV: Also on later section: scoring. #16--5D
14:38:29 <Mike_Elledge> EV: Lots of previous discussion. One question always comes back: no one agrees on it. No widely used metric, lots of disclaimers.
14:38:49 <Zakim> +Tim_Boland
14:39:00 <Mike_Elledge> EV: One outcome, let's look at metrics document and see if we can come up with a few examples, with same disclaimers.
14:39:46 <Mike_Elledge> EV: Problem is looks like we're still proposing scoring. My suggestion is to keep the first paragraph and delete teh rest.
14:40:22 <Mike_Elledge> EV: Seven answered: 6 accept. 1 accept with changes. Vivian says does not meet need for metric, but okay for now.
14:40:47 <Vivienne> q+
14:40:50 <Mike_Elledge> EV: Frederick suggests that research is still going on, not yet closed.
14:40:51 <Vivienne> ack me
14:40:53 <ericvelleman> q?
14:40:54 <Mike_Elledge> +!
14:40:56 <Detlev> q+
14:41:01 <Vivienne> ack me
14:41:07 <Mike_Elledge> V: Happy with F's suggestion about adding note.
14:41:55 <Mike_Elledge> V: As organization we're really looking for help with this. But it's a controversial subject. Can add something that there is ongoing research.
14:42:01 <Mike_Elledge> +1
14:42:06 <ericvelleman> q?
14:42:12 <martijnhoutepen> ack D
14:42:48 <Mike_Elledge> D: Two minds. Try to keep methodolgy short. If we don't have advice will use whatever they use. But don't mind that.
14:43:24 <Mike_Elledge> F: I think it needs to be mentioned. Can be a note. People are looking for it.
14:43:30 <Mike_Elledge> q+
14:43:45 <Mike_Elledge> F: should be indicated on document that it's being addressed.
14:44:24 <Liz> +1
14:44:40 <Mike_Elledge> M: Agree. Keep link to page with metrics document.
14:44:45 <ericvelleman> +1
14:44:47 <Vivienne> +1
14:44:49 <martijnhoutepen> +1
14:45:01 <Mike_Elledge> EV: Add ongoing research and keep link (it is previously in document).
14:45:35 <Mike_Elledge> EV: will lead to changes to current survey. Good work today.
14:46:20 <Mike_Elledge> EV: Surveys open until Tuesday, so still have opportunity to respond. Will make changes to a few, another rewrite and propose again. For next week would like to try to make editor draft of new document.
14:47:06 <Mike_Elledge> EV: Start with re-writes we've agreed to. Will start with latest batch and mark for review. Hope to have by Tuesday. If you have oppty on Wed or Thur have a look. Will keep it open. Work on this and next week.
14:47:24 <Mike_Elledge> EV: Will mark them all. So you can comment on them during telco.
14:47:25 <Vivienne> that's fine thanks
14:47:25 <Detlev> fine
14:47:26 <martijnhoutepen> +1
14:47:28 <Mike_Elledge> EV: Okay?
14:47:29 <Liz> +1
14:47:29 <Vivienne> zakim, mute me
14:47:29 <Zakim> Vivienne should now be muted
14:47:32 <Mike_Elledge> +1
14:47:36 <Tim> +1
14:47:52 <martijnhoutepen> zakim, take up next
14:47:52 <Zakim> I see a speaker queue remaining and respectfully decline to close this agendum, martijnhoutepen
14:47:59 <Detlev> -q
14:48:19 <Mike_Elledge> EV: Results of test run survey...answered by a number of people.. but anything to say Martijn
14:49:27 <Mike_Elledge> M: Yes. Interesting feedback. Tried to put on mailing list. Baseline very different.
14:50:26 <Mike_Elledge> M: With coming re-writes may get to them. People found guidance easy, comments positive. We like that. Some confusion about exploring the website. Order in which you explore different from pages you found for sample. Confusing.
14:51:54 <Mike_Elledge> M: Also, minor text issues. Processes have different understanding of: urls, explain processes a bit more...some ahve user processes, some have website processes. What users can do vs. what can be done on website.
14:52:10 <Mike_Elledge> M: Comments on that also. Can define more clearly.
14:53:34 <Mike_Elledge> EV: Step 2 order in disposition of comments already. People generally enthusiastic about methodology. Not everyone finished entire questionnaire. Not everyone had enough time. Maybe something needs to be done to ensure confidence in using our method.
14:54:24 <Mike_Elledge> M: Not in dispositioin of comments since they didn't get to end of questionnaire.
14:54:56 <Mike_Elledge> EV: Will do a mapping of the comments in survey to disposition of comments. Will cover every comment from public and test run.
14:55:14 <Mike_Elledge> M: Around 10 gave results we can use, five finished questionnaire.
14:55:29 <Mike_Elledge> q+
14:55:49 <Mike_Elledge> EV: Any comments? Other than hundreds we expected.
14:56:11 <Mike_Elledge> ME: Any surprises?
14:56:56 <Mike_Elledge> M: People found easy to use. No, big surprises. Were some big differences in how much time they took to make sample. But overall people generally agreed with it. Results as expected to me.
14:57:05 <Mike_Elledge> EV: Sounds good.
14:57:18 <Mike_Elledge> EV: Leads us to end of telco. Anything else?
14:57:28 <Mike_Elledge> K: Can you send me copies of questions?
14:57:32 <Mike_Elledge> EV: Yes.
14:58:15 <Mike_Elledge> EV: Have good days, nights, etc. Please find time Wednesday and Thursday to review revisions.
14:58:16 <Liz> bye
14:58:17 <Vivienne> good night all,  see you next week.
14:58:20 <Zakim> -Tim_Boland
14:58:21 <Mike_Elledge> Thanks and bye all!
14:58:21 <Zakim> -Detlev
14:58:21 <Zakim> -Kathy_Wahlbin
14:58:25 <Zakim> -Liz
14:58:28 <Zakim> -Vivienne
14:58:30 <Zakim> -Mike_Elledge
14:58:37 <martijnhoutepen> trackbot, end meeting
14:58:37 <trackbot> Zakim, list attendees
14:58:37 <Zakim> As of this point the attendees have been Mike_Elledge, Kathy_Wahlbin, Vivienne, MartijnHoutepen, ericvelleman?, Detlev, Liz, Tim_Boland
14:58:39 <Zakim> -MartijnHoutepen
14:58:42 <Zakim> -ericvelleman?
14:58:44 <Zakim> WAI_ERTWG(Eval TF)10:00AM has ended
14:58:44 <Zakim> Attendees were Mike_Elledge, Kathy_Wahlbin, Vivienne, MartijnHoutepen, ericvelleman?, Detlev, Liz, Tim_Boland
14:58:45 <trackbot> RRSAgent, please draft minutes
14:58:45 <RRSAgent> I have made the request to generate http://www.w3.org/2014/05/08-eval-minutes.html trackbot
14:58:46 <trackbot> RRSAgent, bye
14:58:46 <RRSAgent> I see no action items