See also: IRC log
Welcome everyone. No call last week, only 2 people there. Three surveys out. Eval questionaire #14, 15, 16.
<martijnhoutepen> #2. Proposed rewrite of Step 1.d (evaluation methods)
<martijnhoutepen> Link: https://www.w3.org/2002/09/wbs/48225/evaltfq15/
Idea to keep open until next Tuesday, then discuss next meeting. Can discuss 5D today.
#15 rewrite on 1D, evaluation methods to be used. Change current title or rewrite the section.
<ericvelleman> https://www.w3.org/2002/09/wbs/48225/evaltfq15/results
<Vivienne> Sorry I haven't been too active with travelling, will get the surveys done this week
Results: Seven answers, so far. Can try to discuss on here.
First change to title. Two--accepted change 4--keep 1: do not accept for following reasons. Don't understand what we are asking evaluator to say?
Detlev: Like the new title. don't need to
enumerate all procedures using, concentrate on techniques for testing
particular areas that owner has new method
... Change the title to "Define Evaluation Procedures for particulal success
criteria.
EV: Like two reasons you gave. Retweak the
re-write for comment.
... Comments?
... Think it would be more clear.
<Vivienne> +1
<martijnhoutepen> +1
K: Fine.
<Liz> change to new title
L: Fine.
M: Fine.
<ericvelleman> +1
EV: Will change title to Detlev's suggestion.
Det: Make them go well with text. Keep it short.
EV: Important to add D's two points. Will add to the text.
D: Easier to draft those things when they're in the text.
Ev: Add to comments field or list?
<Vivienne> +1
+1
<ericvelleman> +1
<Liz> +1
K; Yep
<martijnhoutepen> +1 for adding two points from detlevs comment
D: ? Would object. Need for more enumeration. Site owner has to describe what is needed.
<Vivienne> Mike, I think Detlev said Alistair would object
EV: LIke Richard's comment. That evaluator might have own way fo doing things.
M: Thanks Vivienne.
EV: Will send around for further comment.
... Can you change your decision?
<Detlev> I think "Describe Evaluation Procedures for particular Success Criteria" is better vs. "Define Evaluation Procedures for particular Success Criteria"
<Detlev> Richards suggestion
EV: Would be nice, not have to do another
survey.
... Anything else on this one?
<ericvelleman> https://www.w3.org/2002/09/wbs/48225/evaltfq16
EV: Rewrite of 1C.
Six answers so far. Really difficult one.
Two questions: Current text and Guidance for baseline.
Ev: Current text: 1 Add qeustions: 4
D: Discussing second question.
EV: Stick this first. Survey #16 first.
... Can come back to #15.
D: Pointless to list all adaptive technologies, browsers, etc. "When a particular choice of user agents and assistive technologies is specified by the site owner to achieve conformance and these technologies are known to the evaluator, they should be listed here."
Same as what Richard said.
EV: Like D's proposal. See this as restricted lsit.
D: Similar to what Michael said. Make it clear
that you do this step when reviewing a particular site that is tuned to
specific browsers and tech.
... Main thiing to avoid confusion that don't have to list every conceivable
combination of browsers and tech.
EV: Good Change?
<Liz> Yes
<ericvelleman> +1
<Vivienne> yes, the change is good - I just recorded my answers for this question
<martijnhoutepen> +1
K: Feeling a little lost. Have to look at closer.
+1
EV: Will come up with rewrite with this
proposal.
... Main thing is that we discussed a list of questions. Conclude that instead
of list cover it with this change?
+!
+1
<Detlev> +1
<Vivienne> +1
<Liz> +1
<ericvelleman> +1
<martijnhoutepen> +1
<Tim> +1
Never too quick!
Ev: BAck to 15. Second question.
<ericvelleman> https://www.w3.org/2002/09/wbs/48225/evaltfq15/results
Rewrite of 1d.
EV: will rewrite and add D's two points.
... Do not accept by Richard. I am not happy with "This step is optional
because WCAG 2.0 conformance can be determined without the use of documented
evaluation procedures". I think that, when reporting an evaluation, it is
important to explain how it was done in order to give the recipient confidence
that the work was carried out properly. I can live with this step being
optional but do not like the implication that it is unecessary.
... Don't want to make it seem as unnecessary.
... D: Talks about other techniques but not custom. Just a technicality.
D: Pointless to list all adaptive technologies,
browsers, etc. "When a particular choice of user agents and assistive
technologies is specified by the site owner to achieve conformance and these
technologies are known to the evaluator, they should be listed here."
... A good practice to refer to documented procedures.
EV: would look into the text. Would add a lot of bulk.
D: Not really clear what is meant.
EV: Trying to make clearer. Not sure what we
meant.
... Now looks like have to do all the work all the time.
D: Similar to earlier question where specify
particular circumstances.
... 1D
... Maybe can be merged.
EV: Will come up with another text proposal.
... and send around.
... Covered 1 D and 1C.
<ericvelleman> https://www.w3.org/2002/09/wbs/48225/evaltfq17/results
EV: Also on later section: scoring. #16--5D
... Lots of previous discussion. One question always comes back: no one agrees
on it. No widely used metric, lots of disclaimers.
... One outcome, let's look at metrics document and see if we can come up with
a few examples, with same disclaimers.
... Problem is looks like we're still proposing scoring. My suggestion is to
keep the first paragraph and delete teh rest.
... Seven answered: 6 accept. 1 accept with changes. Vivian says does not meet
need for metric, but okay for now.
... Frederick suggests that research is still going on, not yet closed.
+!
V: Happy with F's suggestion about adding
note.
... As organization we're really looking for help with this. But it's a
controversial subject. Can add something that there is ongoing research.
+1
D: Two minds. Try to keep methodolgy short. If we don't have advice will use whatever they use. But don't mind that.
F: I think it needs to be mentioned. Can be a
note. People are looking for it.
... should be indicated on document that it's being addressed.
<Liz> +1
M: Agree. Keep link to page with metrics document.
<ericvelleman> +1
<Vivienne> +1
<martijnhoutepen> +1
EV: Add ongoing research and keep link (it is
previously in document).
... will lead to changes to current survey. Good work today.
... Surveys open until Tuesday, so still have opportunity to respond. Will
make changes to a few, another rewrite and propose again. For next week would
like to try to make editor draft of new document.
... Start with re-writes we've agreed to. Will start with latest batch and
mark for review. Hope to have by Tuesday. If you have oppty on Wed or Thur have
a look. Will keep it open. Work on this and next week.
... Will mark them all. So you can comment on them during telco.
<Vivienne> that's fine thanks
<Detlev> fine
<martijnhoutepen> +1
EV: Okay?
<Liz> +1
+1
<Tim> +1
<Detlev> -q
EV: Results of test run survey...answered by a number of people.. but anything to say Martijn
M: Yes. Interesting feedback. Tried to put on
mailing list. Baseline very different.
... With coming re-writes may get to them. People found guidance easy,
comments positive. We like that. Some confusion about exploring the website.
Order in which you explore different from pages you found for sample.
Confusing.
... Also, minor text issues. Processes have different understanding of: urls,
explain processes a bit more...some ahve user processes, some have website
processes. What users can do vs. what can be done on website.
... Comments on that also. Can define more clearly.
EV: Step 2 order in disposition of comments already. People generally enthusiastic about methodology. Not everyone finished entire questionnaire. Not everyone had enough time. Maybe something needs to be done to ensure confidence in using our method.
M: Not in dispositioin of comments since they didn't get to end of questionnaire.
EV: Will do a mapping of the comments in survey to disposition of comments. Will cover every comment from public and test run.
M: Around 10 gave results we can use, five finished questionnaire.
EV: Any comments? Other than hundreds we expected.
ME: Any surprises?
M: People found easy to use. No, big surprises. Were some big differences in how much time they took to make sample. But overall people generally agreed with it. Results as expected to me.
EV: Sounds good.
... Leads us to end of telco. Anything else?
K: Can you send me copies of questions?
EV: Yes.
... Have good days, nights, etc. Please find time Wednesday and Thursday to
review revisions.