JB - introduced the evaluation suite
SAZ - described the recent work he has been doing on the "Selecting Web Accessibility Evaluation Tools" [http://www.w3.org/WAI/ER/existingtools.html]. Asked for General reactions.
WD - suggests general report interface be the ?
NL - likes information that puts the process in the wider evaluation process, but would like more information about what each tool does (in the Tools list section)
SAZ - in ERT we are looking at how to present the tools list document [http://www.w3.org/WAI/ER/existingtools.html]
JB - has some concern about scope of work necessary to more fully describe the tools.
Maybe have another category: are there other rules or national standards you want the tools to check for other than WCAG.
HBj - also some do checks for usability as well and that would be out of scope that could be added to the "checks other things" list (or other features).
SLH - why even mention 508?
WD - has a class of senior computer students and he was planning to turn them loose on the evaluation tool list and has volunteered the class to do this as a project.
AA - accessibility is more than just the checkpoints? things like usability that are important.
SAZ - "additional guidelines" or "additional tests"
SLH - so are we talking about adding a general category?
NL - Yes, I would like to know what else the tools check for.
JB - Natasha, what do you want, and where do you want it? Because what we are talking specifically about is adding a paragraph under a new heading in section 3.
NL - yes that is OK.
JB - not wanting to see this all tabularized - difficult to do.
Is the list in section 3 overwhelming? There are 14 or 15 items listed now. Will people be frustrated at getting so many questions we don't give answers to.
HBj: a bit overwhelming? maybe sort by tools that are useful to user perspective.
SAZ: might be difficult to decide what is appropriate to specific groups. Thinks section 2 is necessary at a high level to describe the types of tools, rough categories - then section 3 follows to refine the users needs.
JB: any other general comments about the document.
HS: His group asks: how do you evaluate web site? do you use evaluation tools? are the tools certified? Etc. but we don't use tools to certify compliance - except as checks (primarily for code validation). Perhaps we need to modify the introduction. Will come back to this later in the suite discussion.
JB: maybe in the "how evaluation tools can not help you".
JB - not just useful for "conformance evaluation", but for preliminary
evaluation as well. Say "when evaluating a web site against the checkpoint"
and remove reference to "conformance" since that makes it too specific.
[Shadi thinking, but not puzzled]
SAZ: thinks prelim reviews are so different from conformance that we are always trying to bridge the gap. He thinks that tools are more often used in conformance than prelim and might confuse people by having prelim reviewers thinking they need tools.
JB: thinks we shouldn't hint that people should only use tools in conformance.
NL: Not clear to her that she can use the evaluation tools during the
design/development of a web site. Should add something about how tools can
help in the development cycle.
HS: we are talking about evaluation tools but and say can be used in development or design in the process.
CC: but you are still evaluating the accessibility of your development.
SAZ: but we are not really talking about development tools (authoring tools), even though some eval tools may feed into authoring tools.
[Some style comments: look for "you"]
JT: should 'should' become 'can'?
SLH: Need to change the section title: how about "what evaluation tools
can not do"
JB: need to change the negativity of the paragraph: "inaccuracy" could be "limitations"; "evaluation tools have different strengths and weaknesses". The "misleading" sentence becomes difficult. Instead, something like "false negatives or positives" if that is not too jargony or "misleading results".
NL: the first sentence covers a lot.
HS: there is a difference between saying the tool is misleading or that the tool's results are misleading. Would like to see something about how using tools in conformance evaluations but should not be relied upon to certify accessibility. Use them as supports to evaluation.
JB: "At this time there are no 'certified' evaluation tools and tools by themselves cannot completely? NL added "not evaluate a site to the level of qualifying for certification."
SAZ: we have a nice sentence in another section (that he can't find at the moment).
JB: and finish off with "? because, for instance, not all checkpoints can be evaluated automatically."
SAZ: thinks we should avoid term "Certification".
HS: agrees that certification can be avoided.
[Group quietly sung happy birthday to Andrew.]
SAZ: there will be an editorial change in tone on this section. Thinks Natasha's comment about elaborating on how you select tools depending on the phase of development life cycle/organization/type of web site, would be useful here.
HS - thinks the short titles (here and in rest of section) are too
AA: "Report generating tools" and editorially fix other titles to reflect slightly longer, slightly more descriptive tools.
[need a synonym or explanation] [guided interface, aided interface,
[No other comments]
AA: some of these tools highlight the appropriate use of markup as well as
only highlighting errors.
JB: this kind of tool is difficult to explain if you haven't ever seen one in action.
SLH: why not put a picture in the document that shows it.
JB: thinks doing images is very hard to get agreement on? add such to the wish list. Also might lead to questions about why we don't do this for other sections.
SLH: thinks it is easy to do (e.g. screen capture) and minimal discussion.
[consensus: on wish list]
SLH: what is future? Thinks this should be done soon.
HS: what does the last part of the last sentence mean?
[Some discussion about the phrase in and out of context, which led to some discussion of the meaning of the entire paragraph and whether it belongs in this section.]
[Some confusion about what these tools are, and do, and are they accessibility aids or just usability aids.]
JB - how about changing order of the paragraph: first you say what they do, then say how they help you. Make sure the headings work - what would this one become?
JB: Overall, remember to scan for Jargon, e.g. first paragraph in section 2.
JB: thinks that Natasha's comment about org type, dev cycle, etc. should
be rephrased in first paragraph of section 3.
WD: another sentence about "you might need more than one tool"?
SAZ: already appears in section 1 and 2.
JB: thinks it bears repeating in this section too.
JW: concerned that the questions being asked don't help him answer the
question about how to rate the importance of each to his organization.
JB: difficult to predict how each might relate to any specific organization.
NL: what is the strategic implications for choosing such tools? could we add something about it.
JW: thinks it would be easier to categorize by function rather than organization (and NL agrees). We aren't told what we need.
JB: what about a bulleted list in the new section that itemizes in general what the needs might be? Should fit in proposed new section "strategies for selecting tools"
SAZ: What about saying in each paragraph of section 2 what each tool might be best used to accomplish - what strategic or development cycle phase they best apply to.
AA: ok, but introduce the concept earlier and include these as examples in section 2.
JB: concerned that the mapping between needs and tools is not always clear.
SAZ: but how do we relate the roles in section 2 to the tools in section 3.
NL: she would reiterate the needs in relation to each tool.
[no consensus on building needs into every tool in section 3, but yes in section 2]
JB: please think about this document as being almost finished, and try to avoid changes that would cause a redraft the entire document. Editor's discretion on some of the substantive changes.