JB - introduced the evaluation suite
SAZ - described the recent work he has been doing on the "Selecting Web
Accessibility Evaluation Tools"
[http://www.w3.org/WAI/ER/existingtools.html]. Asked for General
reactions.
WD - suggests general report interface be the ?
NL - likes information that puts the process in the wider evaluation process,
but would like more information about what each tool does (in the Tools list
section)
SAZ - in ERT we are looking at how to present the tools list document
[http://www.w3.org/WAI/ER/existingtools.html]
JB - has some concern about scope of work necessary to more fully describe
the tools.
Maybe have another category: are there other rules or national standards you
want the tools to check for other than WCAG.
HBj - also some do checks for usability as well and that would be out of
scope that could be added to the "checks other things" list (or other
features).
SLH - why even mention 508?
WD - has a class of senior computer students and he was planning to turn them
loose on the evaluation tool list and has volunteered the class to do this as
a project.
AA - accessibility is more than just the checkpoints? things like usability
that are important.
SAZ - "additional guidelines" or "additional tests"
SLH - so are we talking about adding a general category?
NL - Yes, I would like to know what else the tools check for.
JB - Natasha, what do you want, and where do you want it? Because what we are
talking specifically about is adding a paragraph under a new heading in
section 3.
NL - yes that is OK.
JB - not wanting to see this all tabularized - difficult to do.
Is the list in section 3 overwhelming? There are 14 or 15 items listed now.
Will people be frustrated at getting so many questions we don't give answers
to.
HBj: a bit overwhelming? maybe sort by tools that are useful to user
perspective.
SAZ: might be difficult to decide what is appropriate to specific groups.
Thinks section 2 is necessary at a high level to describe the types of tools,
rough categories - then section 3 follows to refine the users needs.
JB: any other general comments about the document.
HS: His group asks: how do you evaluate web site? do you use evaluation
tools? are the tools certified? Etc. but we don't use tools to certify
compliance - except as checks (primarily for code validation). Perhaps we
need to modify the introduction. Will come back to this later in the suite
discussion.
JB: maybe in the "how evaluation tools can not help you".
JB - not just useful for "conformance evaluation", but for preliminary
evaluation as well. Say "when evaluating a web site against the checkpoint"
and remove reference to "conformance" since that makes it too specific.
[Shadi thinking, but not puzzled]
SAZ: thinks prelim reviews are so different from conformance that we are
always trying to bridge the gap. He thinks that tools are more often used in
conformance than prelim and might confuse people by having prelim reviewers
thinking they need tools.
JB: thinks we shouldn't hint that people should only use tools in
conformance.
NL: Not clear to her that she can use the evaluation tools during the
design/development of a web site. Should add something about how tools can
help in the development cycle.
HS: we are talking about evaluation tools but and say can be used in
development or design in the process.
CC: but you are still evaluating the accessibility of your development.
SAZ: but we are not really talking about development tools (authoring tools),
even though some eval tools may feed into authoring tools.
[Some style comments: look for "you"]
JT: should 'should' become 'can'?
SLH: Need to change the section title: how about "what evaluation tools
can not do"
JB: need to change the negativity of the paragraph: "inaccuracy" could be
"limitations"; "evaluation tools have different strengths and weaknesses".
The "misleading" sentence becomes difficult. Instead, something like "false
negatives or positives" if that is not too jargony or "misleading
results".
NL: the first sentence covers a lot.
HS: there is a difference between saying the tool is misleading or that the
tool's results are misleading. Would like to see something about how using
tools in conformance evaluations but should not be relied upon to certify
accessibility. Use them as supports to evaluation.
JB: "At this time there are no 'certified' evaluation tools and tools by
themselves cannot completely? NL added "not evaluate a site to the level of
qualifying for certification."
SAZ: we have a nice sentence in another section (that he can't find at the
moment).
JB: and finish off with "? because, for instance, not all checkpoints can be
evaluated automatically."
SAZ: thinks we should avoid term "Certification".
HS: agrees that certification can be avoided.
[Group quietly sung happy birthday to Andrew.]
[No comments]
SAZ: there will be an editorial change in tone on this section. Thinks Natasha's comment about elaborating on how you select tools depending on the phase of development life cycle/organization/type of web site, would be useful here.
HS - thinks the short titles (here and in rest of section) are too
cryptic.
AA: "Report generating tools" and editorially fix other titles to reflect
slightly longer, slightly more descriptive tools.
[need a synonym or explanation] [guided interface, aided interface,
???]
[No other comments]
AA: some of these tools highlight the appropriate use of markup as well as
only highlighting errors.
JB: this kind of tool is difficult to explain if you haven't ever seen one in
action.
SLH: why not put a picture in the document that shows it.
JB: thinks doing images is very hard to get agreement on? add such to the
wish list. Also might lead to questions about why we don't do this for other
sections.
SLH: thinks it is easy to do (e.g. screen capture) and minimal discussion.
[consensus: on wish list]
SLH: what is future? Thinks this should be done soon.
HS: what does the last part of the last sentence mean?
[Some discussion about the phrase in and out of context, which led to some
discussion of the meaning of the entire paragraph and whether it belongs in
this section.]
[Some confusion about what these tools are, and do, and are they
accessibility aids or just usability aids.]
JB - how about changing order of the paragraph: first you say what they do,
then say how they help you. Make sure the headings work - what would this one
become?
JB: Overall, remember to scan for Jargon, e.g. first paragraph in section 2.
JB: thinks that Natasha's comment about org type, dev cycle, etc. should
be rephrased in first paragraph of section 3.
WD: another sentence about "you might need more than one tool"?
SAZ: already appears in section 1 and 2.
JB: thinks it bears repeating in this section too.
JW: concerned that the questions being asked don't help him answer the
question about how to rate the importance of each to his organization.
JB: difficult to predict how each might relate to any specific organization.
NL: what is the strategic implications for choosing such tools? could we add
something about it.
JW: thinks it would be easier to categorize by function rather than
organization (and NL agrees). We aren't told what we need.
JB: what about a bulleted list in the new section that itemizes in general
what the needs might be? Should fit in proposed new section "strategies for
selecting tools"
SAZ: What about saying in each paragraph of section 2 what each tool might be
best used to accomplish - what strategic or development cycle phase they best
apply to.
AA: ok, but introduce the concept earlier and include these as examples in
section 2.
JB: concerned that the mapping between needs and tools is not always
clear.
SAZ: but how do we relate the roles in section 2 to the tools in section
3.
NL: she would reiterate the needs in relation to each tool.
[no consensus on building needs into every tool in section 3, but yes in
section 2]
JB: please think about this document as being almost finished, and try to
avoid changes that would cause a redraft the entire document. Editor's
discretion on some of the substantive changes.