WAI Authoring Tool Guidelines Working Group
Chair: Jutta Treviranus
Date: Tuesday 8 August 2000
Time: 2:30pm - 4:00pm Boston time (1830Z - 2000Z)
Phone number: Tobin Bridge, +1 (617) 252 7000
The Latest Draft is the Recommendation dated 3 February, available at http://www.w3.org/TR/2000/REC-ATAG10-20000203. The latest techniques draft is dated 4 May March, available at http://www.w3.org/WAI/AU/WD-ATAG10-TECHS-20000504. The latest draft of the Accessibiltiy Evaluation and Repair Techniques is dated 15 March at http://www.w3.org/WAI/ER/IG/ert-20000315
JT First question is whether we create the process with different streams or views? e.g. by type of tool. This would allow for more compact evaluation.
CMN It is a significant improvement in ease of use, rather than a primary requirement
JT It is oimportant to consider that as a goal if we want it at some point.
WL I am going to be dragged kicking and screaming into the idea of a template. It uses energy that could be used reviewing.
JR We are not going to do all the reviews. It would be nice to have a template where other people can happily do at least their own preliminary reviews and come up with the same kind of answers
WL I don't see these things working that well. But I could be wrong.
JT It is clearly difficult to do reviews, so this is probably an important part of making it possible for people to do it.
JT What do we want to use as primary structure and secondary structure. Given the number of relative checkpoints (and guideline 7 is an entity in itself) do we want to structure it by WCAG, or via ATAG guideline order?
JR I would prefer following the ATAG order. When WCAAG goes to version 2 it would be nice if that was secondary rather than primary.
CMN In some cases a single test will answer several questions. So running through in ATAG order, but in some cases answers will be filled in in advance, based on earlier tests.
JT If we take this process through the evaluation it will be the evluator who will have to determine that the questions have been addressed.
CMN This is a mior thing, but a neat optimisation when it does occur
JT In going through an authring tool evaluation, if you are looking at the functionality for creating a table (for example) there are a number of different relevant questions that come up in one context. So the question came up should we do it around the WCAG classification?
CMN We may fnd that we use neither order strictly to cope with this (which is why I want to get this testing thing out and start talking about it :-(
JT I can se the value of sticking to ATAG, but from a practical point of view it is nice not to have to return to a function over again instead of dealing with it once.
CMN I agree.
HS Any ordering that helps increase the performance I am in favour of. The order in the guidelines makes sense for product groups to set out priorities and planning, but performing an evaluation you want ti to be as easy and quick as possible
CMN, JR agree.
JT Possibly the report that is generated may produce different kind of reports with different orders.
JR The evaluation might not strictly be presented as checkpoints, but as tasks with questions about them.
WL It rtakes a while to know a tool well enough to evaluate it.
CMN True. But that isn't really a problem - there are people and developers who know the product really well.
MK You need to find people who are comfortable with a product to evaluate it. Apart from developers who will be biased we need actual users who know the product already.
WL I don' think the bias is nearly as significant as the fact that the act of doing this will be helpful. A developer is not trying to disguise what is happening, but finding out what is missing and fix it.
MK That is two different kinds of evaluation
JT Part of what we want in the report is who did the evaluation.
MK Apart from problems like can you install it in the first place. The pespective of a user can be different to that of a developer.
JT A developer depsite best intentions may think something is obvious when it isn't to users - that happens often.
WL The big thing is that the developer is probably going to be the most skilled user
MK Not necessarily. Especiallly where a tool is built by a team - there may be people who know their bit well without an overall view.
JT Next question was whether to assess and see if it fails P1 and then stop, or to do a full assessment for all checkpoints. Should there be an express evaluation?
HS I don't think so. I think it is valuable to say where it failed and didn't
WL I think the typical thing is Webwatch - often a site will be stated to be accessible in certain areas or ways even with a lot of problems. That will be the way it is for most of the forseeable future so it is useful
JR it is a bit of a time constraint - I didn't have time to do a full review and it was so bad there wasn't a lot of point in going into it in great detail
CMN We would prefer to have full reviews. Whether we get them or not is a different question
JR Right. These things are difficult and take a lot of time. So our choice of tools is a little arbitrary. Are we working from most popular to least poopular tools?
JT We haven't got a method of choosing yet. At the moment we are loking at the process, and going with things we feel we can evaluate now.
JR What if we send the thing out to developers.
CMN There are three groups that can help. Developers, User groups, and reviewers.
WL Should this come out with RDF
CMN I would love it to do so. I have been talking to Wendy about this
JT What kind of reports do we want to generate? There are ways we can do comparison charts for consumers, reports for developers, ...
HS I am not sure how the consumer and developer are different in their requirements.
JT In a report for a developer it is useful to have ways to fix the problem, that is often not useful to a consumer - they just want a comparison to make a purchasing decision on. I wuld see a developer report being a bit more verbose
MK Apart from comparing tools, consumer reports would be useful to have workarounds for problem areas.
CMN For example what things you can add to make a tool do whatever it doesn't yet.
CMN We have talked about this a little in ER recently.
WL Have we mentioned script authoring tool.
CMN What I would call a supported programming envronment?
WL Yeah, sure.
MK There are any number of languages you can be using.
WL If someone produces a javascript generator can they assess it against ATAG?
CMN In principle yes. It would be interesting to see a real assessment.
JT Any more thoughts?
CMN The more information we collected the better we can talk about that topic
JT There are some questions that are determined by the uses for the answers so we want to keep that in mind even if it is premature
JT Last question I had is ranking and scoring tests.
CMN We clearly want more than A, double-A, triple-A available. THe next obvious one is checkpoint by checkpoint.
JR So say you have several different partial evaluations. How will the different opinions be weighted?
CMN The scheme I ahve in mind is to be able to identify each test performed, and who did it. In general I think the aggregate of test done is likely to be reliable, but it may be useful to be able to remove a particular set of results
WL I don't think we are going to be flooded with information
JT The point is how much granularity do we want? Do I want specific information about to what extent checkpoint 3.1 is met?
CMN It is possible to compare against profiles - disability requirements, authoring preferences, etc.
JR How do people put an icon on their box.
CMN There is no conformance enforcement or certification at the moment. If someone makes an obviously false claim we pillory themm in public. But Karl is working in that area, and I think by the time the problem is able to arise he will have thought about it.
JT We obviously want granularity - for relative checkpoints we at least want to know which pieces are met or not.
CMN To the extent that we can break things down to lots of yes/no questions that is easier to deal with.
JT I think there are people whoare intersested in doing the work.
JT Our evaluation process is going to be one of Karl's practise areas, so we are having a conference call with him in the next couple of days. We will let you know what is happening.
JT We have talked about the afternoon (Friday) with the WCAG - the rpoposal for saturday is to break out into working groups to work on fleshing out techniques and look at test processes. How do people feel about that.
Action CMN/JT Send preliminary list of tasks to list
Last Modified $Date: 2000/11/08 08:13:13 $