AU Teleconference (Monday, 17 June 2002)

Telephone: +1.617.761.6200
Conference number: 2894

Agenda:

  1. Face to Face Meeting Planning
    http://www.icchp.at
  2. Evaluation Technique Issues
    http://lists.w3.org/Archives/Public/w3c-wai-au/2002AprJun/0039.html
  3. Next steps for the evaluation template
    http://ultrajuan.ic.utoronto.ca/~jan/WAI/ConfEval_Template.html

Attendees:

PJ: Phill Jenkins
JT: Jutta Treviranus
JR: Jan Richards
MM: Matt May
DG: Doug Grude
HS: Heather Swayne
CV: Carlos Velasco

Regrets:

LN: Liddy Nevile

Minutes:

AUWG conference call, 6/17, 12pm

Call times

JT: proposes an alternative time of 4pm ET Mondays
HS: Thursdays best, but can make 1pm PT Mon-Fri
CV: can only make some at 10pm European time
PJ: alternating times? Possible with the bridge? IBM alternates with early morning or early evening to be more convenient in various places
HS: What would be the alternative time?
JT: 8am ET, 5am PT
DG: can make early-morning call
HS: afraid I'd be of no value at 5am
MM: How many in Australia, Japan?
JT: 3 in Australia, 2 potentials and Carlos in Europe. People with difficulties in Europe as well, balanced between Europe and Australia
MM: 4pm is the sweet spot for all three zones. Other times good for Europe/Asia are usually the dead of the night in North America
JT: propose 4pm after Linz meetings. Conflicts with Thursday?
MM: Thursday is WCAG WG
DG: Can rearrange schedule
PJ: Occasionally attends WCAG

RESOLVED: Changing calls to 4pm Mondays beginning 29 July


Evaluation technique issues

JT: What form will evaluation techniques take? Step-by-step, or more generic description requiring interpretation by tester? There wasn't full support for step-by-step approach due to how onerous and time-consuming it would be. Judy was supposed to fill us in on the approach.
JR: Sent out evaluation template this morning. I added to 1.1-1.4 and 7.6 to make sure it works. Added section called background, and one called "eval techniques". Background fills users in on the spirit of the guidelines to allow users to use common sense. In techniques, a little step-by-step, but also tries to be somewhat generic. So if we do adopt this, we can make these techniques much more specific. There's room to add an HTML4 test suite, for example.
CV: Some people might interpret "Background" as a replacement for guidelines. They may ignore the rest of the documents.
JT: Fine line: either we can be very specific and have several instances and mechanistic techniques, or rely more on human judgment. There may be a very complex path.
JR: That path could make the guidelines easy to avoid.
PJ: They may do that if they wanted to meet more than the minimum.
JR: But to meet minimum, they'd have to comply with the same steps.
PJ: For some tools, we could state explicitly that certain techniques do not apply. But I agree with CV regarding the state of the Background section. It should say, if you are to comply, read the guideline, then the checkpoint, then the technique.
JT: Should we link to it or reproduce it?
DG: Could things get out of sync if they're reproduced?
JT: I would imagine we would generate it anyway.
PJ: I prefer reproducing the text for new users, but linking for advanced users. Can we do that?
JT: Yes.
PJ: I prefer that.
JT: I like PJ's idea of highlighting what is not applicable and allowing evaluators to move on.
JT: How detailed should the Yes-No-Not Applicable text be?
PJ: If we have some sample HTML files, we should identify those.
JT: Could create some objective criteria: if x is in your tool, you pass or fail. Could have an allusion to the rules, and depend on human judgment. Could list example ways of passing.
JR: Some techniques aren't exclusive: could be met in other ways. Other techniques are developer-centric.
PJ: Read list of techniques. If they meet them, then they pass.
JT: But they can comply in other ways to meet the checkpoint.
JR: Some techniques are very fine.
JT: One approach we looked at was necessary-but-not-sufficient.
CV: We may lose out on some compliance unless there is a complete test suite.
JT: I don't think we're going to come up with that as a working group.
DG: Do we have those resources?
JT: No. But we agree that we should look at getting more resources and get a suite developed. Would have to go through coordination group and possibly apply for outside funds. But it would be a massive task.
PJ: Is this massive task like an example of every spec tag?
JT: This is not like complying to a technical specification. How you assess something that produces QuickTime versus non-WYSIWYG markup tool is very different, for example.
PJ: We have six categories of authoring tools?
JT: Yes, but we have things that may fit here or there, but don't completely fit.
JR: We decided they're functional categories, so they can be mix and match. You'd need for starters one implementation of techniques for every single language. Another group of checkpoints deals with how the tools look. This group is where techniques are different.
MM: Also dependency with WCAG, which isn't producing normative techniques for the same reason.
JT: We should still produce something that will be a stopgap until someone produces a better spec. So, how far can we go as a group? What would be the most useful thing to do? First, we could assume we cannot have the outside effort to do all we want to do. Second, we can look for the outside effort and put out a framework to build toward that.
HS: I vote for the first.
JR: Agree, but I think we should prototype what we want on some techniques, and if we have the outside effort, we can work on top of that.
PJ: On necessary-but-not-sufficient. If we try to specify what only the best thing to do is, we won't have as much impact. People want to know whether we think x meets a given checkpoint.
CV: Maybe call this "preliminary approach" instead of "evaluation test"? Treat this as a starting point. A "quick and dirty" test. Techniques should be in a document like this. I think this document should be more practical than normative.
JT: We would probably want to put those things in the techniques, and make reference to them in this document.
CV: The document might become too big.
JR: For content, we point to WCAG, and they have their own techniques.
JT: We've gone back and forth on marking techniques as only partially satisfying a checkpoint. Should we reconsider that?
JR: I think not. Almost none of them are like that. On Wombat, we talk about having minima.
JT: We should probably have a way to add that to the evaluation techniques.
JR: I think we should say, put it through a test tool, and here's how it comes out.
JT: Either we want to have a test file repository, or we would want to produce the test files ourselves.
PJ: I think we should have common test suites between AU, UA, and GL.
JT: The test files where we're asking the authoring tools to do something different don't match.
PJ: Should be able to come up with techniques that work with AU.
MM: There's still a user issue there. WCAG has issues with user content that require investigation.
PJ: Someone's output from a tool. Users can still create garbage. We should test that it's capable of meeting checkpoint.
JT: Each type of error (tool, agent, content) can require investigation.
JR: A piece on evaluation output for accessibility?
JT: Some are easier to judge objectively than others. Would it be useful to group types of evaluations that can be done, and look into a structure for each one?
PJ: Or we could go through them and come up with one.
JR: There's a spirit to this that goes beyond the main document.
JT: 7.1 is a list of things that need to happen.
JR: We can put in the background that you have to do 7.1 before getting 7.6. If it is guideline-like text, then it should go into Wombat.
PJ: I think we should decide to work either on the current document or Wombat.
JT: Seems to be more motivation to work on Wombat because we know there is a limited life span.
JR: It might be something that needs to be done in parallel. How we do with this document speaks to what we do with the new one.
JT: So the justification for working with both is that we may have further problems?
JR: Should we just work on techniques for the new Wombat?
PJ: Yes.
CV: Agree.
JT: status of WCAG?
MM: Sometime in 2003. Are we meeting them in Linz?
JT: They want a telecon before meeting. Neither of the chairs will be there so there won't be a quorum.
JT: stopgap proposal would be to work on Wombat. And as WCAG moves along, we can address new issues.
(consensus all around.)
JT: We'll continue to pursue this on list. Next meeting? 1 Jul is Canada Day.
PJ: I can have dependency document done by 8 Jul, but not 24 Jun.
JT: Next meeting 8 Jul, coordinating with GL.