21 Dec 2005


See also: IRC log


Shadi, Johannes, Carlos, Chris, Jim, Charles, Nick


Test Suite Development

SAZ: Last meeting for the year.
... talked about test suite development on previous call

<shadi> http://lists.w3.org/Archives/Public/public-wai-ert/2005Dec/0024

SAZ: brainstorming with Wendy on WCAG and how to coordinate. Interest in ERT doing test suites but not getting into guidelines discussion.

<shadi> http://www.w3.org/TR/WCAG20-HTML-TECHS/#H45

SAZ: Proposal on the table is to take the HTML techniques that WCAG think is done, and we fill in the gaps. E.g. current HTML techniques doc phase 1
... H45 - using alt attributes is the framework we inherit from WCAG. Where the tests stuff is there would ideally be positive and negative tests - something that we could fill in. That would have work in parallel - test suites developed here in ERT. Who is intersted here?

CV: We are going to make public a test suite of XHTML for WCAG 2. Maybe that is a basis to be used

SAZ: Yes, there is a lot of work outside. Chris Ridpath also has some. THe idea would be to use those as inputs and we would go through and open the discussion and make sure that the test reflects testing for the technique.

CV: This is just a potential basis. There are many open issues...
... this stuff will be in the public domain, so it is just a collection that will be available. Can send more details later.

SAZ: That would be helpful. Need to know what questions are open, and need to know that we have some way of getting answers from WCAG
... are you developing for stuff other than XHTML?

CV: Yes, will be doing XHTML2, SVG, Xforms.

SAZ: And XHTML is ready now?

CV: Yes, more or less - we have the cases, we are starting on testing in January.
... so there may be some changes but not really big ones. Next test suite is Xforms.

CMN: I was under the impression that WCAG has people doing this - needs to be done in one place.
... the idea that ERT takes existing stuff where available seems like a good approach.
... We should be focusing our development of new tests where there are none available at the moment.

SAZ: Was a task force in WCAG for techniques. Has been disbanded and now they have 3 task forces on techniques rather than testing level.
... there is a lot of intereaction there between guidelines and techniques, but the testing level isn't really being covered there.
... so either ERT people go to WCAG and do it there, or we do it here in ERT.
... having it in one place is noted as important. Flip side is that if all this is in WCAG there is a lot of other stuff going on - high signal-noise ratio for people who are working only on testing, and would increase it for people working at guidelines level.
... whatever gets submitted needs to go through rigourous testing

CR: Important that test suite be developed in conjunction with group to make sure that the test suite reflects what the guidelines really mean. Test suite should be usable with other guidelines as well (e.g. WCAG 1, 508, Legge Stanca, ...)
... there is a formal process for submitting techniques to WCAG, and you can submit a test to go along with it.

CV: my concern is that this doesn't overwhelm teh working group while we are working on EARL documents. Test suites is very intensive work.
... if WCAG is a moving target that could increase the resource requirement

<Zakim> chaals, you wanted to say it sounds like a good idea to do it in ERT, given that there is a commitment from WCAG to answer questions reasonably fast.

CMN: Like the idea of multiple applicability for tests but think we should be setting specific targets (e.g. WCAG 2) as priority. It is also important to make sure that this doesn't break EARL

SAZ: EARL is important, but think we all agree on teh importance of getting WCAG 2 out and having testing criteria
... how many people are intersted in putting extra resources in to extend the work capacity of the group? Extend the telecon, or break into task forces, although I think this should be done in the main group.

<niq> or an IRC conference - easier to follow than teleconference IMO

<ChrisR> I'm interested in putting time into test suite.

CMN: Opera thinks WCAG 2 is nowhere near ready for Last call, let alone going further, so don't see the need to rush into doing it. Suggest that we allocate a fixed amount of telephone time (15 minutes), as an ongoing background activity.

SAZ: Suggest taking on some areas where there are ongoing discussions, to help out WCAG - moving target is a problem, of course, but that is part of the nature of work in progress
... not sure I understand your perspective on teh value of test suites. This could be reusable for the Mobile Web Initiative.
... How relevant is this to MWI?

CR: There has already been a lot of work done on test suite in WCAG 2 - most guidelines / success criteria have some tests and techniques, and you can see in the archives what has been happening. The real importance is if we find tests that affect accessibility and there are not guidelines in WCAG 2 that cover this then that is important feedback.

<niq> skype died and now gives internal error on every attempt

CR: important to be looking at things that are ground-up, to catch anything that isn't reflected in the guidelines development process.

CMN: Agree that tests are important, and can in some cases be re-used by MWI - should be coordinated with them. But think that EARL is more imoprtant to this group than WCAG 2 and do not want to see this group be sidetracked, so want limits to commitment.

SAZ: Hear the point about limiting the resource commitment when WCAG 2 is a moving target. Does it help to know that we ae going to concentrate only on things that the WCAG group says are stable?

<carlos> I agree with Nick

CMN: No. The only thing that helps me is to have a fixed limit on the resource commitment - 20 minutes, or 30 minutes in a 90 minute call.

CR: I'm OK with that.

SAZ: Want to get back to the relationship between test suites and guidelines - going into guidelines development because we have questions originating from tests is a resource monster. We should be careful about the scope and make sure we push back stuff to the WCAG group...

NIQ: [Think test suite is good, not sure on role of test suite]

CV: Development process is mostly developing language to describe test files, not the test files themselves.
... at the beginning it was a complete nightmare - 1.5 intensive man-years has not been enough.
... I would be scared if we tried to take that on.

CR: Maybe don't want to develop the test suite as part of the group. Rather than making up the tests, we just collect existing tests and "approve" them.
... or not, for use with a given technique / success criterion.

CV: Is there a CVS system in W3C that we can use?

CR: Yes

<niq> hmmm, any thoughts of upgrading cvs to svn?

CMN: My experience both inside and outside W3C has been that what we need is available

SAZ: Lot of people were interested, now I am hearing more scepticism.
... let's put the question - who would be interested in spending extra telecon time, extra work time on test suites.

<ChrisR> is interested

CMN would be prepared to look for some extra resource time.

<niq> yes

<JibberJim> I'm interested in principle but would want to be sure WCAG 2.0 is ready

CV not for the moment - busy working on our own test suites and Xforms - no spare resources.

CV: We prefer to concentrate on EARL

SAZ: Carlos, you are happy to submit your test suites as input?

<carlos> yes

CV: No problem at all - it will all be public domain.

SAZ: Your experience would be valuable.
... couple of people missing on the call. And we also need to look at resources inside WAI - Wendy will be on maternity leave in a few days

EARL 1.0 Guide

SAZ: Thanks to Carlos / Johannes for putting this together. EARLy draft - hope there are a lot of comments.
... would like to step back and think about audience.
... ahve been doing some of this work in EO.
... any overall comments?

<niq> audience starts with tool developers like ChrisR / JibberJim / niq

CMN: There is no actual content about EARL. There is a guide I wrote a while ago that could be cannibilised...

SAZ: Do we want to describe about EARL, or the benefits and so on
... rather than describing the classes and so on.

<niq> brevity++

CM: Think the Schema should be as brief as possible, the guide should be oriented to a practical explanation of how it works and how to use it, but broad overview stuff about what it is good for starts to get to what EO should be doing

<carlos> no problem

CV: There is a different audience - they get into the classes when the schema is updated but there are a lot of changes to come from the face to face and it is hard to now what the schema looks like now

SAZ: Apologise because CMN and SAZ have not had time to get this together

CV: Should be doing guidance on the classes. I know the guide from chaals that can be cannibilised

<niq> hmmm

SAZ: Should we assume people know RDF when reading the guide?

CV: hope so

<niq> an assumption of rdf could be an additional hurdle

CMN: Think we should not assume it - explain the key impacts like syntax flexibility, inheritance subclasses etc, and then poiint them to RDF material if they really want the detail.

SAZ: Think primary audience is tool developers who want to learn about EARL. Assume they know XML, not RDF, although they are capable of understanding it.
... should explain the things they need to know about RDF.
... do people think the primary audience are tool developers who want to know how it works?

<niq> rdf-as-xml gets confusing if you *think* xml. That's a hurdle.

CMN: Makes sense to me. Audience for schema is me - I know RDF, I know what EARL does, I just need to know the classes, what they are meant for, what the restrictions are, so I can implement it intelligently

CV: Niq's point is why I used pseudo-n3 for first example

SAZ: Arc and node diagrams can also be helpful

CMN: Looking for an RDF syntax people all love is like chasing the holy grail - we will never get there. Agree that a variety of representations of examples is helpful for people to get the picture, but should follow W3C Recommendation which is RDF/XML, and make sure that is well explained.

SAZ: Let's look at table of contents

<shadi> http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Guide-20051214

SAZ: Are there other things we need?

CMN: Table of contents seems to point to the right set of information - just need to see it...

SAZ: Doesn't sound like there are major issues. Need to get Schema published so guide can follow it.
... talk to you all next year. First meeting Wed 11 jan (or wednesday closest to then.

Summary of Action Items

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.127 (CVS log)
$Date: 2005/12/21 16:17:50 $