Minutes of QA IG F2F at the W3C Tech Plenary - February 2006

The first meeting of the QA IG has been very successful with a lot of interesting discussions and many projects for the future. Patrick Curran, participant of the QA IG, wrote about the meeting on his weblog and conclude by: The Interest Group met yesterday and today to discuss how we should focus our efforts during the coming months. As always we have much more work to do than we have people to do it, so we would very much welcome assistance from people in the conformance testing community. Whether or not you're able to help, please join our mailing list and read our blog.

The QA IG F2F has been held in Cannes Mandelieu, France during the W3C Technical Plenary from Monday 27th of February to Tuesday 28th of February 2006. This meeting was open to public participation in the room, as well on the phone or IRC (channel irc://irc.w3.org/#qa).

Participants

• On site: Tim Boland (NIST), Patrick Curran (Sun Microsystems), Karl Dubost (W3C), Jacques Durand (Fujitsu), Snorre M. Grimsby (Opera), Dominique Hazaël-Massieux (W3C), Olivier Théreaux (W3C)
• On phone: Lofton Henderson (CGM Open), Lynne Rosenthal (NIST)
• On IRC: Björn Höhrmann, Stephanie Troeth (WASP), Holly Marie Koltz (WASP)

Regrets: Mark Skall

1. Test Cases Metadata
2. W3C Glossary system
3. CUAP and CHIPs
4. QA Findings
5. QA Weblog
6. EBNF discussion
7. Test Assertion Guide
8. Organization of QAIG
9. Templates of QA Handbook and SpecGL
10. Tools
11. Curriculum for Web Standards
12. Validators
13. Joint session with ERT Working Group on EARL

Test Cases Metadata

Test Metadata has been published as a W3C Working Group note in September 2005. The document defines a minimal set of metadata elements that can usefully be applied to tests that are intended for publication within a test suite. We published it quickly. At the time of the publication, it was mentionned that a schema could be useful to help implementers to use it in their tools. Dominique Hazaël-Massieux has suggested that we should start an RDF or XML Schema implementation of these metadata. It should be done in coordination with one or two Working Groups and in the context of a practical use. Mobile Best Practices might be one canditate, and ERT WG with EARL might be another possible implementation of this language.

Snorre mentionned that it could be of interests for Opera which has already plenty of test suites that could be released in W3C venues.

We agreed that we have to be flexible, and make this schema as extensible as possible. So that people can not only use it but adapt it to their own needs. The choice of the format for the schema might be a problem in certain venues: RDF, XML Schema, RelaxNG, ... We will then not recommend only one schema. We will start with one implementation and invite other people to contribute their own schema to be added to the document in appendix.

We could expect a republication in April 2006.

• A survey on Test Suite needs at W3C
• Survey of testing practices in W3C groups

• ACTION: Dom to propose an implementation of Test Metadata in RDF. Deadline: Mid-March
• ACTION: Patrick, Tim and Snorre to review Dom'simplementation of Test Metadata in RDF

W3C Glossary system

The W3C Glossary is not a directly QA related topic, but it has interests with regards to the quality of specifications. it helps harmonization of vocabulary accross specifications and maintains regularity. The project has been started 3 years ago. It collects the data from W3C specifications and is RDF based. A new intern will start to work on this project in May 2006. The goals are translations of W3C Terms in different languages. This is useful for volunteers who translate W3C specifications. Optimization of the code, making the code open source and study of relationship (semantic genealogy) between terms are also goals. For example, what would be the variability of definitions of user agents accross W3C specifications.

The use of the W3C Glossary is encouraged on W3C Editors home page.

CUAP and CHIPs

Common User Agent Problems and Common HTTP Implementation Problems are two Notes which have been published a few years ago. They highlight some common implementation issues in user agents and HTTP-based softwares. We discussed about updating and republishing the two notes making then one note synchronized on Web architecture recommendation. Worries have been expressed about merging the two documents. They might not address exactly the same public: User agents versus servers. We will use the Wiki to collect issues from people about identified HTTP implementation problems using the Web Architecture document as a framework for collecting issues. We have to make very clear that it's not a way to collect issues about Web architecture document itself. The new draft will be posted to QA IG mailing-list for comments.

ACTION: Olivier and Karl to provide an updated version of CUAP and CHIPs - due by end of April

ACTION: Snorre will review the document once it has been published.

QA Findings

Tag Findings are short documents dealing with issues about architecture. The format is quite interesting and gathering the knowledge about one particular topic of Web architecture. Dominique Hazaël-Massieux has suggested that it might be possible to publish QA Findings.

For example, in MWI WG, they are writing best practices, some of these practices are not testable. So, they are not normative. An requirement don't have to be testable to be normative, but life become much harder if non testable normative requirements are provided in a specification. Having a summary of all discussions which have been done for this specific topic would be very useful and could serve as a QA Findings.

When a discussion got enough momentum on the www-qa mailing-list, someone will take the responsibility about prototyping the topic on the QA wiki and collect information. Once the document has reached maturity, we might publish it as a QA Finding, probably on the QA Weblog.

ACTION: Dom to start a wiki page on testability/normativity before mid-March and will report on www-qa.

QA Weblog

The QA home page was used to have only small news. After discussion in the QA Team, we thought about developing a Weblog for the QA home page. This is officially owned by the QA IG, although the policies for writing in it aren't well defined yet. We have already started to publish a few articles with open comments (but moderated). We may receive an email or read a news which raises an interesting question to be develop in a longer post. It has been decided to create the QA Weblog, because it doesn't reach necessary the same public than people reading mailing-list or using the wiki.

Entry will be edited for adding references and/or fix mistakes in the article to ensure a good level of quality. The comments being open, it also gives an opportunity to add information to the post. If we identify a discussion that would make a good contribution for the wiki, we might ask the person to prepare a contribution for the weblog. Recently, there was a discussion about EBNF.

The QA Weblog has not been marketed too much, only by word of mouth. We wanted to see how the community would react, but anyone can talk about it or advertize it.

ACTION: karl to see if Björn is interested in preparing a blog entry on the EBNF discussion.

EBNF discussion

There are EBNF discrepancies or inconsistencies among specifications. It has been proposed by Björn Höhrmann to take existing ENBF and add XML namespaces plus errata to avoid further discrepancies - or guarantee consistency. It is definitely clear that is not the job of the QA IG to fix this document, but XML Core WG role. Karl will coordinate with Björn to contact XML Core.

Test Assertion Guide

Jacques Durand has worked inside OASIS on two efforts -- one in scope of EBXML -- markup of test assertions, other is that specs should be more testable. A simple guide to write test assertions has then been a requirement. There is an educational/PR exercise. The document developed at OASIS is right now a rough draft, but could be benefit of being reviewed by W3C and OASIS. It has been mentionned that would be possible only if the document is publicly available. The document could end up as a FAQ to write test assertions.

ACTION Jacques and Lynne: to find out about making the DRAFT guide on writing test assertions public

ACTION: Lynne to develop document summary (review with Jacques) on DRAFT guide on writing test assertions

Organization of QAIG

The only requirement to be a Member of the QA IG, a person has to be subscribed on qa ig mailing list. There are no currently regular teleconferences. We have to define ways of organizing the QA IG and support the QA IG objectives.

Lynne Rosenthal has also voiced the desire to step down from her role as a co-chair of the group. Her time is too limited. Lynne agrees to continue to participate to the work of the QA IG. Patrick Curran has proposed to take her role. We need to ask W3C Management for confirmation.

The two co-chairs will have regular teleconferences every two weeks to move the work forward, review the action items, and find volunteers (participants) to take responsibilty on work.

ACTION: Karl to ask W3C management about change of co-chairs for QA IG

Templates of QA Handbook and SpecGL

There has been no more work done on conformance template, charter template and QA Process document template. Slightly related a mail has been sent about tagging the QA Handbook to make an index to access the document with different views.

ACTION: Karl to finish conformance template.

ACTION: Karl to finish charter template by March 16

ACTION: Lofton to finish process template in next six weeks

ACTION: Tim to index QA Handbook to answer specific questions

Tools

It would be beneficial for the W3C community to develop tools that would help WG members to write specifications and test them. It could be things like specifications markup process, measuring coverage, tools for tests/test harnesses needed, etc. As a first step, we decided to collect on a wiki page, QA Tools, a list of resources and tools.

ACTION: Patrick to update section QA Tools on the wiki - http://esw.w3.org/topic/QaTools

Curriculum for Web Standards

Both the QA IG and the WASP are interested by creating resources for a Curriculum for Web standards. It's a recurring discussion on W3C mailing lists. The WASP Education Task Force has been involved in talking to universities and colleges, bringing out examples of what's been done. The public output has been in the forms of interviews. The WASP is now interested in developing a kind of curriculum framework to help lecturers into their teaching. The QA activity as large has done nothing in this direction yet, but we have taken contacts, like Ed Bilodeau at Mc Gill University, Montreal, Canada to collect materials that would be useful for such a curriculum. Ed Bilodeau has showed interests in contributing to such a work, maybe under the form of a W3C Group Note. This work could be moved forward quite quickly considering that Stephanie Troeth (WASP) and Ed Bilodeau are both in Montreal.

The audience of this framework would be lecturers, college teachers. It would include the time constraints, budget allocations, resources, etc. It has been shown that sometimes lecturers are quite positive about Web standards but they faced resistance through their department, specifically in Web design faculties. The framework will have to give hints on strategies for change. The format of the package has not yet been decided, but it has to be international then we need to understand how universities function around the world.

There is a need for balance between

1. what needs to be taught
2. what can conceivably be taught

There might be resources at Opera to make available for this effort. Opera has 1 full time + future resources part time and then probably full time working on the "open the web" initiative, but without precisely knowing how much of this could be contributed to QA IG. Some Web Standards User groups are forming at key Universities and Colleges to provide and advocate a method for change - WaSP Edu is providing interviews with these as they find out about them.

The Core EduTF at WASP has four to six people. It seems reasonnable to create a join effort with a regular IRC meetings at least, every 2 weeks. We will push the requirements interview phase and decide where to head from there.

The first IRC meeting is planned for March 21. Time to be defined.

Validators

W3C hosts a number of free public tools that allow developers and maintainers of web documents to check them for conformance.

• Markup validator (HTML and XHTML, SVG or MathML)
• CSS validator
• RDF validator
• Feed validator (developed at Sourceforge) - hosted by W3C

See the full list of individual tools.

We're now running into discrepancies between the ways in which the tools work. Markup validator is based on SGML parser (doesn't handle XML very well) but the CSS validator is based on a better XML parser. (So, Markup validator might pass something that CSS validator will reject.). UIs were different. Over the past 6 months, much of the work has been done to harmonize the tools.

Major effort is under way to provide simple programmable (SOAP) interfaces to these tools. Current status:

• CSS: in production
• Feed: in production
• Markup: still in development

These SOAP interfaces have enabled, for example, the creation of a single AJAX-based front-end to the both the Markup and CSS validators. In future, additional checks could be added (assuming that other tools are available with SOAP interfaces). Tools are written in different languages (PERL, Java, Python) - SOAP allows us to unify. Tools are developer by volunteers, mostly in Europe. The Markup validator mailing list has several hundred subscribers. A hard-core of supporters provide help. CSS also has a mailing list too. RDF validator was not actively managed but Jeremy Carroll (HP) has now agreed to take this on.

There is an appeal to members for assistance with development and maintenance. We need to encourage member companies to use the W3C validators. Some packages are made available for in-house installation (sometimes through distributions like linux flavours) if there are concerns about passing internal documents to an outside instance of the validator. It's hoped that as usage increases some companies will contribute resources for development or bug-fixing.

Some years ago people considered the Markup validator to be the "reference" for HTML rather than the HTML specifications themselves. Now people have understood that this is just a tool.

We might create a low traffic "announcement" list for all validators which would complement the RSS feed of the W3C Open Source software page. We could also look at avenues like VersionTracker, Freshmeat, etc.

ACTION: Olivier to investigate new avenues to advertise new releases of validators

Joint session with ERT Working Group on EARL

We had a joint session with ERT Working Group about EARL on how the language is defined. The test metadata schema could be reused in part by EARL which is likely to make this effort even more urgent. (See Minutes)