The specification food chain

Recent times have seen a number of discussions about W3C, in mailing-lists or weblogs. Good discussions: food for thought and means for actions. Weblogs and their comments feature sound reproaches about W3C Activities and Process, as well as light support. We (W3C Team) are reading as many of these, and participating in these discussions as much as possible.

One striking thing, however, in this mass of very interesting comments, is a lack of knowledge about the W3C ecosystem. Let's try and clarify a few points.

Specification Development Time

Developing a technology and writing a specification takes time. A lot of time. Yet, The Web community often voices two completely opposite opinions. Some will say "Wow, you are moving too fast! I didn't have time to get used to this technology, and a new version is already available." Others will say "Come on, I want to use the cool features of technology X. I want it standardized, and I want it now."

There is no easy answer to this dilemma: the fact is, it takes time to write a good specification. Good cooking takes time.

How much time a technology takes to reach maturity is not arbitrary: it comes from a balance between the energy and motivation of technology implementors on the one hand, and on the other hand constraints due to normative dependencies which have to be checked, coordination with other groups inside the organization and outside the organization, and the feedback of the community to be handled in a timely manner.

Among the things a working groups needs to handle, count:

• proposing features for the technology,
• writing down the prose in a document,
• creating test assertions,
• creating test cases,
• resolving technical issues,
• coordinating with other groups,
• hunting down implementations for interoperability
• publishing some helpful companion materials, if possible

Working Groups participants are people from W3C Membership, a mix of people from IT companies, governmental agencies, and academia. Take 10 to 20 percent of their time, sprinkle a few invited experts from outside W3C membership: that's the recipe for most W3C WGs.

Using these resources, a Working Group will struggle to reach consensus, get the community involved, and ultimately produce solid, well written specifications: each of these steps takes time.

Test Cases and Test Suite

A testing effort is a fundamental stone in any serious software development. W3C is about 10 years old now, a rather young organization, compared to venerable standard organizations like ISO with their long history and experience. Creating test cases and a full test suite for a specification is something which has become part of the life of W3C WGs in the past few years.

There are basically two main models for test case development:

Waterfall:

The specification is developed, and when entering the Candidate Recommendation phase, the WG will seek implementations to prove interoperability and implementability. To help in this task, the WG is producing test cases that developers will use in their software.

Test-Driven:

The specification is almost written to formalize the results of test cases. A feature is suggested to the WG . Even before writing any prose about it, a test case is designed. On the basis of this test case and its implementation, the specification prose will be written. The benefit here is that when entering Candidate Recommendation, the WG is likely to have an already mature test suite.

Some WGs will adopt a mix between the two methods. Nowadays, the QA team is encouraging WGs to build more test cases from the very start of the specification development. It is important, not only for developers who will try to implement the language, but also as a sanity check for the specification itself. If a test case for a particular feature is difficult to conceive, it often means that the feature is too complex, or that the prose describing the feature is too ambiguous.

Interoperability and Candidate Recommendation

Test suites do not guarantee interoperability, but they help achieve this goal. When reaching Candidate Recommendation stage, the WG needs to define minimal criteria for showing that the specification is usable and implementable. Often the chosen criteria will be to have two interoperable implementations of each feature. This has its own limits and issues but gives a good starting point. When implementations of the same feature differ, it can mean that the specification is ambiguous, that the test is wrong, or that one or more implementations are incorrect. The WG analyzes the results and discuss the issue until it identifies a solution.

Before entering the next phase, Proposed Recommendation, the WG has to produce an implementation report showing which features are at risk, and which have been implemented to an extent that proves its worth. The specification in development can even go back to a Working Draft stage, if the implementation results are not good enough, if the group can not satisfactorily prove that the specification is mature enough to become a Recommendation.

Certification and Web Ecosystem

One related topic is often raised in discussions : "Why doesn't W3C do certification?" The QA Activity considered the issue at length, and produced a document called Conformance Testing and Certification Model for W3C, trying to figure out how such an activity would be organized.

The Web has been a fairly open space for developers. Software is developed upon technologies that are freely accessible and implementable on a royalty free basis. Creating a certification activity inside W3C would deeply modify this ecosystem.

Certification is not the result of passing a test suite for one technology. Certification is a legal process between two parties. When developing a certification program, there will be questions like:

• Is it about certifying a product, or people, or school curriculum, or a process in a company?
• What do we want to certify: HTML? CSS? URI? HTTP? and more.
• What is the meaning of "HTML certified"? How do we check it?
• Who is given the authority to certify a product?
• Which version of the product does certification cover? All? Or one?
• How much does it cost?
• How often must it be done?
• What will be the drawbacks for small companies or open source world which do not have the money to be certified?

Certification raises a lot of new issues: it is not simple, nor necessary desirable. If a certification program was launched at in the future, it would certainly modify the whole Web ecosystem.

Public Participation to Specification Development

The W3C is far from being perfect, but it is an amazing environment to develop a technology. There are rules, driving the development of specifications.

Sometimes mistakes are made with regards to these rules: handling comments, advancement of a specification at the next stage, etc. Reproaches recently made to W3C have been encouraging in a way: they showed that the organization is open enough to be accountable to its participants, and to the public. People can voice their disagreement, pushing their issues. And all this can be done in an open field, and start an open debate.

Public feedback is essential for the life of W3C. It helps create better specifications by raising and discussing technical issues, by reviewing materials, by providing fresh input. The public communities also acts as watchdogs for everything done inside W3C.

W3C welcomes the Web communities' feedback, and we hope this QA weblog can provide developers and designers with ways to be more involved in the W3C process: in an upcoming article, we shall list ideas and recipes for a smooth, efficient participation.

Stay tuned.