QA Framework: Specification Guidelines

W3C Working Draft 22 November 2004

This version:
Latest version:
Previous version:
Karl Dubost, W3C
Lynne Rosenthal, NIST
Dominique Hazaël-Massieux, W3C
Lofton Henderson, CGM Open
See Acknowledgments.


A lot of effort goes into writing a good specification. It takes more than knowledge of the technology to make a specification precise, implementable and testable. It takes planning, organization, and foresight about the technology and how it will be implemented and used. The goal of this document is to help W3C editors write better specifications, by making a specification easier to interpret without ambiguity and clearer as to what is required in order to conform. It focuses on how to define and specify conformance for a specification. Additionally, it addresses how a specification might allow variation among conforming implementations. The document is presented as a set of guidelines or principlerequirements, supplemented with good practices, examples, and techniques.

Status of This Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This document is the 22 November 2004 Last Call Working Draft (WD) of “QA Framework: Specification Guidelines”, made available by the the QA Working Group of the W3C Quality Assurance (QA) Activity for discussion by W3C members and other interested parties. For more information about the QA Activity, please see the QA Activity statement.

This is the Last Call, third published version of this document after the decision to completely redesign the QA Framework, resolved by the QA Working Group at its 2004 Technical Plenary face-to-face. It is a lighter-weight, less authoritarian, more user-friendly and useful version of the previously published Candidate Recommendation version of the QA Specification Guidelines. Because of the extensive nature of the changes, it has been returned to Working Draft.The changes since the previous version are documented in the changelog and highlighted in a separate document.

Publication as a Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

Patent disclosures relevant to this specification may be found on the Working Group's Patent disclosure page in conformance with the W3C Patent Policy of 5 February 2004. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) with respect to this specification should disclose the information in accordance with section 6 of the W3C Patent Policy.

The Working Group expects this specification to become a Recommendation. The Last Call review period ends 28 January 2005, at 23:59 EDT. Please send Last Call review comments on this document before that date to www-qa@w3.org, the publicly archived list of the QA Interest Group. Reviewers please note that your comments will be publicly archived and available; do not send information that should not be distributed, such as private data. Last Call Working Draft status is described in section 7.4.2 of the W3C Process Document. To the extent possible, please provide a separate email message for each distinct comment.

Table of contents




This document is a guide for W3C specifications editors and authors. It provides guidelines for writing better specifications. This document differs from other W3C process and publication related documents in that it looks at specifications from a conformance viewpoint. It address the most basic and critical topics with respect to conformance, including how to convey what is required for an implementation in order to conform and how to allow variation among conforming implementations.

What is a specification?

A specification is a set of technical requirements which aim at defining a reliable interface to accomplish a given task.

Specifications can be defined in one or several documents, and can import requirements of other specifications with normative references. Some specifications, denoted below as umbrella specifications, create their own interfaces by simply grouping requirements of existing specifications in a well-defined manner.

Figure 1: Umbrella specification

Graph to express the notion of composite document specification.

On this figure, the technology is composed of two modules (defining functional division of the technology), a profile (defining the requirement of implementation for a specific device) and a primer (introducing the technology and its basic concepts). They are grouped together by a document, “umbrella specification”, which makes it a logical, usable and complete technology specification.

Goals and Motivation

The goal is to enable Working Groups to produce specifications that are precise, easier to interpret without ambiguity or confusion, and clearer as to what is required in order to conform. Good specifications lead to better and more interoperable implementations and foster the development of test suites and tools.

Everyone benefits from having well-written specifications. Editors may have less rework and thus, fewer issues raised during the development of the specification, and fewer errata once it is finished. Implementers can implement sooner and have a better chance to conform to the specification. Test developers are able to derive unambiguous test assertions. Users benefit by having interoperable solutions. W3C gains by having recommendations produced with higher quality and reduced maintenance.

Why Specification Guidelines?

It is not an easy task to write accurate, clear, complete, unbiased specifications. It requires planning, organization, and foresight about the technology, how it will be implemented and used, and how technical decisions affect conformance. This document provides a collection of principlerequirementrequirements, good practices, examples, and techniques that leads the reader through the decisions that need to be made in order to write precise requirements and establish, define, and specify conformance for specifications. Realizing that editors and authors are busy, under pressure to get the specification published, and already have a reading list of W3C documents (a good place to start, W3C Editor's Home Page [EDITOR]) this document can be used as a checklist of things to consider, a how-to guide with examples and techniques which can be adapted, and a reference for understanding conformance concepts and terminology.

Audience of this document

The primary audience of this document is editors and authors, however, it is applicable to a broader audience including:

This document makes no distinction between the terms editors and authors and will refers to them collectively as editors.

About this document

This document is a practical guide to writing a specification, presenting editors with topics to consider. Consequently, much of the document is non-normative recommendations labeled Good Practices. There are however, a limited number of normative requirements, labeled PrincipleRequirements, that must be implemented in order to conform to this document. The overall objective of these principlerequirements and good practices is to facilitate the creation of a complete conformance clause in every specification. A conformance clause template [CONF-TEMPLATE] is provided to assist editors satisfy the requirements of this document and end up with a conformance clause. Note that for some specifications (e.g., The QA Handbook [QA-HANDBOOK], Architecture of the World Wide Web, First Edition [WEB-ARCH]) where conformance is not an issue (e.g., no normative content), the conformance clause may be an explanation of why there is no "conformance to this document" which may not be a separate section but incorporated into another sectionand may be presented in another section rather than in a separate conformance section.

The topics presented are herein inclusive (self-contained) providing information needed to understand and successfully apply the principlerequirement, although related information and advanced topics may be referenced.

If in a hurry - start atjust read the first section, Specifying conformance — this ismay be all you need to read in order to reach the expected outcome of adhering to this document, i.e. specificying conformance. It serves as a roadmap to other parts of this document, which help you achieve specifying conformance.

Structure of this document

This document is organized ininto a series of main topics like Conformance or Extensibilitysuch as Specifying Conformance and Managing Variability. In each of this main topics, a few principlerequirements are developed and explained. Theses principlerequirements are accompanied by techniques and examples. The techniques illustrate basic (and non exhaustive) questions or methods that might help you create a specification. The examples are explanations or extract of specifications which specifically address the point which has been made in the principlerequirement.In each of these topic areas, PrincipleRequirements and Good Practices are presented and explained. Each PrincipleRequirement and Good Practice is accompanied by techniques and examples. The techniques illustrate basic (and non exhaustive) questions or methods to help you to realize the PrincipleRequirement/Good Practice and write bits of the specification. The examples are explanations or extractions from existing W3C specifications which specifically illustrate the point which has been made in the PrincipleRequirement/Good Practice.

The conformance requirements of this document are described in the conformance section of this document. A specification editor who wishes to conform to Specification Guidelines has to followmust satisfy the conformance requirements in the conformance section of this document. It basically defines the principlerequirements as normative and the good practices as informative.

The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY ", and "OPTIONAL" will be used as defined in RFC 2119 [RFC2119]. When used with the normative RFC2119 meanings, they will be all uppercase. Occurrences of these words in lowercase comprise normal prose usage, with no normative implications.

Other QA Framework documents

This document can be used as a stand alone document or as part of a family of QA Framework documents designed to help the Working Groups improve all aspects of their quality practices.


1 Specifying Conformance

Clear presentation of what is meant by conformance to the specification is ultimately crucial to successful interoperability of implementations. The conformance policy and language in the specification not only determine its testability, but also shape the overall look of its implementations landscape — are there numerous fragmented flavors or a few strong, interoperating variations?The conformance model, its description and the language used to convey normative information not only determines the testability of a specification, but also influences the overall implementation landscape  e.g., multiple types of conformance resulting in numerous, fragmented, conforming implementations or a limited types of conformance resulting in few allowable variations among implementations.

Conformance is the fulfillment of a product, process, or service of specified requirements. These requirements are detailed in a standard or specification as either part of a conformance clause orand in the body of the specification. A conformance clause is a section of a specification that states all the requirements or criteria that must be satisfied in order to claim conformance to the specification.

1.1 A conformance clause is essential

With aA good conformance clause is the ultimate goal of these guidelines, and is sanctioned by the conformance to this specification, a specification comes close to fullachieving conformance to this QA Specification Guidelines.

The conformance sectionclause of a specification — commonly called the Conformance Clause — is a high-level description of what is required of implementationsers and application developers. It, in turn, may refers to other parts of the specification for some details. Ideally, it is the root source from which readers can find any conformance-related information.

For some specifications, the conformance landscape may be plain and simple, and the conformance clause template may be almost all that is needed [Example?]. For others, the conformance landscape will be complex or convoluted, and the advanced details and topics of these QA Specification Guidelines — topics like managing variability (section 4) — may be invaluable.For some specifications, the conformance model may be straightforward and simple, and the conformance clause template [CONF-TEMPLATE] when completed, may provide most of the information needed in a conformance clause.  For others, the conformance model will be is complex or convoluted, and the Advanced Topics referencesadvanced details… may be invaluable in creating the conformance clause.

1.1 PrincipleRequirement A: Include a conformance clause.

CR 8.4 Include a conformance section. [Priority 1]

What does it mean? For starters, the conformance clause needs to answer the all-important question: what may conform and how? It may, for conformance purposes, partition the technology into functional subsets, such as profiles, levels, or other structures. Additionally it may specify the permissibility of extensions, options, and alternative approaches and how they are to be handled. For starters, it is the root source from which readers can find any conformance-related information.  The conformance clause provides the answers to the all the important question: what may conform and how? The conformance clause defines at a high-level, what is required of implementers of the specification.  It, in turn can refer to other parts of the specification or other specifications.  The conformance clause may partition the technology into functional subsets, such as profiles, modules or other structures.  Additionally, it may specify minimal requirements for certain functions, as well as the permissibility of extensionsextensibility, optionsoptional features and alternative approaches and how they are to be handled.

Why care? A well constructed conformance clause brings most of the benefit of these QA Specification Guidelines for the least effort. Almost all of SpecGL's significant advice — normative PrincipleRequirements and recommended Good Practices alike — are satisfied with a good conformance clause.The conformance clause provides a common understanding ofdefines what is required to claim conformance: as such,  it provides communication between the standardsspecifications developerscreators, implementers, and users as to what is required, and gives meaning to the phrase, “conforming implementation”.  Moreover, it facilitates the consistent application of conformance within a specification and across related specifications.  It promotes interoperability.


Ruby Annotation [RUBY] specification givesis an example of a short specification with a detailed conformance clause.

SVG 1.0 [SVG11] specification givescontains a detailed conformance clause for a complex modular technology.

1.1 Good Practice B: Define the specification's conformance model in the conformance clause.

What does it mean? The conformance model is the conceptual framework in which conformance is defined. It consists of and is defined by addressing at least these three topics:

Why care? By defining the conformance model, implementers will be able to articulate the different sections of the technology with regards to their implementations. It will be easier for them to focus on the relevant parts of the technology and therefore it will be faster to develop a good quality implementation.The key is to communicate to the reader what conformance to the specification is all about.  It provides a framework for implementers to know what they need to build in order to conform and the different ways that they could claim conformance.  It provides users/buyers a basis to express their requirements.

  1. List the classes of products that are targeted by the specification
    1. Identify the types of products will implement the specification
    2. If not already done (above step), group the products into a generic category – this is the class of product. (e.g., content, user agent, protocol, API, specification)
  2. List conformance designations or conformance concepts. To help realize this, consider these questions.
    1. Does conformance mean something different for different classes of products?
    2. Is more than 1 type of conformance defined – e.g., different designations (well-formed, valid) or degrees of conformance (A, AA, AAA)?
    3. Is conformance tied to the class of product and similarly named (e.g, host-language conformance, document conformance)?
  3. Create a name for each way that conformance can be qualified – i.e., label it
    1. If the specification is subdivided into modules, profiles and/or levels, is there a conformance designation associated with each type of subdivision?
    2. If the specification has options and extensibility, will these have an affect or be affected by the conformance designation?
  4. Draw a diagram to put it all together – sometimes its easer to work from a picture
    1. Diagram the classes of products with associated conformance designations,
    2. Add the subdivisions and any other variability (e.g., options, extensibility)
    3. Extract from this diagram and define the conformance model.
    4. Write the description into the specification. Bonus - if the diagram help to understand the model, include it.

Figure 2: Conformance Model of Ruby Specification

Graph illustrating the Ruby Conformance Model

Ruby Annotation [RUBY] specification defines two types of conformance: Simple and Full, which are tied to the content models. The specification defines six types of products that we could group in 3 classes of products (content, specification, user agent). For each of them the conformance section defines the requirement to fulfill Simple or Full conformance.


XML 1.0 [XML10] has 2 classes of products (document and processor), each of those have 2 conformance degrees (well-formed/valid and validating/non-validating); we could add xml:base, xml namespaces and xlink as "modules" for xml even though they haven't really bebeen formally defined as such

On the other hand, SVG 1.1 [SVG11] has roughly 4 classes of product (markup fragments with various extents, generators, interpreters and viewers). Some of these classes of product have various degrees of conformance (Appendix G: Conformance Criteria [SVG11]) (e.g. static / dynamic for interpreters, static/dynamic * high-quality for viewers); SVG 1.1 also defines modules, which are grouped into profiles (tiny/mobile/full); having a nice diagram summarizing it would be both interesting and instructive as to how complex a conformance model can get once you start playing with dimensions of variability.

1.1 Good Practice C: Specify in the conformance clause how to distinguish normative from informative content.

CR 7.2 Distinguish normative and informative content. [Priority 1]

What does it mean? Normative content is the prescriptive part of the specification, whereas informative content is for informational purposes and assists in the understanding and use of the specification. Content includes all sorts of different forms — not only descriptive prose, but illustrations, examples, use cases, formulae and other formalisms.

Why care? Conformance of implementations is defined by and measured against normative content. It will removes any reading and understanding ambiguities for the different parts of the specification. This Good Practice is aimed principally at the high level partitioning of information in the specification.Conformance of implementations is defined by and measured against normative content.  Distinguishing normative content from that which is informative helps to make sure the reader can find the normative content, knows for sure that it is normative, and doesn’t fail to notice a normative section.  This good practice is aimed at the high level partitioning of information (e.g., sections) in a specification.

  1. For each section of yourthe specification defines if the content is normative or informative.
  2. Once identified, label the section with a wording saying explicitly the normative or informative status of the section.
  3. For each section in the specification,
    • Determine if the content is normative or informative and, explicitly label it as either ‘normative’ or ‘informative’. 
    • Make the label part of the section heading (e.g., Informative References), as parenthetical text with the heading (e.g., Glossary (normative)),
    • Or create a list of each section and indicate its normality, (e.g., this document’s conformance section’s Normative Parts).
  4. In the conformance clause, explained what the wording means and how does it relate to conformance. In the conformance clause, explain what the use and meaning (if necessary) of the words used to convey the normality and informality of the content. See 3.2 PrincipleRequirement A: Use a consistent style for conformance requirements and explain how to distinguish them.
  5. Try to avoid a language which sounds normative in an informative section. It might lead the specification users to wrong assumptions.

XHTML 1.0 [XHTML10] is using the words of RFC2119, but in an extended way (See Definitions [XHTML10]). They define the exact terminology for each word.

1.2 Specify how to make conformance claims

The central message of this section — “have a good conformance clause” —has many ancillary details. Because the conformance clause is the foundation for defining and measuring conformance, it is also the basis for assessing conformance claims. One detail worthy of attention is “valid conformance claims”.

Rather than live with the infinite varieties of creative conformance claims that willcan arise in a vacuum, the specification can be proactive:.

1.2 Good Practice A: Provide the wording for conformance claims.

CR 9.1 Provide specific wording of the claim. [Priority 3]

What does it mean? It is inevitable that people (e.g., vendors, purchasers) will either claim conformance or demand conformance to a technology. In fact, claiming conformance to a technology may be required in certain situations. Thus, it is important to provide a consistent and unambiguous way for these claims to be made. Identification of the specification version, class of products, and conformance label are some of the items that could be part of such wording.

Why care? Having a framework by which conformance claims can be made for a particular usage of the technology, will minimizes confusion by people who are interested in such claims. There are many contexts for which conformance claims are sought, including legal as part of regulations, laws, or policies and commercial when selling or buying a product.


Conformance Claim Template:

Form 1:

  • Specification title, date, and version:  Foo Specification, version 1.0, 29 February 2003:
  • Specification URI: http://www.example.org/TR/2003/FOO-20032902
  • Conformance designation satisfied: "conformant processor”
  • Product name: My_Processor, version 1.2, 1 January 2004
  • Date of claim:  1 April 2004

Form 2:

On 1 April 2004, My_Processor, version 1.2, 1 January 2004 claims conformance as a “conformant processor” to the FOO Specification 1.0, 29 February 2003, available at http://www.example.org/TR/2003/FOO-20032902


WCAG 1.0 [WCAG10] has good examples of a specification explaining how to make a conformance claim depending on the degree of conformance.

1.2 Good Practice B: Provide an Implementation Conformance Statement (ICS) proforma.

CR 9.4 Provide an Implementation Conformance Statement proforma. [Priority 3]

What does it mean? An Implementation Conformance Statement (ICS) provides standardized information about the conformance of an implementation to the specification. It is used to indicate which capabillities and optionsoptional features have been implemented, as well as the limitations of the implementation. An ICS typically takes the form of a questionnaire for implementorimplementer to complete.

This Good Practice suggests that the specification itself include an ICS proforma. (Caveat. The ICS concept may be inapplicable to some types of specifications.)

Why care? An ICS provides detail about conformancea concise view of a specification’s conformance model. View the ICS as a template, where its organization, format and content can provide implementers and users a quick overview of the specification's features, options, subdivisions of the technology, conformance requirements, etc. It can be especially valuable as a statement of conformance, where implementers indicate which mandatory and optional features and options they implement and document the presence of extensions. Once completed by an implementer it can be used as part of the conformance claim. Additionally, an ICS can be used to identify the subset of a conformance test suite that would be applicable to the implementation to be tested: this is useful first when establishing an interoperability report, and then when setting up a conformance testing program.

  1. Create a list, table or form listing all features (capabilities) and indicating if it is mandatory or not.
  2. Provide space for the implementer to check: Yes, No, Not Applicable
  3. Organize the features according to the subdivisions of the specification or in the order they occur in the specification or in some other logical grouping
  4. If there are dependencies, express them. (For instance, if No to this question, jump to the next section.)
  5. You may want give a tool that will help the implementers to fill the ICS and have a formatted report (for example, with EARL).Provide a tool to help implementers fill out the ICS and produce a report (e.g., EARL).

QA Specification Guidelines provides an ICS [QA-SPEC-ICS] to help implementers to verify the conformance to this document. Good practices (informative) and PrincipleRequirements (normative), organized following the sections of the document, are given in a table where implementers can check yes, no or not applicable.

Web Content Accessibility Guidelines 1.0 [WCAG10] provides a checklist of checkpoint which helps the implementer to verify the accessibility of its HTML document. (See checklist of checkpoints in [WCAG10])

1.2 Good Practice C: Require an Implementation Conformance Statement (ICS) as part of valid conformance claims.

CR 9.5 Provide an Implementation Conformance Statement proforma. [Priority 3]

What does it mean? This simply puts together the previous two good practices. Not only could the specification provide an ICS proforma for implementers, but it could require it to be linked from its standardized conformance claim template.

Why care? Providing a filled ICS with the conformance claim might help customers and users to verify easily the level of support of individual requirements of the specifications. It also strengthens the value of the claim.

  1. Explain in the conformance claim section how the developer must fillwhy the implementer must complete the ICS for part of the claim.
  2. Give precise instruction how the ICS must be part of the conformance claim. It might be an external document, it might a link to precise dated document, etc.


2. The Nature of the SpecificationSetting up Groundrules

2.1. Scope

The path to a quality specification begins with its scope.  It is critical to convey what the specification is about by describing its intent and applicability.  As the specification develops, it is a good idea to revisit the scope to make sure it still reflects the intent of the specification or if it needs to be modified.

2.1 PrincipleRequirement A: Define the scope.

What does it mean? Describe what the specification is about.  Let the reader know what is covered in the specification.

Why care? This is one of the first sections a reader reads, so it is important to capture their attention and make sure they understand what the specification is about. It helps to keep the specification content focused.  It also will help readers know the limits or boundaries of the specification and whether it is of interest to them.


Many W3C specifications have included a scope prose in the Abstract section. As it is important that the scope is somehow defined it is a lot better to put that in a separate section as explained previously.We advocate making the scope a separate section in the body of the specification, making it easy to find and insuring that it is an item in the table of contents.

Could have been better:

Composite Capability/Preference Profiles (CC/PP): Structure and Vocabularies 1.0 [CCPP-VOCAB]: From CC/PP Scope section “CC/PP Structure and Vocabularies … defines a client profile data format, and a framework for incorporating application- and operating environment-specific features.”

2.1 Good Practice CB: Use examples or use cases to illustrate.Provide examples, use cases, and graphics.

What does it mean? Illustrate concepts, behaviors, functionality, interaction, etc. through what ever means makes sense, such as examples, use cases, graphics, and sample code. This aids in the understanding of the specification, especially for areas that are innately complex or for which the Working Group has had to explain to its members or the public.  Additionally, a set of broad examples and use cases can help to clarify the specification’s scope.

Why care? It is difficult to understand some concepts, behaviors, functionality or other aspects of a specification without informative interpretations to aid the reader. Providing the reader with additional information beyond the specification’s prose, can only help in achieving implementation and interoperability.


It is up to the Working Group to determine the best way to illustrate the scope and other parts of the specification. Typically, the nature of the specification will influences the type of examples, uses cases, graphics, etc that make sense.


For markup specifications, provide at least one example of each markup construct; illustrate each transformation capability with an example showing input and output.

SVG 1.1 [SVG11]: For each element of the SVG specification, you will have thethere is a verbose definition of the element, the DTD definition, the attribute definitions and an example. For exampleinstance, in the definition of the element rect, you will findthere are precise examples with the markup to generate thea rectangle, a rendering of thise markup as an image to help people to visualize what it should beit and an original version of the markup to test it as a separate file with the said markup.

HTML 4.01 [HTML401]: The HTML 4.01 specification has been designed in a very educative way. It has strong caveats with regards to the implementability and the definition of testability. But By its design, the specification has very good examples. An important thing that you must say when you give an example is whether or not the rendering of the example is mandatory. Many implementers try to mockup a rendered example when a specific rendering is not mandatory.

When HTML 4.01 defines the elements strong and em and gives examples of it, it clearly says that The presentation of phrase elements depends on the user agent. Generally, visual user agents present EM text in italics and STRONG text in bold font. Speech synthesizer user agents may change the synthesis parameters, such as volume, pitch and rate accordingly. It would have even been better to mention that user agents should not assume specific rendering for these elements.

2.2 What needs to conform

2.2 PrincipleRequirement A: Identify who or what will implement the specification.

2.1 Identify classes of product. [Priority� 1]

What does it mean? Clearly identify the class of products (i.e., type of products or services) to which the specification applies.  If multiple classes of products are targeted by the specification, make sure each are described.   Examples of classes of products include: content, producer of content, player, protocol, API, agent, guidelines.

Why care? It helps define the scope of the specification and is needed when defining conformance.  It also helps the reader know what is being targeted by the specification – that is, to discover and focus on what they have turned to the document for and avoid what they may find immaterial.


QA Framework: Specification Guidelines defines one class of product – specifications

Mathematical Markup Language (MathML) Version 2.0 (Second Edition) [MATHML20] there aredefines output-compliant authoring tools and input-compliant rendering/reading tools.

SMIL 2.0 Language Profile ([SMIL20], chapter 13), there arehas 2 classes of products: documents and basic user agents.

SMIL 2.0 Language Profile ([SMIL20], chapter 13), there are 2 classes of products: documents and basic user agents.

The conformance section of Ruby [RUBY] is very explicit and detailed about classes of product. For each of these classes, Ruby conformance section defines a set of rules, the implementers or the users must respect. It defines rules for markup, DTD, document, module, generator, interpreter.

The Conformance definition of XHTML Modularization [XHTML-MOD] introduces also classes of product and defines the conformance for each of these classes.

Schema Part 1 [XML-SCHEMA-1] could be said to define an abstract notion of “schema validity” but this checkpoint can only be satisfied if the Recommendation says explicitly whether it is setting expectations of an XML parser or of a “schema validator” that could be stand-alone.

Schema Part 2 [XML-SCHEMA-2] defines data types, so it is a specification of type 2 above — content/data — and it is used as a foundation by other specifications (e.g., XPath 2.0) as well as being part of the schema validator and hence parser requirements. The specification could define the behavior of a data-input tool that rejects data not fitting the schema, but it probably would not because the tool would be expected to use a parser module to validate the data. To satisfy this checkpoint, the Schema Part 2 specification has to make clear whether it is to be taken as an independent specification of a parser (that produces data from arbitrary strings) or a foundation to be used by other specifications.

2.3 Make a list of nNormative (and non-normative) references

ISSUE: Björn Höhrmann has raised an important issue about the way normative references are given and their implications on the normality of the specification itself. An short abstract of the issue can be foreseen as:

The QA Working Group is still discussing the issue. You are welcome to participate on the QA IG Mailing list (www-qa@w3.org publicly archived), but read first this two mail threads:

A specification is rarely written in isolation. It inherits from previously defined technologies and might set the future of other specifications by defining their base. Then it is essential to clearly defines what is the nature of the specifications (normative, informative), the technology refers to and the implications of these references for the future of the technology itself.

As a side note, there is also an interest to study and understand the conformance model of the technology which is developed in order to minimize the difficulties of other specifications using it in a conformant way.

2.3 PrincipleRequirement A: Make a list of normative references. @@Something about normative references@@ including time considerations, super setting or sub setting, and breaking conformance.

CR 7.5 Make normative reference to specifications on which the current specification depends [Priority 1]

What does it mean? A specification is rarely developed from scratch: it usually relies on other technologies defined in different specifications. The Working Group has to identify any specifications which defines the core technologies of the developed technology.

Why care? For the Working Group, it has an immediate benefit: “do not reinvent the wheel”. Using features which are already defined in other documents helps to minimize the size of the document which is developed and avoid too many ambiguities by rewriting the same concepts.

For the implementers, it has the huge benefits of knowing which part of the specification is based on another technology. It makes clear what are the implications for the conformance. It may help them to minimize their work by using conformant libraries already implemented elsewhere.

For the users, it might help them to understand where the technology is coming from and therefore how to use it in combination with other technologies they might already know.


Most W3C specifications contain a list of normative references, which are clearly identified at the end of the document.

2.3 Good Practice B: Do systematic reviews of normative references and their implications.

What does it mean? Each time, a normative reference is added to the specification, it has deep implications on the way the technology is defined. Each choice has to be reviewed and checked for the consequences in terms of time, consistency, precision, and use of the technology in specific conditions.

Why care? A specification defines a technology which will be used for a long time. The choice of precise and exact normative references is thus fundamental. Using a normative reference which evolves over time might endanger the specification or other specifications relying on it.

For the Working Group, reducing the degree of ambiguity or variation in the normative references minimizes or removes the possibility of misunderstanding. For implementers, it removes ambiguities and contradictions between different sets of technologies. It creates a stable environment into which they can make stable developments.


For the sake of illustration, we give here a list of cases that illustrates some of the problems and their implications for the technology. It is not exhaustive review of all possible cases. Imagine, a technology has to use the notion of URI or URI-references. The technology is already defined in a specification which is called RFC2396 and has for title “Uniform Resource Identifiers (URI): Generic Syntax”.

Let's create a simple definition and give examples of possible problems that arise from this definition:

The value of the attribute is a URI as in [RFC2396]

which can be expressed by


Precise designation and reference: The first example is illegal as the example uses a URI Reference as opposed to only a URI, RFC 2396 clearly distinguishes between those constructs. It would have been better to say:

The value of the attribute is a URI Reference as defined in section 4 of [RFC2396]

Superset of the reference and interpretation: RFC 2396 does not include support for IPv6 literals, the syntax has been introduced in RFC 2732 which does not update RFC 2396. It is not correct to assume that it does even if it seems logical. Do not interpret the intention of the external reference.

Contradictory choices and references: Using the concept of URI-reference as a normative reference and then in the prose of the specification, define that it is an error to use it.

3. Pieces needed to Specify ConformanceDefining and using terminology

3.1 Define your terms

3.1 PrincipleRequirement A: Define the terms used in the normative parts of the specification.

What does it mean? The normative parts of a specification often uses technical terms in a very restricted sense; write down the definitions behind these terms. Use the same phrases to convey the same meaning: although repetition is often seen as a stylistic error in prose, in a technical specification, re-using the same phrases diminishes ambiguity, and lowers the threshold of vocabulary needed to understand the specification.

Why care? English (as any other natural languages) areis ambiguous, such that a term used in a given context can be interpreted differently in a different context. Interoperability between implementations can only be achieved if implementers have the same understanding of the specification; defining the terms used in the normative parts promotes this common understanding which contributes to accurate implementation.

Also, once well-defined, the terms can be re-used in other specifications.

  1. Review yourthe conformance requirements/test assertions: the most important terms to define in yourthe specification are usually easy to identify when reviewing conformance requirements or test assertions.
  2. Use the element dfn in HTML to indicate that this is the defining instance of the enclosed term. It will be easier to create a glossary of your terms later on. For example in this document
    <dfn>Conformance</dfn> is the fulfillment 
    of a product, process, or service of specified 

Cascading Style Sheets, level 2 (CSS2) Specification [CSS20]: The interoperability of the implementation of the CSS2 box model has been problematic, due to the lack of definition for when a property is set (see discussion on this topic on the www-style Mailing list in March 2001)

3.1 PrincipleRequirement B: Create conformance labels for each part of the conformance model.

What does it mean? Most specifications define different types of conformance, e.g. a language often defines conformance for a parser and for a document. Associate well-defined labels to these types of conformance (like a well-formed XML document).Many specifications define more than one type of conformance, where each type is applicable to a different class of product. For example, a language specification may define 2 conformance types – one for a parser and another for documents (i.e., content). Associate a well-defined label for each different type of conformance.

Why care? Having a label associated with each type of conformance helps interoperability, testing, and branding. It gives implementers a way to identify and discuss their implementations and express the degree to which an implementation has met a specific set of requirements in the conformance clause. It gives test developers a meaningful set of requirements to develop tests for and for which conformance claims can be made against. It gives users a way to articulate their buyer requirements by having a unique way to refer to conforming implementations.

  1. Review all the different types of conformance in yourthe specification (ideally, they should be listed in the conformance clause)
  2. Make sure they have an associated label and this label is clearly confined. Have a summary of the labels of conformance with the name and definition

XML 1.0 [XML10] defines a well-formed XML document, a valid XML document

WCAG 1.0 [WCAG10] defines a level A conformant document

SVG 1.1 [SVG11] defines “conforming SVG document fragments”, “conforming interpreters”, “conforming viewers”.

3.1 Good Practice C: Define the unfamiliar terms in-line, and consolidate the definitions in a glossary section.

What does it mean? Put the definition of any new term along with its first occurrence in the text, but make sure that all the definitions are also available from a central glossary section.

Why care? Having the definition in-line makes it easier to read the whole specification from top to bottom, while having a centralized section of defined terms makes it handy to look up a term when looking at a specific section, or for others to re-use yourthe definitions.

  • if you useWith XMLspec to edit the specification, using <termdef> and <term> to markup the in-line definitions makes it possible to get automatically a glossary with <glist>
  • if you useWith XHTML, extracting the glossary from the in-line definitions(@@Need to find an elegant way to give the reference@@) can be done with an XSLT.

Thanks to Using the techniques of XMLspec, you can see in XML Schema Part 1 [XML-SCHEMA-1],has both the in-line definition and the glossary

3.1 Good Practice D: Re-use existing terms, and don't redefine themUse terms already defined without changing their definition.

What does it mean? When a definition for a term already exists (e.g. in a different specification) and matches yourthe specification needs, re-use the term and its definition without changing it, and provide a reference to the source.

Why care? Re-using existing terms reduces the cost of crafting new definitions and makes it easier for readers already familiar with other specifications to get into the new one. Also, conflicting definitions for the same term leads to reduced interoperability.

  • Use the W3C Glossary [W3C-GLOSSARY] system to check if there are existing definitions of the given term you want to re-use
  • When quoting a definition from another specification, put a link back to the specification

QA Specification Guidelines reuses the terms defined globally in the QA Glossary [QA-GLOSSARY] and used by all documents of the QA Framework

3.2 What is mandatory

3.2 PrincipleRequirement A: Use a consistent style for conformance requirements and explain how to distinguish them.

What does it mean? Different styles are used across specifications to convey conformance requirements: RFC 2119 [RFC2119] keywords, imperative voice, descriptive assertions, … Stick to one type of style, and tell yourthe specification readers which style is used in the text.

Why care? It's important for specification readers to be able to differentiate requirements in the specification from non-requirements in order to either implement or review them.

  • Using RFC 2119 [RFC2119] Keywords (MUST, SHOULD, MAY, ...) makes it easy to spot conformance requirements, due to their specific uppercase formatting; according to the RFC itself, they should be used only to establish interoperabilitytion [WIKI-RFC-KEYWORDS];

    A good conformance requirement using RFC Keyword is of the form: subject rfc_keyword operation, where subject is one of the classes of product, rfc_keyword one of MUST, SHOULD, MAY, and operation a verb describing one of the operations the classes of product can do (e.g. “send a message”, “process a request”)

  • The descriptive style takes a different approach: it describe the semantics of the language, rather than how it must be handled; the benefits of this approach is that it allows a wider reuse of the said semantics, but at the cost of not defining a common behavior between implementations, which may lead to interoperability issues (see also the wiki discussion on Meaning vs Behavior [WIKI-MEANING-BEHAVIOR]). Whatever the descriptive style you choose, stick to it.
  • The imperative style uses the imperative form to convey the requirement; it is often used in guidelines or specifications that needs readers involvement. Its defaultsweaknesses are that it doesn't necessarily make clear what needs to conform (since there is no subject), and the use of the imperative voice may make it harder to translate in some languages
  • avoid using language that looks normative (e.g. with RFC Keywords) in informative section

Using one of these styles doesn't preclude using another one in a different part of the specification as long as this is made very clear to the user. For instance, when defining a language, it is a good idea to define first its semantics using the descriptive style, and then the behavior of one (or more) type of implementations using RFC Keywords. See example for SVG 1.1


SVG 1.1 [SVG11] Recommendation, where the semantics are defined in descriptive style, and the implementations requirements with RFC Keywords in the SVG 1.1 Conformance section.

3.2 PrincipleRequirement B: ExplainIndicate which conformance requirements are mandatory, which are suggestedrecommended and which are optional.

What does it mean? Depending on the way you specify the conformance requirements are specified, it may or may not be clear if an implementation needs to implement all of them or only part of them. Try and make sure one can easily distinguish the type of requirements they have.

Why care? If implementers don't have the same understanding of what's required, interoperability is likely to suffer in the end.


For instance, HTTP 1.1 [HTTP11] defines two types of conformance, one where all the MUST are respected, and one where all the MUST and the SHOULD are implemented.

4. Managing Variability

Specifications allow some sort of variation between conforming implementations for different goals, e.g. adaptation to a particular hardware capacity, possibility to use extensions ; this variability, while often wanted to allow a broader usage of the technology, may have bad consequences on interoperability and needs to be carefully designed to find the right trade-off.Specifications allow for variation between conforming implementations for different reasons, e.g., adaptation to hardware capacities and extensibility.  Variability, while it can provide for broader usage of the technology, may impede interoperability.  Watch out for excessive variability – that which goes beyond what is needed.  Look for a balance between what is needed to allow for flexibility while still achieving the desired interoperability.

This section gives advices to helpon finding thisthe right balance; the reader would also gain to readwill also benefits from reading, Variability in Specifications [VIS] which goes into more details on the analysis of this variability.

4.1 Subdivide

Subdividing the technology should be done carefully.  Too many divisions complicates conformance and can hinder interoperability by increasing the chances of conflict with other aspects of the specification (e.g., other subdivisions).  Be smart when dividing the technology so that it is not excessive and provides a positive impact on implementation and interoperability.  The benefits of subdividing should outweigh the drawbacks.


Benefits: Subdividing can

  • make the technology easier to implement
  • facilitate incremental implementation
  • increase interoperability by focusing the technology on specific needs 
  • help organize the structure of the technology
  • provide better normative guidance than SHOULD provisions ofrecommended or optional features in the "core" spec
  • provide names for feature bundles, facilitating automated negotiation between sending and receiving products

Drawbacks: Too many divisions can

  • complicate conformance – need to account for interrelationships with other subdivisions and variability (e.g., extensionsextensibility)
  • hurt interoperability – increases likelihood of incompatible implementations
  • increase misinterpretation or cause conflict of requirements due to multiple or duplicative requirements. 

4.1 Good Practice A: Subdivide the technology to foster implementationCreate subdivisions of the technology when warranted by the variety of use cases.

ISSUE: Need umbrella to encompass more than use cases, but also requirements, technology, etc.

What does it mean? It may make sense to subdivide the technology into related groups of functionality to target specific constituencies, address specific capabilities or hardware considerations, provide for incremental implementation, facilitate usability, etc.

Figure 3: One possible organization of Profiles, Modules and Levels

Graph illustrating one possible organization of profiles modules levels

  • Module A and Module B are core modules. They are mandatory for all implementations. We could say that define the first level of conformance which we might call Basic Conformance.
  • Module C and Module D are the set of modules that have to be implemented to reach the second level of conformance. The Level 2 includes Level 1.
  • Module E is an optional module of the technology, not part of the conformance requirements.
  • Profile X defines an implementation of Level 2 (then Level 1 too) for a specific class of products for example.
  • Profile Z defines an implementation of Level 1 and an optional Module E.

Why care? For some specifications (e.g., huge, multi-disciplined), bundling functionality into named or anonymous packages can

  • provide an additional level of scoping control,
  • improve readability and the ability to find areas of interest, and
  • facilitate implementation and interoperability, since implementing the entire, monolithic specification may be impractical and undesirable
  • Use profiles. Profiles are subsets of a technology tailored to meet specific functional requirements of application communities.  The specification may define individual profiles, or mayand may also define rules for creating new profiles, or both.  An individual profile defines the requirements for classes of products that conform to that profile.  Rules for profiles define validity criteria for profiles themselves.
  • Use functional levels (aka levels).   Functional levels are a hierarchy of nested subsets, ranging from minimal or core functionality to full or complete functionality.  Levels are a good way to facilitate incremental development and implementation.  
  • Use modules.  Modules are discrete collections of semantically-related units of functionality that do not necessarily fit in a simple hierarchical structure.  Use modules when functionality can be implemented independently of one another  e.g., audio vs video module.  It is common to have implementers pick and choose multiple modules to implement.

Profile:  Mobile SVG Profiles: SVG Tiny and SVG Basic [SVG-TINY] is a profile aimed at mobile phones.

Profile: XHTML Modularization [XHTML-MOD] in section 3 and Synchronized Multimedia Integration Language (SMIL 2.0) Specification [SMIL20] specifications define rules for profiles.

Levels: There are no examples in W3C where levels are defined explicitly in a single edition of a specification.  CSS and DOM technologies are examples where levels are the result of progressive historical development and technology enrichment and are realized in a series of specifications. 

Profile/Level combo: Mobile SVG Profile: SVG Tiny, Version 1.2 [SVG-MOBILE] define 3 nested profiles -  Tiny, Basic, Full  - which are nested, like levels really 3 levels, each targeted a specific hardware communities. 

Modules: XHTML Modularization [XHTML-MOD] and Synchronized Multimedia Integration Language (SMIL 2.0) Specification [SMIL20] gives good examples of modules usage for a technology.

STOP: Only proceed if you did the above Good Practice.  Else, skip the rest of this section.

4.1 PrincipleRequirement B: If the technology is subdivided, then indicate which subdivisions are mandatory for conformance.

What does it mean? Regardless of the subdivision technique (i.e., profile, level or module), state whether one or more of the subdivisions is required for conformance.

Why care? Subdividing the technology affects and can complicate conformance with all the various combination of choices it provides – (a Chinese menu syndrome – 1 from column A, 2 from column B and desert is included).  Thinking about the various possibilities helps to structure the conformance model, taking into account how the subdivision willcan affect various classes of products.  Implementers as well as users need to know what is mandatory, optional, prohibited or conditional with respect to choosing what to implement and be conformantstill be conforming.


In the conformance clause, list the subdivisions that are mandatory for conformance. To help build this list, consider the following questions for each subdivision:

  1. Can an implementation conform without implementing the subdivision?
  2. Does the subdivision apply to specific classes of products and not to others?
  3. Is the subdivision dependent upon other subdivisions (that is, if it is implemented must others also be implemented)?

Content can be required to conform to one of the subdivisions (e.g., profiles)  or it may be conformant to the specification independently of a subdivision.  The question arises for a producer (of content): is it conforming if it generates content that is otherwise valid but does not conform to the subdivision.

WCAG 1.0 [WCAG10] defines three levels of Conformance depending on the levels.

XHTML Modularization [XHTML-MOD] defines minimal requirements for including certain basic modules if you designedwhen designing a conformant XHTML Mod document.

4.1 PrincipleRequirement C: If the technology is subdivided, then address subdivision conditions or constraints.

What does it mean? This is a corollary to the PrincipleRequirement above. Beyond being mandatory or not, subdivisions usually have conformance consequences due to minimal or new requirements, restrictions, interrelationships, and variability, etc.  As part of the conformance clause, describe the conditions or constraints associated with each subdivision

Why care? Creating subdivisions can get complicated, not just for the specification editor but for implementers who must pick and choose from the set of subdivisions.   Well designed subdivisions that convey the conditions, constraints, interrelationships, etc. willcan improve the clarity and understanding of the specification, conformance to the specification and facilitate implementation and interoperability.


In the conformance clause, describe the conditions or constraints on subdivision usage.  To help accomplish this, model or graph the subdivisions indicating the following that applies:

  1. Atomicity of the subdivisions: Represent each subdivision showing its atomicity.
  2. Mandatory subdivisions: Label the subdivisions that are mandatory
  3. Minimal requirements: List minimal requirements for each subdivision.
  4. Dependencies among subdivisions: Show dependencies and interrelationships of the subdivisions. For example, modules that requires and builds on functionally related modules, i.e., modules that require modules from other functional areas.
  5. Conditions and constraints on subdivisions groups: Indicate conditions and constraints for combined occurrences of subdivision pairs or groups:
  6. Conditions or constraints associated with specific classes of products: Indicate which conditions and constraints are applicable to specific classes or products.
  7. Other conditions or constraints beyond these: Indicate any other conditions or constraints.

Synchronized Multimedia Integration Language (SMIL 2.0) Specification [SMIL20] has a SMIL 2.0 Language Profile for user agents (See section 13 [SMIL20]) but also provides a SMIL 2.0 Basic Profile for wireless and embedded devices (See section 14.3 in [SMIL20]). The SMIL 2.0 language Profile requires that a user agent implement the BasicAnimation module but not the SplineAnimation Module.  The SMIL 2.0 Basic Profile on the other hand does not require implementation of any of the animation modules.

Dependency or intertwined relationship between profiles and modules is common – The combination of XHTML 1.0 [XHTML10], SMIL 2.0 [SMIL20] and SVG 1.1 [SVG11] is an example.

[@@ would be great to graph/model a technology]

4.1 Good Practice D: Define rules for creating profilesIf the technology is profiled, define rules for creating new profiles.

What does it mean? If profiles are used and profiles can be developed outside the specification itself, then provide the rules for creating these derived profiles.  These profile rules provide instructions for building profiles (e.g., requirements on structure, functionality, encoding, etc.). Derived profiles should not contradict predefined profiles, if there are any in the base specification.

Why care? Well-defined rules will facilitate the creation of derived profiles that are well-specified, implementable, andtestable, and can foster a better interoperability across profiles.  If these rules are defined and followed, then the derived profile will conform to the specification.

  1. Create two particular profiles
  2. Identify the common requirements in these two profiles
  3. Identify the rules that you have followed to create these profiles and write them donedown
  4. Try to create a third profile using the defined rules

WebCGM 1.0 [WEBCGM10] defines rules to create Profiles (See section 1.2 in [WEBCGM10])

4.2 Optionality and Options

Options in specifications provide implementationsers the freedom to make choices about

These choices also called discretionary items, give implementers of the technology the opportunity to decide from alternatives when building their applications and tools.They describe or allow optionality of behavior, functionality, parameter values, error handling, etc. They may be considered necessary because of environmental conditions (e.g., hardware limitations or software configuration), locality differences (e.g., language or time zones), dependencies on other technologies, or the need for flexibility.

Although there are perceived benefits to providing optionsoptional features, there is also a downside: optionsoptional features increase the variations that can exist among implementations. The greatest way to undo the utility of a specification is with too many optionsoptional features. Basically, different implementations may specifyuse different combinations of optionsoptional features. This makes comparisons between implementations difficult as well as complicates conformance testing and increases the chance of non-interoperable implementations.

ISSUE: Lynne: “Suggest changing the Story to an Example.  We don’t have any other Stories in this document.”

A concise XSLT 1.0 [XSLT10] example: implementers have been granted separate discretion about all of the following aspects of creating attributes in the output:

  1. Whether to raise an error when the supplied name is not a valid QName
  2. Whether to raise an error when an attempt is made to create an attribute after children of the element have been created
  3. Whether to raise an error when an attempt is made to create an attribute on a node that is not an element
  4. Whether to raise an error when the content of an attribute is not plain text
  5. Whether to raise an error when two attribute-sets of the same precedence contain an attribute of the same name
  6. Whether to raise an error when attempting to create an attribute directly under the root of a result tree fragment

In each case, there is one prescribed behavior for an implementation that chooses not to raise an error. Thus, the 6 separate binary choices give rise to 64 different possible behaviors for conformant processors. Typically, an implementer would be content to make a more global choice about raising errors when there is an attempt to create non-well-formed XML results.

4.2 Good Practice A: Determine the need for each option. Make sure there is a real need for the option.Make sure there is a need for the optional feature.

What does it mean? Examine the reason for the optional feature - is it to address a real, existing need? Should it really be an optional or made a mandatory part of the specification? Be careful not to provide optionsoptional features in anticipation for something that sounds like a good idea but will probably never be realized - ask the implementers if they ever plan to need this. Think about the implications of both implementing the optional feature and of not implementing the optionit. Don't make something an option just because the Working Group can't decide on what to do or can't reach consensus. As the specification progresses, if the optionfeature is not implemented, consider removing it.

Why care? A concise list of optionsoptional features helps to keep the specification focused and will greatly increases the likelihood of obtaining interoperable solutions. Ensuring that only necessary opotionsoptional features are contained in the specification also makes the job of the implementer easier and reduces costs.


Tests were used to determine if optionsoptional features were needed. As part of its exit criteria for Candidate Recommendation, a Working Group created a set of tests to 'test the specification'. The tests were able to show where there was need for optionality (e.g., diversity among implementations and flexibility justified) and where it was possible to narrow the choices (e.g., implementations used a much narrower set of values than what was permitted).

4.2 Good Practice B: Make options easy to find. Use tags to label options.Clearly identify optional features.

What does it mean? When introducing an optional feature in the specification, label it as such, so that it's easy to find all the optional features defined in the specification.

Why care? Options can be useful, but non-judicious use of optionsoptional features increases the variability among conforming implementations. In order to minimize the unnecessary optionality, it is necessarya good idea to be able to easilyprovide an easy way to identify them. To make it easier for implementers, test creators and users to discover the choices and variability allowed. For test creators, having a name for each item and a link target for each one, allows systematic reference to individual items from test cases as well as elsewhere in the specification. In addition to its benefits for writing test cases, tThe use of labels for optionsoptional features also helps in constructing pro forma, conformance claims, comparisons between two implementations, reports to the W3C Director about implementations, etc.


Options can be tagged in many ways. Any technique that distinguishes the optional feature from the required feature is acceptable. Some possibilities include:




4.2 Good Practice C: Indicate any limitations or constraints on optional features.

CR: 5.3 Indicate any constraints on the number of choices/options that can be implemented for discretionary items. [Priority 2]

What does it mean? Provide as much information as possible to narrow the allowable choices and to increase predictability. For implementation dependent values or features, if possible, provide a range or set of permitted values rather than leaving it completely open. Design the extension mechanism to allow for the creation of extensions that do not interfere with conformance to the original specification.

Discuss the implications of either using the optional feature or not. It may be helpful to provide rationale for choosing one option over another or for not using the option at all. Consider the unintended consequence of the optional feature - on other optionsoptional features, other parts of the specification, on other specifications.

Why care? Narrowing choices and increasing predictability enhance the likelihood of interoperability since the sample space for the implementer to choose from is diminished. Narrow choices and providing more information also increases the chances of correct implementations since incorrect choices are eliminated. An enumerated list of values is one way to constrain the choice of optionality.


For optionsoptional features, especially enumerated lists, make sure that the number of choices/options that can be implemented is known. Specifically, can zero, exactly one, or several of the allowable choices be implemented? Does this number depend on other parts of the specification or other chosen options?

Questions to consider include:


CSS 2.0 [CSS20] defines often properties the constraints on the values that can take these properties. For instance, the property font-weight can have the following value and only these ones: “normal | bold | bolder | lighter | 100 | 200 | 300 | 400 | 500 | 600 | 700 | 800 | 900 | inherit” and a default value being “normal”

In XPath 2.0 [XPATH20] and its associated functions, the Working Group intends to require support for collation by Unicode code-point, and allow implementations to implement as many other collations as they wish.

4.2 Good Practice C: Address the impact of the option.

In discussion. See mailing list thread

What does it mean?Think about the implications of both implementing the optional feature and of not implementing the optionit

Discuss the implications of either using the optional feature or not. It may be helpful to provide rationale for choosing one option over another or for not using the option at all. Consider the unintended consequence of the optional feature - on other optionsoptional features, other parts of the specification, on other specifications.

Why care? Thinking about implications of optionsoptional features forces the specification developerscreators to seriously consider the utility of each option, which may eventually lead to the elimination of unnecessary optionsoptional features. Discussiong the implications of using the option or not and providing rationale as to why to choose one option over another allows implementers to make informed choices, and to understand the consequences, both intended and unintended, of their decisions.


Questions to consider include:

  • What affect will implementing the optional feature have on a conforming implementation, on interoperability?
  • Is there any affect or negative consequence if the option is not implemented?
  • Must the option be implemented?
  • Is there a default value for the option?
  • Are there any dependencies, that is, the option can only be implemented if certain conditions are true?
  • Are the optionsoptional features mutually exclusive, that is, can one option override and void another?



4.3 Extensibility and Extensions

To accommodate changes in technology and information on the Web, a specification can be designed for extensibility. A specification is extensible when it provides a mechanism to allow anyan external party to create extensions. Extensions incorporate additional features beyond what is defined in the specification. However, extensions can compromise interoperability if there are too many differences between implementations. The impact of extensions can be mitigated through features specifically designed to allow new functionality. These features provide a 'standard way to be non-standard' by including hooks, conformance rules, or other mechanisms by which new functionality may be added in a conforming way, designated as extensibility mechanism.

4.3 PrincipleRequirement A: Address the extensibility topic inside specification.Address Extensibility.

What does it mean? Extensions might be encouraged or discouraged by the Working Group. There is a benefit to addressing the value or danger of extensibility for the technology which is currently being developed. Formalizing the position of the Working Group by a clear defined section and prose will removes ambiguities for the specification users about the possibility of developing extension or not.

Why care? There is a strong likelihood that developersimplementers will create extensions to awant to extend the functionalities defined in the specification, because they have specific needs which are not covered by the specificationit. Writing clearly how the extensions need to be developedDefining clearly how these extended functionalities can be hooked in conformance implementations will help ensure that extensions are defined in a consistent manner leading to predictable handling of extensions and minimize issues such as: interoperability problems, minimal support, and harmonious future development.

The Working Group may consider that the technology is complete, self-sufficient and and doesn't need any extensionsto be extensible. In this case, it is necessary to write this clearly in the specification and to explain why the technology is not considered extensible. It might be just for the social benefit of the community to ensure a maximum interoperability.

  1. Create a section of yourin the specification dedicated to the extensionsextensibility topic
  2. Call it Extensions
  3. Make a table of contents entry for it
  4. Address the following principles and good practices of this section

The specification CSS3 module: Syntax [CSS3-SYNTAX] has addressed the topic of extensions for CSS 3. It is clearly identified in the table of contents and there's a specific section for it.

4.3 Good Practice B: Define the extension mechanismIf extensibility is allowed, define an extension mechanism.

What does it mean? Extensions increase variability between implementations. Defining a mechanism will helps to ensure extensions are defined in a consistent manner leading to predictable handling of extensions, including the ability to take appropriate actions (e.g., do the extension, ignore, or take a fall-back behavior).

Why care? By planning for extensions and designing a specification for extensibility, there is less chance that extensions will interfere with conformance to the base specification and a better chance at achieving interoperability. On the other handConversely, there may be areas in a specification that would not benefit from extensibility and extensions are strictly forbidden (e.g., permissible characters in XML 1.0 names for elements and attributes, most of the built-in datatypes in Schema Part 2). This is called strict conformance. Strict conformance is defined as conformance of an implementation that employs only the requirements and/or functionality defined in the specification and no more.

Reasons for designing extensibility into a specification include:

  • providing a stable, useful framework to manage the rapid pace of technology change,
  • reducing the chances that extensions will interfere with conformance and increasing the chances for achieving interoperability,
  • providing the ability to 'test-drive' new functionality that may migrate into future specifications,
  • enabling implementers to include functionality that they consider to be required by their customers.

Technology and application needs will dictate the best strategy for enabling extensibility.

When designing for extensibility, it can get complicated. QuestionsPoints to consider that can impact design decisions include, but are definitely not limited to:

  • Does the conformance section address the use of extension? Address the use of extension in the conformance section.
  • Is there a well defined template to create exension?Give a well defined template to create extension.
  • Does this mechanism prevent the creation of extension conflicting with the semantics of the specification?Verify that this mechanism prevents the creation of extension conflicting with the semantics of the specification.
  • When needed, is the scope of extensions is defined? (e.g., Are extensions authorized only for certain parts of the technology?)When needed, define the scope of extensions. (e.g., Are extensions authorized only for certain parts of the technology?).
  • As an example, create a fake extension as an example for developersimplementers what would be the right way to do it
  • CanIndicate if extensions interact with other extensions.
  • CanIndicate if extensions can be combined or are mutually exclusive.
  • CanIndicate if extensions apply to a specification's profiles or modules and not the core part of the specification.
  • As an exercise, create a fake extension to illustrate problems that could arise interacting with the specification?
  • Can the extension mechanism be propagated? (e.g., Using the extension mechanism of Specification UNO, Specification TOO is created. Can this same mechanism be used create Specification TOOTOO, an extension of Specification TOO?)
Mechanism defined within the specification, extension indicator, error handling instructions

XSLT 1.0 [XSLT10] provides extension mechanisms that allow an XSLT style sheet to determine whether the XSLT processor by which it is being processed:

  • has implementations of particular extensions available,
  • and to specify what should happen if those extensions are not available.

It defines two boolean functions: function-available (QName) and element-available (QName) that must be present in every implementation. These functions inform the XSLT processor that there is an extension, therefore the XSLT processor can return a value of false and provides handling instructions (e.g., signal an error, perform fallback and not signal error), if the extension is not available.

WSDL 2.0 [WSDL20] defines binding extension elements which are used to provide information specific to a particular binding. It is a two-part extensibility model based on namespace-qualified elements and attributes. It provides the syntax and semantics for signaling extensions. Extension elements can be marked as mandatory, meaning that they MUST be processed correctly by the WSDL processor - i.e., either agree to fully abide by all the rules and semantics signaled by the extension or immediate cease processing (fault).

Mechanism based on another specification's extension mechanism

CC/PP exchange protocol based on HTTP Extension Framework [CCPP-EXCHANGE] defines a HTTP extension to exchange CC/PP descriptions effectively. The CC/PP exchange protocol conforms to HTTP/1.1 and is a generic extension mechanism for HTTP 1.1 [HTTP11] designed to inter-operate with existing HTTP applications. The CC/PP exchange protocol uses an extension declaration to indicate that an extension has been applied to a message and possibly to reserve a part of the header namespace. It provides rules for which of the HTTP Extension Framework [HTTP-EXTENSION] extension declaration strengths and extension declaration scopes to use and defines the syntax and semantics of the header fields.

OWL Reference [OWL-REF] is a vocabulary extension of RDF Semantics (See section 6 in [RDF-MT]). OWL imposes additional semantic conditions on RDF called semantic extensions of RDF. These semantic extensions conform to the semantic conditions for simple interpretations described in the RDF Semantics Recommendation. The OWL semantics is consistent with RDF semantics, but OWL when understood as RDF is 'incomplete' versus the same OWL when understood as OWL. Thus, by understanding OWL more is learned and nothing that is learned contradicts what is learnt by RDF alone.

Mechanism defined as conformance rules

In XHTML Modularization [XHTML-MOD], the conformance section defines a set of conformance rules to allow for the extension of XHTML's layout and presentation capabiltiies. This method uses the extensibility of XML without breaking the XHTML standardspecification. It defines the extension by using XML DTD or Schema which is uniquely identifiable, does not redefine behavior of the technology it extends, and the syntactic and semantic requirements are described and documented. Additionally, it cn have some organization and naming constraints defined by the technology it extends.

CSS3 module: Syntax [CSS3-SYNTAX] has an vendor specific extension mechanism perfectly described. The CSS Working Group has established a strict set of rules to define extensions which are:

A proprietary name should have a prefix consisting of:

  1. an underscore ("_") or a dash ("-"),
  2. the (possibly abbreviated) name of the company, organization, etc. that created it,
  3. an underscore or a dash.

4.3 Good Practice C: Warn implementers to create extensions that do not interfere with conformance.

What does it mean? Include in the specification a warning to those who are creating extensions that extensions should not contradict or negate conformance to the original specification.

Why care? This is probably obvious, but sometimes it makes sense to reinforce an important point. The intent is to uphold conformance regardless of whether there is an extension or not – if it conformed without the extension, then conformance should hold true with the extension.


Include statements in the specification such as:


In XSLT 1.0 [XSLT10], extension attributes (from other namespaces) can be present on the official XSLT elements, but they are prohibited from changing the specified behavior within the detectability of conforming behavior. Thus, an extension attribute can cause the element to perform faster but cannot change the result.

CSS3 module: Syntax [CSS3-SYNTAX] defines an extension mechanism which uses a not valid start character for identifiers in CSS, so it is guaranteed never to be used by any current or future level of CSS. CSS-conforming parsers will skip rules that contain identifiers with such a character. Therefore it is not authorized to redefine the properties defined in the specification.

4.3 Good Practice D: Define error handling for unknown extensions.

What does it mean? Include error handling instructions for when an extension is not available or understood.

Why care? When using a strict conforming application, users might have to deal with documents, data which are considered invalid because they are containing errors, or they are extended. Developers will need to know what must beis the expected the behavior of the application in such context.

A consistent error handling helps interoperability with regards to usability. There is the well known Postel law which saysstates Be liberal in what you accept, and conservative in what you sendBe Strict in what you produce, Be tolerant in what you accept. Though it has been shown that often this has been the start of a nightmare for implementers who have to deal with all the possible cases to recover from exotic documents (non conformant or extended). Giving a consistent error handling mechanism eases the work of developersimplementers.


There are typically two approaches: (see section 4.2.3 Extensibility from Architecture of the World Wide Web, First Edition [WEB-ARCH])

A good way to handle these two approaches is to have a way in the syntax to distinguish which behavior is expected, e.g., mustUnderstand attribute in Soap 1.2

Don't forget to address all yourthe classes of products. For example, an authoring tool and a rendering tool might have to behave in a different way or not.


Ruby Annotation [RUBY] Specification defines what parts must be ignored and displayed (See section 1.2.2 in [RUBY]) when an unknown element is met.

HTML 4.01 [HTML401] defines the behavior of user agents with regards to invalid documents (See section B.1 [HTML401])

4.4 Deprecation

The need for deprecation comes about when there are features (e.g., function argument, element, attribute) definesd in the specification that have become outdated and isare being phased out, usually in favor of a specified replacement. It may means for instance that the feature is still there and functional, but

  1. there is a better way of achieving the same thing and the Working Group prefers you use this better way is used
  2. or the feature is not being used and the Working Group wants to clean up the specification and eventually remove the feature.

From a user point of view, a deprecated feature is one that should not be used anymore, since it may be removed from future versions of the specification. Deprecated features are no longer recommended for use and may become obsolete and no longer defined in future versions of the specification.

4.4 PrincipleRequirement A: Identify each deprecated feature.Identify deprecated features.

What does it mean? When a new version of a technology evolves, it is important to provide a list of the features being deprecated from the previous version of the specification. This list serves to notify developersimplementers and users of the technology.

Why care? It helps developersimplementers know which features will be supersededare expected to be obsoleted in the next version of the technology. Moreover, it provides them time to remove it from their code and if appropriate implement an alternative method.This gives them the opportunity to adjust their implementation to phase out the relevant features, and provide new ways to accomplish the same task.

It helps users of the technology know which features to avoid using when they create documents.

  1. Create a list of all deprecated features that are not encouraged anymore.
  2. Create a dedicated section for it
  3. Create a entry in the table of content going to this list
  4. For each deprecated feature, create a link to the appropriate definition in the specification

HTML 4.01 [HTML401]: The specification has a full list of elements and attributes. The deprecation status is given in the two lists. There is an entry in the table of content to these two lists. Each element/attribute is linked to its definition in the specification

Avoid doing things like in Namespaces in XML 1.1 [NAMESPACES11], Section 2.2.2 Use of IRIs as Namespace Names in Namespaces in XML 1.1 discusses the deprecation of relative IRI references, although the information is difficult. Difficult to find, luckily this is a very short Rec.

4.4 PrincipleRequirement B: Define how to handle each deprecated feature.Define how deprecated feature is handled by each class of product.

What does it mean? By deprecating a feature, the Working Group indicates its desire that the feature disappear from a future version of the specification. The motivation may be to convert an old feature to a newer one or to remove an old, dangerous, redundant or undesirable feature. Regardless of the reason, it is important to define the affect this will havehas on implementations that may encounter this feature (e.g., consumer products such as user agents or producer products such as authoring tools). Will use of the deprecated feature be tolerated? Will it signal an error or a warning? Typically, it is expected that a deprecated feature would not affect a consumer (e.g. user agent), while a producer (e.g. authoring tool) should issue a warning.

Why care? Defining how deprecated features are handled provides a smoother transition for the users of the specified technology, and ensure more consistency of the behavior across implementations. It is also particularly important for implementations that needs to support different versions of the specification.

For instance, the specification may require that an implementation supports both the features of the new and the old specifications, or suggest a converting mechanism.

  1. Consider the effect of deprecation on all classes of products that implement the specification (e.g., authoring tools, converter, user agents).
  2. Specify the degree of support required for each deprecated feature.
  3. Define how it affects conformance its conformance consequence.

In Mathematical Markup Language (MathML) Version 2.0 (Second Edition) [MATHML20], MathML2.0 section describes deprecated MathML 1.x features in terms of MathML-output-conformant authoring tools, MathML-input-conformant rendering/reading tools, and MathML-roundtrip-conformant processors.

HTML 4.01 [HTML401]: In the conformance section of HTML 4.01, there is the definition of deprecation and what user agents should do. Though tThe behavior for other kind of products is not defined though.

User agents should continue to support deprecated elements for reasons of backward compatibility.

4.4 Good Practice C: Explain how to avoid using a deprecated feature.

What does it mean? Deprecating a features implies that its The use of a deprecated feature is discouraged, often because there is a better technique already available to achieve the same result. For each deprecated feature, it is necessary to explain how implementers and users can avoid using it. It might be helpful to give additional information providing insight into the deprecation motivation.

Why care? Examples and information about each deprecated features will help users to smoothly evolve toward the new version of the technology, understanding its benefits. It will help to haveThis enables a faster adoption of the technology.

It will helps developersimplementers understand the rationale for implementing the new technology, to implement an alternative mechanism, and to tool tips or conversion scenarios to help users with the transition.

  1. For each deprecated feature, give one or more examples showing the old way and the new way. If the new way is explained in another document, you might want to give a link to this document, though don't forget that often readers benefit of having everything in one place.
  2. For each deprecated feature, give an explanation why the Working Group has decided to deprecate the feature. What are the benefits?
  3. Say if the given example is a generic mechanism or a specific example. Users of the technology might adopt the specific example in their implementations and it willmay limit the use of the feature.

Namespaces XML 1.1 [NAMESPACES11] deprecation of IRI references includes a link to the deprecation ballot results, providing background information on the proposal to deprecate, what this means with respect to conformance to XML 1.0 and Namespaces as well as the affect on other specifications (i.e., DOM, XPath).

HTML 4.01 [HTML401] gives numerous examples on how to avoid the markup which was used in previous versions for deprecated elements.

4.4 Good Practice ED: Identify obsolete features.

What does it mean? A feature which has been deprecated in a previous version of the specification was at risk of being removed. Making it obsolete indicates that the feature is no longer in use and removed from this presente current version of the technology. Providing a list of obsolete features informs developersimplementers and users what is no longer in the specification.

Why care? It gives a clear message to users and developersimplementers that obsolete features are forbidden and not part of the specification anymore. It helps avoid the creation of documents mixing old and new techniques which willwould be invalid.

It helps avoid name clashing. When an extension to a technology is created, developersimplementers willare likely to use syntax token for their extended features name. Giving the name of obsolete features will helps developersimplementers to avoid using the names of previous features which are now obsolete.

  1. Create a list of all features that are obsolete.
  2. Create a dedicated section for it.
  3. Create a entry in the table of content going to this list.
  4. For each deprecated feature, create a link to the appropriate definition in the previous specification.

HTML 4.01 [HTML401]: The specification has a full list of elements and attributes. The obsolete status is given in the two lists. There is an entry in the table of content to these two lists. Each element/attribute is linked to its definition in the specification

HTML 4.01, Appendix A: Changes lists the elements that are obsolete and suggests an alternative element for use. The following elements are obsolete: LISTING, PLAINTEXT, and XMP. For all of them, authors should use the PRE element instead.

4.5 Error Handling?

4.5 Good Practice A: Define an error handling mechanism.

What does it mean? For a language, address what effect an error (be it syntactic or semantic) in the input has to a processor of this language. For a protocol, address how should behave a party to this protocol when a bogus message is received.

Why care? There are many reasons a system may fail; to make a technology reliable, it is crucial to define how it should react to bogus input. Defining error handling also makes it possible for a user of the technology to react appropriately to the error condition.


Different methodologies exist to handle errors in a technology:


In HTML 4.01 [HTML401], the specification doesn't define an error handling policy, although it encourages a "mustIgnore" policy.

XML 1.0 [XML10] is well-known for its strictness with error conditions, where an ill-formed XML document must not be processed.

HTTP 1.1 [HTTP11] specification defines a set of well-known error, standardized through their error codes.

the XSLT 1.0 specification allows a processor to recover to some of the defined errors, http://www.w3.org/TR/1999/REC-xslt-19991116#conformance

5. Do Quality Control

One Working Group's document entered Candidate Recommendation prior to a through quality review, resulting in a huge number of issues to resolve and the eventual retreat back to Working Draft for major revisions. Many of the comments could have been prevented, such as: inconsistency and incompleteness of the document (e.g., links to supplementary materials which did not exist or were not complete, overlapping and conflicting requirements), undefined terminology, and unimplementable requirements.

As all editors know, the work is not finished when the writing is completed. In fact, various review and checks of all W3C documents are required prior to their publication and advancement. There are many things that can be done to improve the quality and clarity of the document, aside from the W3C process and publication rules and before Last Call reviews. Many of these things can be an integrated part of the specification development. Ensuring a quality document prior to external reviews can save time and energy in that the Working Group may get fewer comments and issues to resolve.

Example: When defining a language (such as XHTML) some requirements are defined in a natural language (e.g., English) while others are defined using a formal specification. Often there is overlap between the two approaches - especially with XML Languages where a DTD or an XML Schema can be used for two purposes: to define the syntactic requirements set by the language and/or to allow validating documents conforming to the said language. This overlap can lead to contradictions between the prose and the formal language, i.e., the prose says more than the formal language, formal language is more constraining than the prose, or the formal language says something different than the prose.

When a Working Group is developing a technology, each Working Group individual memberparticipant could be tempted to add as many features as possible (because it will besounds cool, because it is needed here or there, etc.). But adding these features doesn't mean:

5 PrincipleGood Practice A: Define an internal publication and review process.

What does it mean? Define (formally or not) an internal process for reviewing and developing new parts of the specifications, and how they appear in the drafts published as Technical Reports. Each time, a version of the document is published, the Working Group mustshould ensure that each individual section, whether a full chapter or simply the explanation of a given feature, is coherent and complete, or documents what is missing.

Why care? The more the specification work is organized, the more chances you get control on development process of your specification, the more chances to move smoothly across W3C Process, and to have a better final product. Setting up an internal publication/review process allows to avoid recurrent errors in the document, and allows a wider participation to the editorial work, making it possible to develop the specification faster.Publishing a specification with incomplete section is verycan be damaging at many levels :

  • Image of the Working Group
  • Understanding of the technology
  • Possibility of good review and comments from people outside the Working Group.

All these issues will tend to slow down the process and the advancement of the development of the technology.


Make it fun to do quality control on the specification, by providing tools and templates. For instance:

  1. At the start of the work on the specification, create guidelines to help peopleWorking Group participants progress on the technology and write submissions for the specification.
  2. Follow some or all of these following good practices
  3. when publishing a version with incomplete sections,If you really need to put an incomplete section, make it clear
    • that it's incomplete
    • what the expectations on comments are — are comments and contributions desired, or should reviewers ignore the section for now?
  4. Divide the work in small units, so people can see regular progress
  5. Quality can be fun, make it fun.

QA Specification Guidelines: For the edition of each Good Practice and PrincipleRequirement of QA Specification Guidelines, a template and a method has been given to make the edition of each part easy to follow and discuss by the QA Working Group and easy to incorporate in the final document for the editor.

5 Good Practice B: Do a systematic and thorough review.

What does it mean? Each part of a technology should be reviewed by the Working Group before publication, but also during the editing phase. It will helps the Working Group to identify missing pieces, spelling mistakes, ambiguities, dependencies. With a well defined review process inside the Working Group, it should not be difficult to accomplish this task.

Why care? When a Working Draft is published with incomplete or very raw sections, the Working Group might receive a lot of comments which will have to be answered at best or the incomplete text will go unnoticed and appear in the final Recommendation at worst.

  1. Create a simple and light review process. Perhaps, establish a team, where each person focuses on a different aspect of the specification's correctness.
  2. Organize at least one review cycle before publication (more if you canpossible)
  3. Organize the review by topic and expertise. For example, each person checks a different part of the specification or do it by expertise - the grammarian checks for consistency, grammar, spelling, readability, the test builder checks requirements for precision and implementability, the conformant checks that conformance criteria exists.

XHTML 1.0: If you submit a web page address to the W3C MarkUp Validator, it could happen that the result page tells you that your document is "Valid XHTML 1.0 Strict" and provides a link to the XHTML 1.0 Recommendation. If you follow this link you will encounter a problem: There is no definition of what it means for a document to be identified as "Valid XHTML 1.0 Strict".

For QA Specification Guidelines, a template ([QA-SPEC-TEMPLATE], archived mail) was produced to guide authors, ensuring that each principlerequirement and good practice was written consistently and addressed the same set of information. Once a principlerequirement or good practice was written, the text was circulated to the entire Working Group for comments and at the same time, a specific member of the Working Group was assigned to review the text. This ensured that at least someone in the Working Group had the reviewing responsibility and it would not fall through the cracks. Multiple authors produced the QA Specification Guidelines, following a template

5 Good Practice C: Write sample code or tests.

What does it mean? For each feature, the Working Group might seek early implementation to demonstrate the feature. If early implementations are not available (e.g., due to commercial constraints, IPR, etc.), it is beneficial to write test cases to illustrate a concept or use case of the technology. Additionally, these test cases can be incorporated into a test suite.

Why care? Developing a partial implementation (sample code) or test cases can provide an understanding of a concept or feature as well as help to keep it focused. It can save the Working Group and eventually implementers time and resources by:

  1. Encourage developersimplementers (Open Source or Commercial) to work with the Working Group to get pre-implementation at the same time the technology is developed.Encourage the development of proofs of concept implementations of the technology.
  2. For each feature, request that Working Group members provide at least one test case demonstrating the use of the feature.Provide at least one example of the feature, which may also be used as the basis of a future test case.
  3. Do not put a feature into a specification without the corresponding test cases.
  4. Create an authoring template for how to define/describe a feature which include a test case section.Create a template for new feature proposals that includes a request for associated test cases.

The OWL WG has synchronized the publication of their specification and the publication of the OWL Test Cases [OWL-TEST]. They even went a bit further by making the test case, the necessary step to develop a feature with its requirements.

@@CSS editors are encouraged (required?) to provide tests with all new content.@@

5 Good Practice D: Write Test Assertions.

What does it mean? A test assertion is a statement of behavior, action, or condition that can be measured or tested. It is contained within or is derived from the specification's requirements and provides a normative foundation from which one or more test cases can be built.

Why care? Test assertions facilitate the development of consistent, complete specifications and promote the early development of test cases. This exercise helps to uncover inconsistencies, ambiguities, gaps, and non-testable statements in the specification. It can provide early feedback to the editors regarding areas that need attention. An added benefit is that the assertions can be used as input to test development efforts.

  1. Create an authoring template for how to define/describe a feature which includes a section for adding test assertions.Create a template for new feature proposals that includes a section for adding test assertions.
  2. Identify all requirements in yourthe specification and try to write corresponding test assertions.
  3. Try toand write assertions as you add features gets added to a specification. If you can't write thea test assertion can't be written for the feature it suggests that there is a problem in the way you haveit is designed or explained the feature.

When trying to write test assertions for all its conformance requirements, one Working Group found that informative information was needed to complete the normative requirement. Consequently, the conformance requirements were rewritten.

SOAP version 1.2 Test Assertions [SOAP12-TA]

Assertion x1-conformance-part2

Location of the assertion: SOAP 1.2 Part 1, Section 1.2

Text from the specification: The implementation of an Adjunct MUST implement all the pertinent mandatory requirements expressed in the specification of the Adjunct to claim conformance with the Adjunct.

Comments: This statement applies to all assertions in part 2 and as such will not be tested separately.

HTML 4.01 Test Suite [HTML401-TEST]

Assertion 6.14-1

Reference: Section 6.14

(must) Script data ( %Script; in the DTD) can be the content of the SCRIPT element and the value of intrinsic event attributes. User agents must not evaluate script data as HTML markup but instead must pass it on as data to a script engine.

Tests: 6_14-BF-01.html

XML Test Suite [XML-TEST]

Section: Documents

Type: Well_Formed

Purpose: A well formed document must have one or more elements one

Level 1

5 Good Practice E: Use formal languages and define which from prose and formal languages has priority.

What does it mean? If an existing formal language (e.g. DTD, Schemas, ...) is expressive enough to describe the technical requirements of the specification, try to use it, and make it clear when the English prose and the formal language overlaps which one should be held for true in case of discrepancy.

Why care? English prose can often leads to ambiguity; using a formal language to describe conformance requirements can allow to avoid some of this ambiguity, while permitting to re-use existing tools for the given language to facilitate testing and validation.

But prose remains necessary to allow implementers to understand the specification, as well as to express additional requirements the formal language cannot express; this means that there are possible overlaps between the prose and the formal language, in which case, it is important to define which one is the main point of reference in case of disjunction.

  • There are plenty of formal languages used across W3C specifications: DTD, XML Schema, Relax NG, EBNF, Z Notation, ... Picking the right one depends on the kind of specifications developed (language, XML or not, protocol) and the use needed from the formal language.
  • To avoid discrepancies between the English prose and the formal language, set up a process so that a given section is bound to a given part of the formal language, and one can't modified without the other.
  • Use the formal language tools to validate the examples given in the specification, so as to ensure they match.
  • When using several formal languages in combination, generate random content according to the rules defined in one of them and try to validate it with the others, to find discrepancies.

XQuery Formal Semantics Notice section 1.1, where they defines where theis document is normative over the grammar specs (separate for XPath and XQuery) and where the grammar specs are normative.

Conformance to this document

The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY ", and "OPTIONAL" are used as defined in RFC 2119 [RFC2119] in this conformance clause. Occurrences of these words in lowercase have normative implications.

Conformance criteria

Conformance to the QA Framework: Specification Guidelines is very simple. 

The conformance requirements of this document are denoted as PrincipleRequirements.  All PrincipleRequirements are written in imperative voice, with the assumed “you must” in front of the statement.  In addition to PrincipleRequirements, the document contains Good Practices.  Good Practices are optional and considered recommendations.  Their implementation or non-implementation does not affect conformance to this Specification Guidelines document.  

To conform to this Specification Guidelines, all PrincipleRequirements must be implemented.

One way to satisfy the PrincipleRequirements and Good Practices is to implement one or more of the suggested techniques given for each PrincipleRequirement and Good Practice.  Note that this is not the only way to satisfy the PrincipleRequirement or Good Practice.  An Implementation Conformance Statement is provided for assistance in keeping track of which PrincipleRequirements and Good Practices are implemented.  It takes the form of a checklist [QA-SPEC-ICS].  If all the PrincipleRequirements are checked as being satisfied, then conformance can be claimed. 

Normative Parts

The normative parts of this specification are identified by markup and style as well as the labels, ‘normative’ and ‘informative’ within sections. The following list indicates the normality of parts in this specification.

Normative and informative status of the different parts
Part Status
PrincipleRequirements Normative
Good Practice Informative
What does this mean? Informative
Why Care? Informative
Techniques Informative
Examples Informative

Text that is designated as normative is directly applicable to achieving conformance to this document.  Informative parts of this document consist of examples, extended explanations, and other matter that contains information that should be understood for proper implementation of this document. 

Specification Guidelines Extensibility

This specification is extensible.  That is, adding conformance related information and structure to the document in ways beyond what is presented as PrincipleRequirements in this specification, is allowed and encouraged.  Extensions to this specification must not contradict nor negate the requirements in this specification.

Conformance claims

To claim conformance to the QA Framework: Specification Guidelines, Working Groups must specify:

You can claim conformance to this document by using this conformance claim and replacing the content holders delimited by square brackets by the appropriate values:

On [date of the publication], this specification [name of the specification], edited by [name of the publishing entity], is conformant to Specification Guidelines WD, November 22, 2004 published at http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/.


The following QA Working Group and Interest Group participants have contributed significantly to the content of this document:


Normative References

IETF RFC 2119: Key words for use in RFCs to Indicate Requirement Levels, S. Bradner, March 1997. Available at http://www.ietf.org/rfc/rfc2119.txt.

Informative References

CC/PP exchange protocol based on HTTP Extension Framework , Hidetaka Ohto, Johan Hjelm, Editors, W3C Note, 24 June 1999, http://www.w3.org/1999/06/NOTE-CCPPexchange-19990624 . Latest version available at http://www.w3.org/TR/NOTE-CCPPexchange .
Composite Capability/Preference Profiles (CC/PP): Structure and Vocabularies 1.0 , L. Tran, F. Reynolds, M. H. Butler, C. Woodrow, J. Hjelm, H. Ohto, G. Klyne, Editors, W3C Recommendation, 15 January 2004, http://www.w3.org/TR/2004/REC-CCPP-struct-vocab-20040115/ . Latest version available at http://www.w3.org/TR/CCPP-struct-vocab/ .
Specification Guidelines Conformance Clause Template, Lofton Henderson, Lynne Rosenthal, 30 August 2004, available at http://www.w3.org/QA/2004/08/SpecGL-template-root.html .)
Cascading Style Sheets, level 2 (CSS2) Specification , H. Lie, I. Jacobs, C. Lilley, B. Bos, Editors, W3C Recommendation, 12 May 1998, http://www.w3.org/TR/1998/REC-CSS2-19980512 . Latest version available at http://www.w3.org/TR/REC-CSS2 .
CSS3 module: Syntax , L. Baron, Editor, W3C Working Draft (work in progress), 13 August 2003, http://www.w3.org/TR/2003/WD-css3-syntax-20030813 . Latest version available at http://www.w3.org/TR/css3-syntax .
CSS Test Suite Documentation, Tantek Çelı̇k, Ian Hickson, 29 January 2003, http://www.w3.org/Style/CSS/Test/testsuitedocumentation-20030129.html Latest version available at http://www.w3.org/Style/CSS/Test/testsuitedocumentation.html .
Document Object Model (DOM) Level 3 Core Specification , P. Le Hégaret, G. Nicol, L. Wood, M. Champion, J. Robie, S. Byrne, A. Le Hors, Editors, W3C Recommendation, 7 April 2004, http://www.w3.org/TR/2004/REC-DOM-Level-3-Core-20040407 . Latest version available at http://www.w3.org/TR/DOM-Level-3-Core/ .
W3C Editors' Home Page , Communication Team, Available at http://www.w3.org/2003/Editors/ .
QA WIKI Extension Speclite , Available at http://esw.w3.org/topic/ExtensionSpeclite .
HTML 4.01 Specification , A. Le Hors, D. Raggett, I. Jacobs, Editors, W3C Recommendation, 24 December 1999, http://www.w3.org/TR/1999/REC-html401-19991224 . Latest version available at http://www.w3.org/TR/html401 .
HTML 4.01 Test Suite , Available at http://www.w3.org/MarkUp/Test/HTML401/ .
IETF An HTTP Extension Framework , H. Nielsen, P. Leach, S. Lawrence, February 2000 Available at http://www.ietf.org/rfc/rfc2774.txt .
IETF RFC 2616: Hypertext Transfer Protocol — HTTP/1.1, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, T. Berners-Lee, June 1999. Available at http://www.ietf.org/rfc/rfc2616.txt.
W3C Manual of Style Susan Lesch, Available at http://www.w3.org/2001/06/manual .
Mathematical Markup Language (MathML) Version 2.0 (Second Edition) , P. Ion, R. Miner, D. Carlisle, N. Poppelier, Editors, W3C Recommendation, 21 October 2003, http://www.w3.org/TR/2003/REC-MathML2-20031021/ . Latest version available at http://www.w3.org/TR/MathML2/ .
Namespaces in XML 1.1 , R. Tobin, T. Bray, A. Layman, D. Hollander, Editors, W3C Recommendation, 4 February 2004, http://www.w3.org/TR/2004/REC-xml-names11-20040204 . Latest version available at http://www.w3.org/TR/xml-names11/ .
OWL Web Ontology Language Reference , M. Dean, G. Schreiber, Editors, W3C Recommendation, 10 February 2004, http://www.w3.org/TR/2004/REC-owl-ref-20040210/ . Latest version available at http://www.w3.org/TR/owl-ref/ .
OWL Web Ontology Language Test Cases , J. De Roo, J. J. Carroll, Editors, W3C Recommendation, 10 February 2004, http://www.w3.org/TR/2004/REC-owl-test-20040210/ . Latest version available at http://www.w3.org/TR/owl-test/ .
Comprehensive glossary of QA terminology. (Under construction, at http://www.w3.org/QA/glossary.)
The QA Handbook, L. Henderson, Editor, W3C Working Group Note (work in progress), 22 November 2004, http://www.w3.org/TR/2004/NOTE-qa-handbook-20041122/ . Latest version available at http://www.w3.org/TR/qa-handbook/ .
QA Specification Guidelines ICS, Karl Dubost, 22 November 2004, http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/specgl-ics.html
QA Specification Guidelines Template, Karl Dubost, 10 June 2004, http://lists.w3.org/Archives/Public/www-qa-wg/2004Jun/0023.html
RDF Semantics , P. Hayes, Editor, W3C Recommendation, 10 February 2004, http://www.w3.org/TR/2004/REC-rdf-mt-20040210/ . Latest version available at http://www.w3.org/TR/rdf-mt/ .
Ruby Annotation , T. Texin, M. Suignard, M. Ishikawa, M. Sawicki, M. J. Dürst, Editors, W3C Recommendation, 31 May 2001, http://www.w3.org/TR/2001/REC-ruby-20010531 . Latest version available at http://www.w3.org/TR/ruby/ .
Synchronized Multimedia Integration Language (SMIL 2.0) Specification , E. Hodge, P. Hoschka, K. Kubota, J. van Ossenbruggen, E. Hyche, M. Jourdan, L. Rutledge, B. Saccocio, W. ten Kate, P. Schmitz, R. Lanphier, N. Layaïda, J. Ayars, T. Michel, D. Bulterman, D. Newman, A. Cohen, K. Day, Editors, W3C Recommendation, 7 August 2001, http://www.w3.org/TR/2001/REC-smil20-20010807/ . Latest version available at http://www.w3.org/TR/smil20/ .
SOAP version 1.2 Test Assertions, Available at http://www.w3.org/TR/2003/REC-soap12-testcollection-20030624/ .
Mobile SVG Profile: SVG Tiny, Version 1.2 , T. Capin, V. Hardy, O. Andersson, Editors, W3C Working Draft (work in progress), 13 August 2004, http://www.w3.org/TR/2004/WD-SVGMobile12-20040813/ . Latest version available at http://www.w3.org/TR/SVGMobile12/ .
Mobile SVG Profiles: SVG Tiny and SVG Basic , T. Capin, Editor, W3C Recommendation, 14 January 2003, http://www.w3.org/TR/2003/REC-SVGMobile-20030114/ . Latest version available at http://www.w3.org/TR/SVGMobile/ .
Scalable Vector Graphics (SVG) 1.1 Specification , 藤沢, J. Ferraiolo, D. Jackson, Editors, W3C Recommendation, 14 January 2003, http://www.w3.org/TR/2003/REC-SVG11-20030114/ . Latest version available at http://www.w3.org/TR/SVG11/ .
Variability in Specification , Dominique Hazaël-Massieux, Lynne Rosenthal, W3C Working Draft 30 August 2004, http://www.w3.org/TR/2004/WD-spec-variability-20040830/ . Latest version available at http://www.w3.org/TR/spec-variability/ .
W3C Glossary and Dictionary , http://www.w3.org/2003/glossary/
Web Content Accessibility Guidelines 1.0 , G. Vanderheiden, W. Chisholm, I. Jacobs, Editors, W3C Recommendation, 5 May 1999, http://www.w3.org/TR/1999/WAI-WEBCONTENT-19990505/ . Latest version available at http://www.w3.org/TR/WAI-WEBCONTENT .
Architecture of the World Wide Web, First Edition , N. Walsh, I. Jacobs, Editors, W3C Proposed Recommendation (work in progress), 5 November 2004, http://www.w3.org/TR/2004/PR-webarch-20041105/ . Latest version available at http://www.w3.org/TR/webarch/ .
WebCGM 1.0 Second Release , R. Platon, J. Gebhardt, L. Henderson, D. Weidenbrueck, D. Cruikshank, Editors, W3C Recommendation, 17 December 2001, http://www.w3.org/TR/2001/REC-WebCGM-20011217/ . Latest version available at http://www.w3.org/TR/REC-WebCGM .
QA WIKI Formal Language vs Prose , Available at http://esw.w3.org/topic/FormalLanguageVsProse .
QA WIKI Extensibility Good Or Bad , Available at http://esw.w3.org/topic/ExtensibilityGoodOrBad .
QA WIKI Normative References , Available at esw.w3.org/topic/NormativeReferences .
QA WIKI Meaning versus Behavior , Available at http://esw.w3.org/topic/MeaningVsBehavior .
QA WIKI RFC Keywords , Available at http://esw.w3.org/topic/RfcKeywords .
QA WIKI Testable Or Not , Available at http://esw.w3.org/topic/TestableOrNot .
Web Services Description Language (WSDL) Version 2.0 Part 1: Core Language , J. C. Schlimmer, R. Chinnici, M. Gudgin, S. Weerawarana, J. Moreau, Editors, W3C Working Draft (work in progress), 3 August 2004, http://www.w3.org/TR/2004/WD-wsdl20-20040803 . Latest version available at http://www.w3.org/TR/wsdl20 .
Modularization of XHTML™ , M. Altheim, F. Boumphrey, S. McCarron, S. Schnitzenbaumer, T. Wugofski, S. Dooley, Editors, W3C Recommendation, 10 April 2001, http://www.w3.org/TR/2001/REC-xhtml-modularization-20010410/ . Latest version available at http://www.w3.org/TR/xhtml-modularization/ .
XHTML™ 1.0 The Extensible HyperText Markup Language (Second Edition) , S. Pemberton, Editor, W3C Recommendation, 1 August 2002, http://www.w3.org/TR/2002/REC-xhtml1-20020801 . Latest version available at http://www.w3.org/TR/xhtml1 .
XML Schema Part 1: Structures Second Edition , D. Beech, M. Maloney, H. S. Thompson, N. Mendelsohn, Editors, W3C Recommendation, 28 October 2004, http://www.w3.org/TR/2004/REC-xmlschema-1-20041028/ . Latest version available at http://www.w3.org/TR/xmlschema-1/ .
XML Schema Part 2: Datatypes , A. Malhotra, P. V. Biron, Editors, W3C Recommendation, 2 May 2001, http://www.w3.org/TR/2001/REC-xmlschema-2-20010502/ . Latest version available at http://www.w3.org/TR/xmlschema-2/ .
Extensible Markup Language (XML) Conformance Test Suites , Available at http://www.w3.org/XML/Test/ .
Extensible Markup Language (XML) 1.0 (Third Edition) , E. Maler, J. Paoli, F. Yergeau, T. Bray, C. M. Sperberg-McQueen, Editors, W3C Recommendation, 4 February 2004, http://www.w3.org/TR/2004/REC-xml-20040204 . Latest version available at http://www.w3.org/TR/REC-xml .
XML Path Language (XPath) 2.0 , D. Chamberlin, M. Kay, J. Siméon, M. F. Fernández, J. Robie, A. Berglund, S. Boag, Editors, W3C Working Draft (work in progress), 29 October 2004, http://www.w3.org/TR/2004/WD-xpath20-20041029/ . Latest version available at http://www.w3.org/TR/xpath20 .
XQuery 1.0 and XPath 2.0 Formal Semantics , M. F. Fernández, K. Rose, D. Draper, M. Rys, J. Siméon, P. Wadler, P. Fankhauser, A. Malhotra, Editors, W3C Working Draft (work in progress), 20 February 2004, http://www.w3.org/TR/2004/WD-xquery-semantics-20040220/ . Latest version available at http://www.w3.org/TR/xquery-semantics/ .
XSL Transformations (XSLT) Version 1.0 , J. Clark, Editor, W3C Recommendation, 16 November 1999, http://www.w3.org/TR/1999/REC-xslt-19991116 . Latest version available at http://www.w3.org/TR/xslt .

Further Reading

The following references have been inspirational to the ideas captured in this document.


Class of Products
generic name for the group of products or services that would implement, for the same purpose, the specification, (i.e., target of the specification). A specification may identify several classes of products.
fulfillment by a product, process, systems, or service of a specified set of requirements.
Conformance clause
a section of the specification that defines the requirements, criteria, or conditions to be satisfied by an implementation in order to claim conformance.
Deprecated feature
An existing feature that has become outdated and is in the process of being phased out, usually in favor of a specified replacement. Deprecated features are no longer recommended for use and may cease to exist in future versions of the specification.
Dimensions of Variability (DoV)
the ways in which different products that are conformant to a specification may vary among themselves.
Discretionary Items
deliberate and explicit grants of discretion by the specification to the implementations, that describe or allow optionality of behavior, functionality, parameter values, error handling, etc.
The ability to incorporate additional functionality beyond what is defined in the specification. It broadens the possibility of the technology.
The ability of a specification to accept extensions in a define way. A specification is extensible if it provides a mechanism for any party to create extensions
An implementation is a realization of a technology in accordance to the principles defined in the technical specifications for this technology. This implementation can be a document, product, application, process, service, system, or other entity.
A technology subset that is one of a hierarchy of nested subsets, ranging from minimal or core functionality to full or complete functionally.
A collection of semantically-related features that represents a unit of functionality.
A subset of a technology that is tailored to meet specific functional requirements of a particular application community.
Obsolete feature
An existing or deprecated feature has ceased to exist and that is listed for historical purpose.
Strict Conformance
conformance of an implementation that employs only the requirements and/or functionality defined in the specification and no more (i.e., no extensions to the specification are implemented).
A proposition is testable if there is such a procedure that assesses the truth value of a proposition with a high confidence level. Whether the confidence level is measurable and what confidence level is high enough depends on the proposition and its framework. (@@ better definition?)
Test Assertion
a statement of behavior, action, or condition that can be measured or tested. It is derived from the specification's requirements.

History of changes

October 2004

August 2004

July 2004

June 2004 - QA WG F2F in Santa Clara