W3C

QA Framework: Specification Guidelines

QA WG Working Draft 18 August 2003

This version:
http://www.w3.org/QA/WG/2003/08/qaframe-spec
Latest version:
http://www.w3.org/QA/WG/qaframe-spec/
Previous version:
http://www.w3.org/QA/WG/2003/07/qaframe-spec
Editors:
Dominique Hazaël-Massieux (dom@w3.org)
Lofton Henderson (lofton@rockynet.com)
Lynne Rosenthal (lynne.rosenthal@nist.gov)
Dimitris Dimitriadis (dimitris@ontologicon.com)
Kirill Gavrylyuk (kirillg@microsoft.com)
Contributors:
See Acknowledgments.

Abstract

The principal goal of this document is to help W3C Working Groups to write clearer, more implementable, and better testable technical reports. It provides both a common framework for specifying conformance requirements and definitions, and also addresses how a specification might allow variation among conforming implementations,, both of which facilitate the generation of test materials. The material is presented as a set of organizing guidelines and verifiable checkpoints. This document is one in a family of Framework documents of the Quality Assurance (QA) Activity, which includes the other existing or in-progress specifications: Introduction; Operational Guidelines; and, Test Guidelines.

Status of this document

This is the WG version of the SpecGL document. It's set for discussions among the WG members towards a future publication in TR space.

The draft changes are more visible in the running editor's version.

This section describes the status of this document at the time of its publication. Other documents may supersede this document. The latest status of this document series is maintained at the W3C.

This document is a W3C Working Draft (WD), made available by the W3C Quality Assurance (QA) Activity for discussion by W3C Members and other interested parties. For more information about the QA Activity, please see the QA Activity statement.

This Working Draft is published for Last Call review. This version incorporates the closure of all open QAWG (QA Working Group) issues about this specification, some new checkpoints and rewording of existing ones. For details, please see Change history.

This version supersedes all previous drafts. Future progression of this document beyond Working Draft is planned, but the final status has not been determined at this time. See QAWG issue #18 and issue #71. It is anticipated that this specification will eventually advance to Candidate Recommendation (CR), after successful discussion and resolution of any and all issues that arise during Last Call review. The QAWG has discussed criteria for finishing CR phase and entrance to Proposed Recommendation (PR). The agreed criteria are: for each Priority 1 and Priority 2 checkpoint, two example implementations that successfully fulfill the checkpoint.

This part of the Framework document family has a companion QA Framework: Operational Examples & Techniques. That informative companion document is currently being progressed as a W3C Note. At least until this guidelines document stabilizes, that Examples & Techniques companion will be maintained and frequently updated in QA Working Group Web space (as opposed to /TR/). Although that document is not the principal subject of this review, the QAWG welcomes feedback on it as well.

The QA Working Group Patent Disclosure page contains details on known patents related to this specification, in conformance with W3C policy requirements.

Please use this form to make your comments. If for some reason you are unable to use the form, you may email comments to www-qa@w3.org, the publicly archived list of the QA Interest Group [QAIG]. Please note that comments that you make will be publicly archived and available, do not send information you would not want to see distributed, such as private data.

Publication of this document does not imply endorsement by the W3C, its membership or its staff. This is a draft document and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use W3C Working Drafts as reference material or to cite them as other than "work in progress".

A list of current W3C Recommendations and other technical documents can be found at http://www.w3.org/TR/.

Table of contents

  1. Introduction
  2. Concepts
  3. Guidelines
  4. Conformance
  5. Definitions
  6. Acknowledgments
  7. References
  8. Change history

Two appendices to this document [SPEC-CHECKLIST] and [SPEC-ICS] present all checkpoints in a tabular form sorted in their original order and sorted by priority, for convenient reference.


1. Introduction

1.1. Scope and Goals

The scope of this document is a set of requirements for W3C Technical Reports (TRs) that if satisfied, will enhance the clarity, implementability, and testability of TRs. It describes what goes into a TR with respect to conformance and conformance topics, dealing with how a TR establishes, defines, and presents its conformance policy. This document addresses the most basic and critical topics with respect to conformance and testabily; more advanced topics will be addressed in future versions. This document includes:

A separate document, entitled "Specification Examples and Techniques 1.0" (the "ExTech document" from here on) provides suggestions and examples of how each checkpoint might be satisfied. The techniques in the ExTech document are informative examples only, and other strategies may be used or required to satisfy the checkpoints. The ExTech document is expected to be updated more frequently than the current guidelines. Developers, W3C Working Groups, users, and others are encouraged to contribute techniques and examples.

1.2. Class of Product and Audience

The class of product or target of this specification is W3C Technical Reports. Within this Specification Guidelines document, the term "specifications" is specifically limited to W3C Technical Reports, even though these guidelines could be applied to other documents.

The primary target audience of this document is specification authors, however, it is applicable to a broader audience including:

These guidelines are designed so that the WGs can apply them in a common-sense and workable manner.

1.3. Motivation and expected benefits

Good specifications lead to better implementations and foster the development of conformance test suites and tools. Conforming implementations lead to interoperability.

The quality of the specification has direct impact on the quality of implementations. Quality encompasses utility which refers to the usefulness of the specification to the intended users and objectivity which focuses on the whether the specification is presented in an accurate, clear, complete, and unbiased manner.

Providing requirements and definitions about conformance topics, as well as guidance in the structure and anatomy of specifications, will foster a mutual understanding among those developing specifications, implementations, and conformance test materials. Specifically, well-structured specifications with clear and comprehensive conformance requirements:

1.4. Relationship to other specifications

This document is part of a family of QA Framework documents designed to help the WGs improve all aspects of their quality practices. The QA Framework documents are:

Although the QA Framework documents are interrelated and complement each other,they are independently implementable. For example, the anatomy of a specification is related to the type of test materials that will be built, hence there is interrelationship between this document and the Test Guidelines. The reader is strongly encouraged to be familiar with the other documents in the family.

The Framework as a whole is intended for all Working Groups, as well as developers of conformance materials for W3C specifications. Not only are the Working Groups the consumers of these guidelines, they are also key contributors. The guidelines capture the experiences, good practices, activities, and lessons learned of the Working Groups and present them in a comprehensive, cohesive set of documents for all to use and benefit from. The objective is to reuse what works rather than reinvent and to foster consistency across the various Working Group quality activities and deliverables.

This document does not preclude the need to apply the W3C Manual of Style [STYLE-MAN] and to conform to the Publication Rules [PUBRULES] (Member-only). It is intended to complement those resources. .

1.5. Understanding and using this document

This document employs the WAI (Web Accessibility Initiative) model for representing guidelines or general principles for the development of conformance materials. See, for example, Web Content Accessibility Guidelines 1.0 [WCAG10]. Each guideline includes:

The guidelines are of two general types:

The checkpoints in each guideline define specification characteristics and requirements needed to fulfill the purpose of the guideline. Each checkpoint definition includes:

Each checkpoint is intended to be specific enough so that someone can implement the checkpoint as well as verify that the checkpoint has been satisfied. A checkpoint will contain at least one, and may contain multiple individual requirements , that use RFC2119 normative keywords. See the "Conformance" chapter for further discussion of requirements and test assertions.

Two separate appendices to this document [SPEC-CHECKLIST] and [SPEC-ICS] present all checkpoints in a tabular form sorted in their original order and sorted by their priorities, for convenient reference. The latter is an Implementation Conformance Statement (ICS) pro-forma for this specification. (See GL12.)

1.6. Usage of terminology in this document

The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY ", and "OPTIONAL" will be used as defined in RFC 2119 [RFC2119]. When used with the normative RFC2119 meanings, they will be all uppercase. Occurrences of these words in lowercase comprise normal prose usage, with no normative implications.

Unusual terms in these framework documents are defined when first used. When used in this specification, terms have the meaning assigned in the "Definitions" chapter and the QA Glossary [QA-GLOSSARY]. Terms in Definitions may supplement or build on the definitions in the generic QA Glossary, further refining the definition in the context of the Specifications guidelines. Terms herein that also appear in the QA Glossary include a link to the QA definition.They will not contradict the generic definitions.

1.7. Checkpoint priorities

Some checkpoints are more critical than others for producing a high quality, testable standard that is a sound basis for successfully interoperable products. Therefore each checkpoint is assigned a priority level based on the checkpoint's impact on the quality of the specifications produced by the Working Groups.

[Priority 1]
Critical/Essential. These checkpoints are considered to be basic requirements for ensuring an acceptable level of quality and testability in the specification.
[Priority 2]
Important/desirable. Satisfying these checkpoints, in addition to the priority 1 checkpoints, should significantly improve the clarity, quality, and testability of the standard, as well as its suitability as a basis for building quality test materials.
[Priority 3]
Useful/beneficial. Satisfying these checkpoints, on top of all the others, will further improve the quality, usability, and testability of the specification.

1.8. Use cases

The Specification Guidelines are intended to be used in the following typical cases:

2. Concepts

This section details some of the concepts that are used in the following guidelines. The reader is invited to become familiar with them before further reading.

2.1. Dimensions of Variability (DoV)

Each of the latter set of eight guidelines GL2 through GL9 addresses a way in which the conformance policy of a specification might allow variation among conforming implementations. For example, a specification might allow implementations to choose between one of two well defined behaviors for a given functionality, thus two conforming implementations might vary on that aspect.

The Working Group has identified eight ways in which specification can allow variability, that are refered as dimensions of variability (DoV). The eight dimensions of variability recognized by this document are:

The above are not necessarily all orthogonal to one another. There are many possible associations, dependencies, and interrelationships. As a general policy, these specification guidelines do not attempt to legislate correct or proper relationships among the DoV. Rather, they try to clarify the nature of each dimension, and require specification to make deliberate and well documented choices. Some discussion of possible interrelationships, including examples, will be found in the Specification Examples & Techniques.

The dimensions of variability are one of the principal concepts in this guidelines document to help organize, classify and assess the conformance characteristics of W3C specifications. The eight DoV get special attention because, since they are at the core of the definition of a specification's conformance policy, there is significant potential for negative interoperability impacts if they are handled carelessly or without careful deliberation.

As a general principle, variability complicates interoperability. In theory, interoperability is best when there are numerous identical, complete, correct implementations. However, in practice, the net effect of conformance variability is not necessarily negative in all cases, when compared to the alternatives. For example profiles — subdivisions of the technology targeted at specific applications communities — introduce variability among implementations. Some will implement Profile ABC, some will implement Profile XYZ, and the two might not intercommunicate well if ABC and XYZ are fairly different. However, if ABC and XYZ are subsets of a large monolithic specification — too large for many implementors to tackle in total -- and if they are well targeted at actual application sectors, then subdivision by profiles may actually enhance interoperability.

Different sorts of variability have different negative and positive impacts. The principal danger is "excessive" variability - variability which goes beyond that needed for positive interoperability trade-off, and which unnecessarily complicates the conformance landscape. Specification writers need to carefully consider and justify any conformance variability allowed, do so by reference to the project requirements and use cases, and explicitly document the choices made.

Note that the variability addressed by the so called dimensions of variability is only considered with regard to conformance with a well-defined specification. As such, the changes introduced in the conformance requirements between two versions or two editions of the specification are not considered as dimensions of variability.

2.2. Specification category and class of product

To answer the question "what needs to conform?" it helps to first look at the nature of the specification and categorize it and then look at the types of products that would implement the specification.Categorizing the specification provides a basis for classifying the software that may be affected by the specification. The specification category is the generic name for the type of specification and the technology it describes.

The following list is a non-exhaustive list of specification categories:

  1. foundation or abstract (e.g., Infoset),
  2. content/data (e.g., MathML, SVG),
  3. protocols (e.g., SOAP),
  4. processors (e.g., XSLT, XML Query),
  5. APIs (e.g., DOM),
  6. notation/syntax (e.g., XPath),
  7. set of events (e.g., one part of XForms)
  8. rules for deriving profiles (e.g., part of SMIL)

The categories indicate what the specification describes. One specification could potentially fall into more than one category.

From this categorization of specifications, the WG can identify the class of products that are affected by the specification. Classes of products can be generalized as either producers or consumers, or as content itself. Identifying which are producers and consumers is clear for a protocol-type specification, the two parties to the dialog are the targets. For a processor-type specification, the processor is the consumer of an XML vocabulary defined in the specification. For content-type specifications, there may be one or more consumers that take the content and 'play' it in some way.

The following list is a non-exhaustive list of classes of products for W3C specifications.

One specification could define more than one player. For example, MathML addresses the behavior of display of math notation and also a consumer that parses the MathML as a formula and uses it for mathematical processing.

2.3. Profiles, Modules, Levels

Profiles, modules and levels are three ways to subdivide a specification in related groups of conformance requirements. Because these three dimensions of variability define subsets of a technology, they share some characteristics in the way they affect conformance and interoperability.

A profile is a subset of the technology that supports a particular functional objective or a subset of a set of technologies defining how they are required to operate together (e.g., XHTML plus MathML plus SVG).

Profiles can be based on hardware considerations associated with target product classes — for example, SVG Tiny is aimed at mobile phones — or they may be driven by other functional requirements of their target constituencies — for example, a graphical profile tailored for technical illustrations in aircraft maintenance manuals.

The use of profiles to divide the technology is described in the specification, and may or may not be reflected and paralleled by the structure and organization of the specification.

Specifications may define individual profiles, or may define rules for profiles, or both. An individual profile defines the requirements for classes of products that conform to that profile. Rules for profiles define validity criteria for profiles themselves — i.e., if others (users, applications, or other standards) define their own profiles of the standard (called derived profiles of the specification), then rules for profiles define the constraints that those derived profiles must satisfy in order to be considered valid profiles of the specification.

For example, XHTML Modularization ([XHTML-MOD], section 3) and Synchronized Multimedia Integration Language (SMIL 2.0), [SMIL20] specifications both define rules for profiles -- what constraints must a profile meet in order to be classified as a "Host Language Profile" or an "Integration Set Profile." SMIL further defines some specific profiles, using the modularization. Separate recommendations -- XHTML Basic [XHTML-BASIC] and XHTML 1.1 [XHTML11] — define specific profiles based on the XHTML modularization.

Modules are discrete divisions or functional groupings of the technologyand do not necessarily fit in a simple hierarchical structure.

Modules generally can be implemented independently of one another — e.g., audio vs. video module. That notwithstanding, it is possible for one module's definition (and therefore implementation) to have explicit dependency upon another module. It is not only possible, but common to implement multiple modules.

Functional levels — or in common usage, simply levels — are used to group functionality into nested subsets, ranging from minimal or core functionality to full or complete functionally. Level 1 is the minimum or core of the technology. Level 2 includes all of level 1 plus additional functionality. This nesting continues until level n, which consists of the entire technology.

Levels may result from progressive historical development and enrichment of the technology in a series of specifications, as in the case of CSS and DOM. Levels could also be defined explicitly in a single edition of the specification, but no examples of this are found in W3C specifications. Rather, it is more common in current W3C practice to use profiles to accomplish this. For example, SVG 1.1 [SVG11] together with SVG Mobile [Mobile [SVG-MOBILE] define three nested profiles — Tiny, Basic, Full — which are each targeted at a specific graphics hardware community (mobile phone, hand-held computer, desktop computer).

2.4. Addressing relationships among Dimensions of Variability used

As stated above, variability affects interoperability and it is important for the specification writers to have this in mind while designing the specification. But it is even more important to take into account the multiplicative effect on variability created by combining several dimensions of variability.

Hence, each pair of dimensions of variability used in a specification needs to be assessed with regard to the variability it creates; the writers should document the limited ways an implementation can combine two dimensions. For instance, deprecated features in HTML 4.01 [HTML4] are allowed in the Transitional profile, and forbidden in the Strict profile.

3. Guidelines

Guideline 1. Define Scope.

When writing specifications it is critical to understand their primary purpose and scope. Clearly defined scope helps to keep the specification content focused and unambiguous.

Checkpoints

Checkpoint 1.1. Include a scope statement [Priority 1]
Definition

The scope describes the areas covered by the specification, thereby indicating the intent or applicability of the specification. It does not specify requirements and is worded as a series of statements of fact.

Conformance requirements

The specification MUST define the subject matter of the specification, and SHOULD include a statement of objectives and/or goals. This information MUST appear at the beginning of the document.

Rationale

It helps the writer and the reader know the limits of what is specified in a normative document. It tells the reader how the specification could be used and by whom.

Discussion

Note that defining the classes of product as required by the checkpoint 2.1 contributes to defining the scope of a document.

Examples & Techniques for this checkpoint.

Checkpoint 1.2. Illustrate scope [Priority 2]
Conformance requirements

The specification MUST provide examples or use cases illustrating what is in scope.

Rationale

A set of broadexamples and/or use cases helps to clarify the specification's scope. It gives the reader additional information in understanding the purpose, audience, and applicability of the specification.

Discussion

It is up to the Working Group to determine the best way to illustrate the scope, i.e., to use examples or use cases or both.

Examples & Techniques for this checkpoint.

Checkpoint 1.3. Illustrate functional details [Priority 2]
Conformance requirements

The specification MUST provide one or more examples of the functionality, concepts, and behaviors of the specification.

Rationale

It is difficult to understand some concepts, behavior, or functionality without informative interpretations to aid the reader. These examples are specific in nature and are used to make clear a concept, behavior, interaction, etc.that is innately complex or for which the WG has had to explain to its members or the public.

Discussion

The nature of the specification will influence the type of examples that are relevant. For example, for markup specifications, provide at least one example of each markup construct; for transformation specifications, illustrate each transformation capability with an example showing input and output.

Examples & Techniques for this checkpoint.

Examples & Techniques for this checkpoint.

Guideline 2. Identify what needs to conform and how.

Identifying what needs to conform to the specification is essential for the specification writers as well as for the implementors. This guideline relies on the concepts of classes of products and specification categories.

Checkpoints

Checkpoint 2.1. Identify classes of product. [Priority 1]
Conformance requirements

A specification:

Rationale

Doing so helps define the scope of the specification. It also helps the reader know what is being targeted by the specification (i.e., what needs to conform to the specification).

Example

Example. Within the SMIL 2.0 Language Profile ([SMIL20], chapter 13), there are 2 classes of products: documents and basic user agents. Within Mathematical Markup Language (MathML) 2.0 [MATHML20] there are output-compliant, authoring tools and input-compliant, rendering/reading tools.

Examples & Techniques for this checkpoint.

Checkpoint 2.2. For each class of product, define the conformance requirements. [Priority 1]
Definition

The conformance requirements indicate the conditions to be satisfied in order to claim conformance. It is likely that these conformance requirements will reference normative text within the specification or in other related specifications.

Conformance requirements

The specification MUST define the conformance requirements for each class of product identified in checkpoint 2.1.

Discussion

The list of classes of products enumerate the most common classes of products. If your class of product matches one or more terms in the list, then use the terms in the list. If no term is an appropriate match, then define your own.

Examples & Techniques for this checkpoint.

Checkpoint 2.3. Identify the applicable specification categories [Priority 3]
Conformance requirements

The specification MUST identify in its Introductory section the specification category or categories.

Rationale

Understanding the specification category that is being written contributes towards understanding the classes of products, as required by the preceding checkpoint. Moreover, it helps both the author(s) and readers understand the scope of the specification and its focus.

Discussion

If your specification matches one or more a categories in the list, then use those categories. If no category is an appropriate match, then define your own.

Example

Some specifications define more than one of the enumerated category types. XForms [XFORMS] is an example. It defines: Content, via an XML vocabulary; Content, via named datatypes; Syntax, in the form of a set of functions to supplement the XPath core set; behavior of a processor; behavior of a user agent; a set of events.

Examples & Techniques for this checkpoint.

Checkpoint 2.4. If there are several classes of products, define their relationships and interaction with other dimensions of variability. [Priority 2]
Conformance requirements

The specification MUST address and discuss the relations and interactions between classes of product and the other dimensions of variability. It is not applicable if there is only one class of product.

Examples & Techniques for this checkpoint.

Guideline 3. Address the use of profiles, modules and levels to divide the technology.

To address various needs (re-usability, evolution, scaling, ...), technologies are often divided in different ways. These divisions have an important impact on the conformance to a specification. This guideline relies on the concepts of profiles, modules and levels that the QA WG has identified as being the three main division types in use.

Checkpoints

Checkpoint 3.1. Indicate whether or not the use of profiles is mandatory for conformance. [Priority 1]
Conformance requirements

A specification

It is not applicable if profiles are not used.

Example

For example, is content required to conform to one of the profiles, or is there a concept of conformance of content independent of conformance to one of the profiles? Is a producer (of content) conforming if it generates content that is otherwise valid but does not conform to a profile?

An example of additional conditions on profiles would be to require that only one profile can be implemented at a time.

Discussion

Note: If there is a "full" profile defined — for example, incorporating all of the defined functionality of the specification, including extensibility features — then any valid content, as well as any correct producers and fully capable consumers, might seem to be automatically using that profile. However, profiles (e.g., of content) often include self-identification requirements, and these would have to be observed for conformance of valid content to even a "full" profile.

Examples & Techniques for this checkpoint.

Checkpoint 3.2. If profiles are chosen, define any minimal requirements. [Priority 2]
Conformance requirements

The specification MUST define for each profile the minimal required features/support for each identified class of product. It is not applicable if profiles are not used.

Rationale

Because profile places explicit requirements on each class of product that have specific and often limited operating environments these requirements must be defined for each class of product that is affected.

Example

To illustrate, SMIL 2.0 [SMIL20] has a SMIL 2.0 Language Profile for user agents but also provides a SMIL 2.0 Basic Profile for wireless and embedded devices. The SMIL 2.0 Language Profile requires that a user agent implement the BasicAnimation module but not the SplineAnimation Module. The SMIL 2.0 Basic Profile on the other hand does not require implementation of any of the Animation modules.

Examples & Techniques for this checkpoint.

Examples & Techniques for this checkpoint.

Checkpoint 3.3. If profiles are chosen, address rules for profiles. [Priority 2]
Conformance requirements

If the specification allows the creation of derived profiles, the specification MUST provide requirements for derived profiles and these requirements MUST be testable. It is not applicable if profiles are not used.

Rationale

Well-designed rules for profiles will facilitate the creation of derived profiles that are well-specified, and testability will enable an independent tester to verify conformance of a derived profile to the rules.

Discussion

Note that experience shows that requirements for derived profiles should impose at least these two rules on derived profiles: derived profiles should be specified in a way that meets all the pertinent checkpoints of this document (QA Framework: Specification Guidelines); and, derived profiles should not contradict predefined profiles, if there are any in the base specification.

Examples & Techniques for this checkpoint.

Checkpoint 3.4. Define the relationships between profiles, modules, levels and other dimensions of variability [Priority 2]
Conformance Requirements

The specification MUST address and discuss the relationships between profiles, modules, levels and the other dimensions of variability used.

Examples & Techniques for this checkpoint.

Checkpoint 3.5. If modules are chosen, indicate any mandatory conditions or constraints on their usage. [Priority 1]
Conformance requirements

The specification MUST document any identified mandatory conditions or constraints on their usage. Such conditions include,

It is not applicable if modules are not used.

Discussion

The conditions or constraints normally will be tailored according to class of product.

Examples & Techniques for this checkpoint.

Examples & Techniques for this checkpoint.

Checkpoint 3.6. If profiles, modules or levels are used, define their relationships and interaction with other dimensions of variability. [Priority 2]
Conformance requirements

The specification MUST address and document any identified relationships and interaction with other dimensions of variability. It is not applicable if profiles, modules or levels are not used.

Example

Dependency or interrelationship between profiles and modules is common -- XHTML [XHTML-MOD], SMIL [SMIL20], SVG 1.1 [SVG11]. Less often, deprecated features, levels, discretionary choices, or extensions could depend on profiles.

Discussion

Often there is dependency or interrelationship among modules, on the one hand, and profiles or discretionary choices on the other. Modules may have levels or deprecated features. Extensions could be defined based on modules.

Levels can be dependent on, or apply to, modules. Less often, there can be a relationship between levels, on the one hand, and profiles or deprecated features on the other.

Examples & Techniques for this checkpoint.

Guideline 4. Identify the relation between deprecated features and conformance.

A deprecated feature is an existing feature that has been outdated by newer constructs or is no longer viable. Deprecated features should be avoided (e.g. for a format language, not be used by the authors and authoring tools) and may be removed in some future version, at which time the feature becomes obsolete.

After the initial publication of a specification, specification developers may consider the deprecation of a feature (e.g., function argument, element or attribute) defined in the specification. Deprecated features may become obsolete and no longer defined in future versions of the specification. Deprecation of a feature may warn implementers that the feature was a bad idea and it may be withdrawn in the future. Specification developers need to consider the effect of deprecation on all the classes of products that implement the specification (e.g., authoring tools, user agents) as well as the conformance consequences on each class of product. For the purpose of backward compatibility, it may be necessary to specify different requirements for the support of deprecated features for each class of product. For example, authoring tools (producers) would not use the feature, but user agents (consumers) would continue to support it.

Checkpoints

Checkpoint 4.1. Identify each deprecated feature. [Priority 1]
Conformance requirements

The specification MUST document in a normative section each deprecated feature. It is not applicable if there are no deprecated features.

Rationale

It helps the reader find the deprecated features by presenting them as a collection in one place rather than distributed throughout the document. This collection may be derived from distributed deprecated features and may take the form of a a list of links to where the features appear in the document.

Checkpoint 4.2. For each class of product, specify the degree of support required for each deprecated feature and the conformance consequences of the deprecation. [Priority 1]
Conformance requirements

The specification MUST specify the degree of support required for each deprecated feature for each class of product and the conformance consequences of the deprecation. It is not applicable if there areno deprecated features.

Example

For example, a deprecated-features section of MathML 2.0 ([MATHML20], section 7.2.1.2) describes, about deprecated MathML 1.x features, that MathML-output-compliant authoring tools may not generate MathML markup containing deprecated features; whereas MathML-input-compliant rendering/reading tools must support deprecated features.

Examples & Techniques for this checkpoint.

Checkpoint 4.3. If deprecated features exist, define their relationships and interaction with other dimensions of variability. [Priority 2]
Conformance requirements

The specification MUST indicate, for each deprecated feature, either (1) that it is independent of all other DoV or (2) discuss the relationship between the feature and each of the other DoV.

Rationale

Depending of the breadth of the feature being deprecated, it may impact other DoV. The effect may be contained (e.g., within a single class of product or modules), be wide ranging (e.g., across all classes of products), be a DoV (e.g., deprecate an entire module), or introduce changes to a DoV (spawn new discretionary choices).

Examples & Techniques for this checkpoint.

Checkpoint 4.4. Include an explanation for the deprecation. [Priority 3]
Conformance requirements

The specification MUST document each deprecated feature with a rationale for deprecation. It is not applicable if there are no deprecated features.

Rationale

Providing the rationale for deprecating a feature helps implementers and users to understand the motivation for the deprecation, the impact and consequences on current and future implementations, and perhaps insight into its eventual disappearance from the specification.

Examples & Techniques for this checkpoint.

Checkpoint 4.5. Include examples to illustrate how to avoid using deprecated features. [Priority 3]
Conformance requirements

The specification MUST provide an example for each deprecated feature showing how to avoid using it. It is not applicable if there are no deprecated features.

Rationale

Examples are helpful in providing alternatives or better ways to get the same results. Showing what can be done in place of the deprecated feature will help get implementers to discontinue use of the deprecated feature.

Discussion

Note that this checkpoint only makes sense for specifications that define the behavior of a producer.

Examples & Techniques for this checkpoint.

Checkpoint 4.6. Identify each obsolete feature. [Priority 3]
Conformance requirements

The specification MUST document each obsolete feature. It is not applicable if there is no obsolete features.

Rationale

Obsolete features are listed for historical purposes. There is no guarantee of support for obsolete featrues by implementations of the specification.

Example

Obsolete features can be listed in the Change section of a specification, e.g., HTML 4.01.

Examples & Techniques for this checkpoint.

Guideline 5. Define discretionary items.

Discretionary items are defined as deliberate and explicit grants of discretion by the specification, to the implementations, that describe or allow optionality of behavior, functionality, parameter values, error handling, etc.

Discretionary items are often made available in specifications, to give implementers of the technology the opportunity to decide from alternatives when building applications and tools. Discretionary items may be considered necessary because of environmental conditions (e.g., hardware limitations or software configuration, or external systems), locality (e.g., time zone or language), optional choices providing flexibility of implementation, dependence on other specifications, etc.

Discretionary items come in three basic variants:

discretionary choices
a value or behavior may be chosen from a well-defined enumerated set of two or more possibilities;
optional features
a well-defined feature may be supported or not (if supported, then the requirements are clear and unambiguous)
implementation dependent values (or features)
it is open ended and undefined, what set of values an element or attribute may have, or the behaviors of a product that implements a feature, etc

Checkpoints

Checkpoint 5.1. State the circumstances for when discretionary items are allowed [Priority 1]
Conformance requirements

The specification MUST indicate the rationale for the discretionary items and MUST identify by labeling all discretionary items. It is not applicable for specifications that do not have discretionary items.

Rationale

Doing this helps readers, implementers and testers to find these discretionary items and understand the need for them.

Discussion

Note that references to use cases and project requirements are usually a good way to justify discretionary items.

Examples & Techniques for this checkpoint.

Checkpoint 5.2. For implementation dependencies, address the allowable differences between implementations [Priority 1]
Conformance requirements

The specification MUST describe any permitted variations or constraints for how the implementation dependency is realized by implementations.

Example

Examples of permitted variations or constraints to be addressed include:

Examples & Techniques for this checkpoint.

Checkpoint 5.3. Indicate any constraints on the number of choices/options that can be implemented for discretionary choices [Priority 2]
Conformance requirements

The specification MUST indicate, for each identified discretionary item, whether zero, exactly one, or several of the choices/options are allowed to be implemented. If the allowable number is dependent on other dimensions of variability, the dependencies MUST be stated. It is not applicable for specifications that do not use discretionary items.

Rationale

The test framework can provide a tailoring mechanism so that each implementation is tested with a set of tests tailored to the choices made by the implementer. The tailoring mechanism for choose-exactly-one items will not be the same as the tailoring mechanism for choose-one-or-more items. The tailoring mechanism needs to accommodate modules, profiles, and levels, when they are used, which may affect the range of choices on some discretionary items (and may drop the the range to zero in some situations).

Example

Examples of constraints include mandating that an implementation implement only one choice, implement every choice (allow the user to choose), be allowed to implement none of the choices, or be required to implement one specified value and as many additional values as they wish from an open-ended list. An example of a constraint being dependent on another dimension of variability is a rule that there must be exactly one instance of the item for each profile that is supported.

Examples & Techniques for this checkpoint.

Checkpoint 5.4. Promote consistent handling of discretionary choices. [Priority 2]
Conformance requirements

The specification MUST document the identified policies for handling discretionary choices

Rationale

This helps identify where the specification could actually factorize these policies, so that implementations could consistently handle discretionary choices - users have an expectation of what to expect and should be able to count on getting the same results under the same conditions with a given implementation.

Discussion

Note that the Working Group believes that given identical conditions, the effect of a discretionary choice should be consistent within a single implementation, and thus, that specifications should enforce it.

Examples & Techniques for this checkpoint.

Checkpoint 5.5. If discretionary items are used, define their relationships and interaction with other dimensions of variability. [Priority 2]
Conformance requirements

The specification MUST address and discuss the relations and interactions between deprecated features and all the other DoV.

Examples & Techniques for this checkpoint.

Guideline 6. Allow extensions or not

An extension to a specification is a mechanism to incorporate functionality beyond what is defined in the specification. Extensions can be implemented individually (e.g., one function at a time) or by defining new profiles, modules, or levels that incorporate additional functionality beyond what is defined in the specification.

Allowing extensions affects how conformance is defined as well as what conformance claims can be made. Exercise caution in determining the extent to which extensions are allowed or not allowed. Since extensions can seriously compromise interoperability, specification writers should carefully consider whether extensions should be allowed.

Disallowing extensions for any part of the specification is called strict conformance. Strict conformance is defined as conformance of an implementation that employs only the requirements and/or functionality defined in the specification and no more (i.e., no extensions to the specification are implemented). Dimensions of variability (e.g., modules, profiles, levels) are not extensions if the specification defines them or allows them to be defined.

Extensions may be private (often vendor specific) or public (a full description of the extension is public). Private extensions are usually truly private, i.e., valid for a specific implementation or are only known by prior agreement between implementations. Public extensions are extensions in which the syntax, semantics, identifiers, etc. are defined and published allowing anyone to implement the extended functionality.

Specifications allow extensions for various reasons. Extensions allow implementers to include features that they consider to be required by their customers. Also, extensions often define new features that may migrate into future versions of the specifications. However, the use of extensions can have a severe negative impact on interoperability. Some methods for enabling extension have less impact on interoperability than other methods.

Checkpoints

Examples & Techniques for this checkpoint.

Checkpoint 6.1. Indicate if the specification is extensible, and if extensions are allowed, define their scope and constraints. [Priority 1]
Conformance requirements

The specification MUST state

and MUST document the rationale for allowing extensions by referencing use cases and/or project requirements.

Rationale

Readers should be able to understand the motivation for the inclusion of an extension and its intended use. Documenting the scope and rationale for extensions helps assess the impact of extensions on interoperability.

If the specification writer wants to enhance interoperability by constraining implementer extensions, wording in the specification must indicate this.

If extensions are not allowed, then it should be clear to the reader that not only are extensions not allowed, but the circumstances under which they are not allowed. Strict conformance is often imposed on applications or content (e.g., a software program or document instance). This prohibition of extensions could be applied to a specific profile or module, rather than to the entire specification.

Examples & Techniques for this checkpoint.

Checkpoint 6.2. Prevent extensions from contradicting the specification. [Priority 1]
Conformance requirements

The specification MUST state that extensions cannot negate or change support for required functionality. It is not applicable if extensions are not allowed.

Discussion

Requirements in the specification cannot be compromised or contradicted by adding extensions.

Examples & Techniques for this checkpoint.

Checkpoint 6.3. Define a uniform mechanism to create an extension [Priority 3]
Conformance requirements

The specification MUST provide a uniform way to define that extensibility is being invoked and MUST provide the syntax to be used to indicate the extension. It is not applicable if extensions are not allowed.

Rationale

This helps to ensure predictable handling of extensions, that is, its recognition as such and the appropriate actions (i.e., to ignore or to implement).

Examples & Techniques for this checkpoint.

Checkpoint 6.4. Require that extensions be published. [Priority 3]
Conformance requirements

The specification MUST require that the syntax and semantics of the extension be publicly documented. It is not applicable if extensions are not allowed.

Rationale

Public availability with a full description allows the extension to be implemented by anyone without prior arrangement. This enhances interoperability by allowing the same functionality to be uniformly implemented across different implementations.

Examples & Techniques for this checkpoint.

Checkpoint 6.5. Require that implementations provide interoperable alternatives to extensions [Priority 3]
Conformance requirements

The specification MUST indicate via conformance requirements that implementations provide a mode under which they produce only conforming content. This checkpoint is not applicable if extensions are not allowed. This checkpoint is only applicable to specifications that identify producer of content as one of its classes of products.

Rationale

This checkpoint can be used to ensure that there is a way to produce (generate) only conforming content.

Examples & Techniques for this checkpoint.

Checkpoint 6.6. If extensions are allowed, define their relationships and interaction with other dimensions of variability [Priority 2]
Conformance requirements

The specification MUST address and discuss the relations and interactions between extensions and all the other DoV.

Example

For example, a specification could define a strict conformance policy for one class of product, while allowing extensions on another class.

Examples & Techniques for this checkpoint.

Guideline 7. Clearly identify conformance requirements.

The checkpoints of this guideline aim to ensure that normative content and conformance requirements are easily identifiable in specifications. It is important that specifications provide unambiguous statements, that clearly define what is required in order to claim conformance and what is optional. It is important also that the conformance statements are easily identifiable as such, and easily locatable. Clarity is fostered by uniformity of structure and style, and consistency of terminology and phraseology.

There is a lot to be said for consistency and clarity within a document - it facilitates the understanding and comprehension of the document. Authors and editors of specifications should already be familiar with the Publication Rules (Member-only) [PUBRULES] and W3C Manual of Style [STYLE-MAN], which help to achieve clarity and uniformity. This guideline builds on those general document conventions, with a particular focus on the presentation and identification of conformance information.

Checkpoints

Checkpoint 7.1. Use conformance key words. [Priority 1]
Conformance requirements

The specification MUST use RFC 2119 keywords to denote whether or not requirements are mandatory, recommended, or optional.

Rationale

Using these keywords helps to identify the testable statements in a specification as well as those statements that are desireable or allowed. Guidance on the proper usage of these keywords is given in RFC 2119.

Examples & Techniques for this checkpoint.

Checkpoint 7.2. Distinguish normative and informative text. [Priority 1]
Definition

Normative statements are the prescriptive parts of the specification whereas informative statements are for informational purposes and assist in the understanding or use of the specification.

Conformance requirements

The specification MUST distinguish normative text from informative text.

Rationale

It is important that the reader be able to distinguish between normative and informative statements in order to know what is required to claim conformance. SMIL 2.0 is an example, indicating within every subsection whether it is normative or informative, and even separately labelling pieces of subsections that contain both kinds of text.

Examples & Techniques for this checkpoint.

Checkpoint 7.3. Use consistent terminology. [Priority 1]
Conformance requirements

The specification MUST use identical wording to express identical provisions, and analogous wording to express analogous provisions.

Rationale

Being consistent and following established conventions and W3C editorial practices will enhance the readability and comprehensibility of the specification.

Examples & Techniques for this checkpoint.

Checkpoint 7.4. Provide a fast way to find conformance information [Priority 2]
Conformance requirements

The specification MUST provide at least one navigation mechanism that allows the reader to locate by direct access, all conformance-related information that is relevant to the specification. The mechanism MUST minimally locate:

Rationale

A reader must be able to easily identify and locate all the information necessary to understand the conformance policy and related conformance information without having to read the document from cover to cover. A navigation mechanism adds to the usability of the specification.

Discussion

A table of contents entry is one way to accomplish this. In addition to the minimal required set above, other conformance related information such as the ICS, location of test suites, etc, may be helpful to users and implementers. Also see checkpoints requiring conformance information about the Dimensions of Variability that are used (e.g., Checkpoint 6.1).

Note that having this mechanism working in some way in hardcopy can be very useful for test suite developers.

Examples & Techniques for this checkpoint.

Checkpoint 7.5. Make normative reference to specifications on which the current specification depends [Priority 1]
Definition

A specification depends on another specification when it relies on or requires functionality (or behavior) from the other specification in order to work (function) correctly. This other specification provides a necessary condition or functionality.

Conformance requirements

The specification MUST have normative references to any identified specification it depends on and MUST describe the relationship between the specifications and any identified conformance implications

Rationale

Dependence on other specifications affects the conformance boundaries of the current specification, and thereby affects the requirements on conformant products.

Discussion

The linking parts of the Manual of Style ([STYLE-MAN], section 11.5.1) describe the recommended way of citing an external reference from the prose of a specification, as well as how to construct an entry in its References section.

Example

For example, SVG 1.0 requires that the class of product called "user agent" be consistent with the XML 1.0 Recommendation [XML10] and (conditionally) support Cascading Style Sheets, level 2 [CSS2].

Examples & Techniques for this checkpoint.

Guideline 8. Specify a conformance policy.

A look at various W3C Technical Reports shows that the term "conformance" is often qualified, resulting in more than one type of conformance. It is important to convey an understanding of what is meant by conformance and how it applies to each class of product as well as each dimension of variability (e.g., modules) if applicable. For example, if the specification defines behavior for more than one class of product, there may be a separate conformance policy for each class. Similarly, if the specification defines modules, there may be a different conformance policy for each module.

In particular, a reader intending to implement a product in one of the product classes addressed by the specification should know what the WG wants for interoperability among products in the class. The developer should understand what forms, if any, of "product differentiation" are allowed among conformant products. The specification may need to explain how the rules apply and provide examples.

A conformance clause is a part or collection of parts of a specification that defines the requirements, criteria, or conditions to be satisfied by an implementation or application in order to claim conformance. Typically the conformance clause is a high-level description of what is required of implementations and applications.

Checkpoints

Checkpoint 8.1. Specify any universal requirements for minimum functionality. [Priority 2]
Conformance requirements

The specification MUST include a normative section enumerating the minimal requirements that apply across conforming products of a class.

Rationale

The reader must be able to recognize any minimum functionality, or support that applies to conforming products of a specific class. These minimum requirements can be considered a starting point for a checklist to be used by a team developing a conforming product. It helps the reader find these requirements by presenting them as a collection, in one place.

Discussion

Usually, the individual requirements comprising the minima all occur elsewhere, as conformance requirements distributed throughout the document. In principle, the collection could therefore be derived from the distributed requirements, although such a derivation could be difficult. The collection of minimal requirements may apply to a single class of product or across classes of products.

Example

If levels are used (see Guideline 6), the lowest level may represent the minimum set of requirements. If profiles are used (see Guideline 4), there may be different minima for each profile. If modules are used (see Guideline 5), there may be different minima for each module. Furthermore, a module may itself be a minimum (i.e., required) for a particular class of product.

Examples & Techniques for this checkpoint.

Checkpoint 8.2. If special conformance concepts are used, include a definition in the specification. [Priority 1]
Conformance requirements

Any conformance concepts used in the specification MUST be defined, either by reference or by including the definition in the text.

Rationale

It is necessary to define concepts that govern application of the conformance provisions. Ideally, all concepts are from QA documents and other existing literature and need only be cited. If specialconcepts are constructed, such as to combine modules and levels or modules and discretionary choices, they need to be defined in the specification.

Examples & Techniques for this checkpoint.

Checkpoint 8.3. Justify any usage of a dimension of variability [Priority 1]
Conformance requirements

A specification MUST justify each Dimension of Variability the specification uses.

Rationale

DoV add complexity to a specification; explaining what's the usage of each of them can help implement and test them better, and can help reviewers understanding their necessity.

Discussion

References to usage scenarios and requirements are usually a good way to justify the uses of dimensions of variability.

Examples & Techniques for this checkpoint.

Checkpoint 8.4. Include a conformance section. [Priority 1]
Conformance requirements

The specification MUST document its conformance policy.

Rationale

Having the conformance clause exist as a separate section within the specification makes it clearly identifiable, allowing a reader to find the overall conformance policy, as well as all specific conformance provisions from a single starting point.

Examples & Techniques for this checkpoint.

Checkpoint 8.5. Identify and define all conformance designations. [Priority 1]
Conformance requirements

The specification MUST identify and define all the conformance designations used.

Example

In current W3C practice, a number of different naming conventions are used to label conformance, in cases where there is not a single, monolithic (strict) conformance definition. The naming convention used to label the conformance can provide useful information. Degrees, for example, implies incremental importance or difficulty. This Specification Guidelines document uses "degrees" for example, to refer to three successively more demanding degrees of conformance (A, AA, AAA).

Discussion

Commonly used conformance designations include categories, degrees, and levels. Use of "conformance levels" is discouraged in new specifications, because of the potential for confusion with "functional levels".

Examples & Techniques for this checkpoint.

Guideline 9. Specify how to make conformance claims.

A specification may differentiate conformance claims by designating different degrees or types of conformance in order to apply and group requirements according to modules, profiles, levels, or priorities. When a conformance claim is linked to functionality, impact and/or incremental degrees of implementation, the term 'conformance level' is often used to indicate the varying degrees of conformance. The WG includes in the specification the way they want people to claim their conformance.

An Implementation Conformance Statement (ICS) provides a mechanism whereby a supplier of an implementation of the specification provides information about the implementation in a standardized manner. It is used to indicate which capabilities and options have been implemented, as well as the limitations of the implementation. An ICS usually takes the form of a questionnaire where product implementors report on the conformance of their product to the named specification.

An ICS is useful in disclosing optional functionality and discretionary behavior and values. The results of the ICS can be used to identify the subset of test cases from a conformance test suite that are applicable to the implementation to be tested. This will allow the implementation to be tested for conformance against only the relevant requirements.

The basic and detailed information that an ICS provides can also be used to assess and deduce the interoperability potential of two or more products.

Checkpoints

Checkpoint 9.1. Provide specific wording of the claim. [Priority 3]
Conformance requirements

The specification MUST provide specific wording of the claim and the specific wording MUST at least include the specification name, its version, the conformance level satisfied and information about the subject that which is claiming conformance and the date of the claim.

Examples & Techniques for this checkpoint.

Checkpoint 9.2. Provide a conformance disclaimer. [Priority 3]
Conformance requirements

The specification MUST provide a conformance disclaimer.

Rationale

Although it is possible to prove with certainty when something does not conform, the reverse is not necessarily true. Especially for functional specifications, where a claim goes beyond syntax testing, a claim of conformance is not a guarantee that the claimant is 100% conforming with the specification. A disclaimer can help clarify the meaning of a conformance claim as well as its limitations. For example, this document contains a conformance disclaimer.

Examples & Techniques for this checkpoint.

Checkpoint 9.3. Impose no restrictions about who can make a claim or where claims can be published. [Priority 1]
Conformance requirements

The specification MUST NOT include any restriction about who can make a claim nor where claims can be published.

Discussion

Claimants (or relevant assuring parties) are solely responsible for the validity of their claims, keeping claims up to date, and proper use of any conformance icons.

Examples & Techniques for this checkpoint.

Checkpoint 9.4. Provide an Implementation Conformance Statement proforma. [Priority 3]
Conformance requirements

The specification MUST either:

Rationale

The inclusion of an ICS may not be applicable to all specifications. A specification's ICS can be thought of as providing a minimal list of items supported by the implementation, capturing information the WG deems necessary to support conformance claims.

Discussion

An ICS that is part of a specification does not preclude other organizations such as certification organizations, from having their own ICS.

Examples & Techniques for this checkpoint.

Checkpoint 9.5. Require the ICS be completed as part of the conformance claim. [Priority 3]
Conformance requirements

The specification MUST include the ICS as part of its conformance claim requirements. This checkpoint is not applicable, if an ICS is not applicable to the specification.

Rationale

The ICS is coupled with the requirements for making a conformance claim (guideline 11), thus providing specific information about the implementation and substantiating the conformance claim.

Examples & Techniques for this checkpoint.

Guideline 10. Provide test assertions.

A test assertion is a statement of behavior, action, or condition that can be measured or tested. It is derived from the specification's requirements and provides a normative foundation from which test cases can be built.. Each test assertion is an independent, complete, testable statement for requirements in the specification. Each test assertion results in one or more test cases. Multiple test assertions can be combined to form a test case, in this case one tests multiple facets of a particular behavior. It is recommended that test assertions be available by the time a specification enters Proposed Recommendation.

Checkpoints

Checkpoint 10.1. Provide test assertions [Priority 2]
Conformance requirements

The specification MUST provide a normative list of test assertions .

Rationale

Providing test assertions facilitates and promotes the development of test materials. Tests can point directly to the test assertion. Specific benefits include:

Discussion

In order to enable traceability from the tests back to the test assertions, use a mechanism for making explicit test assertions in the specification. The list of test assersions can be included in the specification or in a separate document that is referenced by the specification. Test assertions may be developed by the specification authors, others members of the WG or people external to the WG.

Examples & Techniques for this checkpoint.

Checkpoint 10.2. Provide a mapping between the specification and the test assertions list [Priority 2]
Conformance requirements

The specification MUST provide a mechanism linking each test assertion to the part of the specification it is stated.

Rationale

This allows both to ensure consistency between the specification and the test assertions list and to facilitate building a test suite framework based on the test assertions list.

Examples & Techniques for this checkpoint.

4. Conformance

This section defines conformance of Working Group specifications — i.e., technical reports — to the requirements of this QA Framework guidelines specification. The requirements of this guidelines specification are detailed in the checkpoints of the preceding "Guidelines" chapter, and apply to the technical reports produced by Working Groups.

4.1. Normative parts

The following parts ofthis document are normative:

Text that is designated as normative is directly applicable to achieving conformance to this document. Informative parts of this document consist of examples, extended explanations, terminology, and other matter that contains information that should be understood for proper implementation of this document.

4.2. Extensibility

This specification is extensible. That is, adding conformance related information and structure to the document in ways beyond what is presented in this specification is allowed and encouraged. Extensions to this specification MUST not contradict nor negate the requirements of this specification.

The reationale for allowing Working Groups to define extenstions to these specficiation guidelines is that these requirements are considered to be the minimal requirements writing testable technical reports. Doing more than the minumum is not only acceptable, but beneficial. Extensions also allow Working Groups to tailor their specifications more closely to the technolocy and their specific needs.

4.3. Conformance requirements and test assertions

Within each prioritized checkpoint there is at least one conformance requirement, and there may be more than one. These are prefaced by "Conformance requirements:" and highlighted in a different style. This Specification Guidelines document does not enumerate a list of test assertions. A test assertion can be derived from each MUST requirement in a straighforward manner.

4.4. Conformance definition

This section defines three degrees of conformance to this guidelines specification:

A checkpoint is satisfied by satisfying all of the mandatory @@conformance requirements@@. Mandatory requirements are those that use the conformance keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", or "SHALL NOT".

Note that it is allowed (even encouraged) to implement checkpoints in addition to those required to satisfy a particular degree of conformance (A, AA, or AAA). The more checkpoints that are satisfied, the better. However, there is no additional conformance designation for such intermediate collections of checkpoints (i.e., for all checkpoints of a given level plus some, but not all, of the checkpoints of the next levels).

The minimum recommended conformance to this specification is A-conforming -- all Priority 1 (critical/essential) checkpoins satisfied.

4.5. Conformance claims

A specification conforms to the QA Framework: Specification Guidelines at degree X (A, AA, or AAA) if the Working Group meets at least all degree X conformance requirements.

An assertion of conformance to this specification -- i.e., a conformance claim -- MUST specify:

Example:

On 23 August 2003, W3C's QA Framework: Operational Guidelines (http://www.w3.org/TR/2003/WD-qaframe-op-20030822), dated 22 August 2003 conforms to W3C's QA Framework: Specification Guidelines, available at http://www.w3.org/TR/2003/WD-qaframe-spec-20030210/, Conformance level AA.

The checklist for this specification ([SPEC-ICS]) is the Implementation Conformance Statement (ICS) pro-forma for this specification. Any assertion of conformance to this specification MUST link to a completed ICS.

4.6. Conformance disclaimer

The checkpoints of this guidelines specification present verifiable conformance requirements about the specifications (technical reports) that Working Groups produce. As with any verifiable conformance requirements, users should be aware that:

Passing all of the requirements to achieve a given degree of conformance — A, AA, or AAA — does not guarantee that the subject specification is well-suited to or will achieve its intended purposes, nor does it guarantee the quality or suitability of test materials produced from the specification.

5. Definitions

This section contains terms used in this specification, with functional or contextual definitions appropriate for this specification. See also [QA-GLOSSARY]. Some terms in this section have been borrowed or adapted from other specifications.

class of product
the generic name for the group of products that would implement, for the same purposed, the specification, (i.e., target of the specification). The class of product is the object of the conformance claim. A specification may identify several classes of products.
conformance clause
a part or collection of parts of a specification that defines the requirements, criteria, or conditions to be satisfied by an implementation or application in order to claim conformance
conformance level
a variety of conformance designation. Other designations include conformance category, conformance degree, conformance xxx. "Conformance level" is discouraged in new specifications, because of confusion with "functional level".
conformance requirement
a condition for conformance of an impelmetnation to a specification. Conformance requirements can have different levels of necessity: mandatory, recommended or optional.
deprecated
An existing feature that has become outdated by a newer construct or is no longer viable. Deprecated features should be avoided, since they may be removed in some future version.
derived profile
a profile that is created from a set of profile rules, where these profile rules provide instructions for buiding (i.e., defining profiles) and the rules are defined in a specification.
dimensions of variability (DoV)
the ways in which different products that are conformant to a specification may vary among themselves. In this Specification Guidelines document, the dimensions of variability are used to help organize, classify and assess the conformance characteristics of W3C specifications
discretionary items
deliberate and explicit grants of discretion by the specification, to the implementations, that describe or allow optionality of behavior, functionality, parameter values, error handling, etc.
functional level
a technology subset that is one of a hierarchy of nested subsets, ranging from minimal or core functionality to full or complete functionally.
implementation conformance statement (ICS)
a mechanism for providing standardized information about an implementation of a named specification, usually in the form of a questionnaire on which product implementers report about the product's conformance to the specification. An ICS is used to indicate which requirements, capabilities and options have and have not been implemented.
informative text
text in a specification whose purpose is informational or assistive in the understanding or use of the specification, and which contains no test assertions or conformance requirements.
level
a commonly used shorthand for functional level.
module
a collection of semantically-related elements, attributes, and attribute values that represents a unit of functionality. Modules are non-hierarchical, discrete divisions that are defined in coherent sets.
normative text
text in a specification which is prescriptive or contains conformance requirements.
obsolete
feature that is no longer defined in the specification. A feature is often deprecated before becoming obsolete.
profile
a subset of a technology that is tailored to meet specific functional requirements of a particular application community. A profile may address a single technology; or, a profile can also group a set of technologies (i.e., from different specifications) and define how they operate together. Profiles may be based on hardware considerations associated with target product classes, or they may be driven by other functional requirements of their target communities.
profiling
a method for defining subsets of a technology by identifying the functionality, parameters, options, and/or implementation requirements necessary to satisfy the requirements of a particular community of users.
specification category
the generic name for the type of specification and the technology it describes.
strict conformance
conformance of an implementation that employs only the requirements and/or functionality defined in the specification and no more (i.e., no extensions to the specification are implemented).
test assertion
a statement of behavior, action or condition that can be measured or tested(See also QA Glossary [QA-GLOSSARY].)
use case
a specification mechanism or technique that captures the ways a specification would be used, including the set of interactions between the user and the specification as well as the services, tasks, and functions the specification is required to perform.
usage scenario
an instance of a use case, that represents a single path through the use case. Thus, there may be a scenario for the main flow through the use case and another scenarios for each possible variation of flow through the use case (e.g., representing each option).

6. Acknowledgments

The following QA Working Group and Interest Group participants have contributed significantly to the content of this document:

7. References

7.1. Normative

RFC2119
Key words for use in RFCs to Indicate Requirement Levels, March 1997, available at http://www.ietf.org/rfc/rfc2119.txt.

7.2. Informative

CSS2
Cascading Style Sheets, Level 2, W3C Recommendation, 12 May 1998, available at http://www.w3.org/TR/REC-CSS2/ .
HTML4
HTML 4.01 Specification, W3C Recommendation, 24 December 1999, available at http://www.w3.org/TR/html401/ .
MATHML20
Mathematical Markup Language (MathML) Version 2.0, W3C Recommendation, 21 February 2001, available at http://www.w3.org/TR/MathML2/.
PUBRULES
W3C Publication Rules, available at http://www.w3.org/Guide/pubrules.html (Member-only).
QA-GLOSSARY
Comprehensive glossary of QA terminology. (Under construction, at http://www.w3.org/QA/glossary .)
QAF-INTRO
QA Framework: Introduction, L. Henderson, D. Dimitriadis, K. Gavrylyuk, L. Rosenthal, Eds., W3C Working Draft, February 2003, companion version to this document, available at http://www.w3.org/TR/2003/WD-qaframe-intro-20030210/ .
QAF-OPS
QA Framework: Operational Guidelines, K. Gavrylyuk, D. Dimitriadis, L. Henderson, L. Rosenthal, Eds., W3C Working Draft, February 2003, companion version to this document, available at http://www.w3.org/TR/2003/WD-qaframe-ops-20030210/ .
QAF-TEST
QA Framework: Test Guidelines, K. Gavrylyuk, D. Dimitriadis, L. Henderson, M. Skall, P. Fawcett, Eds., W3C Working Draft, December 2002, available at http://www.w3.org/TR/2002/WD-qaframe-test-20021220/ .
QAIG
Quality Assurance Interest Group of the W3C QA Activity, which may be found at http://www.w3.org/QA/IG/.
QAWG
Quality Assurance Working Group of the W3C QA Activity, which may be found at http://www.w3.org/QA/WG/.
SMIL20
Synchronized Multimedia Integration Language (SMIL 2.0), W3C Recommendation, 07 August 2001, available at http://www.w3.org/TR/smil20/.
SPEC-CHECKLIST
An appendix to this specification guidelines document presents all checkpoints in tabular form sorted in their original order. Available at http://www.w3.org/TR/2003/WD-qaframe-spec-20030210/qaframe-spec-checklist .
SPEC-ICS
An appendix to this specification guidelines document presents all checkpoints in tabular form sorted by priorities. Available at http://www.w3.org/TR/2003/WD-qaframe-spec-20030210/qaframe-spec-ics .
SPEC-EXTECH
QA Framework: Specification Examples & Techniques, available at http://www.w3.org/QA/WG/2003/02/qaframe-spec-extech .
STYLE-MAN
W3C Manual of Style, summarizing the style and publication rules for W3C technical reports, available at http://www.w3.org/2001/06/manual/.
SVG11
Scalable Vector Graphics (SVG) 1.1 Specification, D. Jackson, J. Ferraiolo, J. Fujisawa, Eds., W3C Recommendation 14 January 2003, available at http://www.w3.org/TR/2003/REC-SVG11-20030114/ .
SVG-MOBILE
Mobile SVG Profiles: SVG Tiny and SVG Basic, T. Capin, Editor, W3C Recommendation, 14 January 2003, available at http://www.w3.org/TR/2003/REC-SVGMobile-20030114/ .
WCAG10
Web Content Accessibility Guidelines 1.0, W. Chisholm, I. Jacobs, G. Vanderheiden, Eds., W3C Recommendation, 5 May 1999, available at http://www.w3.org/TR/WCAG10/.
XFORMS
XForms 1.0, M. Dubinko, L. L. Klotz Jr., R. Merrick, T. V. Raman, Eds., W3C Candidate Recommendation, 12 November 2002, available at http://www.w3.org/TR/2002/CR-xforms-20021112/ .
XHTML-MOD
Modularization of XHTML, M. Altheim, F. Boumphrey, S. Dooley, S. McCarron, S. Schnitzenbaumer, T. Wugofski,Eds., W3C Recommendation, 10 April 2001, available at http://www.w3.org/TR/xhtml-modularization/.
XHTML-BASIC
XHTML Basic, M. Baker, M. Ishikawa, S. Matsui, P. Stark, T. Wugofski, T. Yamakami, Eds., W3C Recommendation, 19 December 2000, available at http://www.w3.org/TR/xhtml-basic/.
XHTML11
XHTML 1.1 - Module-based XHTML, M. Altheim, S. McCarron, Eds., W3C Recommendation, 31 May 2001, available at http://www.w3.org/TR/xhtml11/
XML10
Extensible Markup Language (XML) 1.0 (Second Edition), T. Bray, J. Paoli, C. M. Sperberg-McQueen, E. Maler, Eds., W3C Recommendation, 6 October 2000, available at http://www.w3.org/TR/REC-xml .

8. Change history

2003-02-10, Last Call WD

This version integrates the resolutions taken by the WG on the issues raised since the last Working Draft; a good set of them have been taken during the QA WG F2F in Seattle. The structure hasn't really changed since the last draft, except that some checkpoints were removed, some were added.

2002-11-08, third published WD

While keeping most of the principles behind the second published WD, this version has re-ordered the guidelines in a more logical way, and takes a more formal approach in the design of CP, where all CP has a set of test assertions. Besides, repetitions have been diminished through factorization, and non-testable or arguable CP have been removed or marked as to be moved as examples and techniques. Finally, it integrates all the issues resolutions agreed during the QA WG Tokyo face-to-face meeting and the weeks following.

2002-08-26, second published WD

Significantly reorganized and revised the first published WD. This version produced as a series of editor's drafts. The changes below are reverse chronological (most recent first), so more recent ones may build on older ones.

2002-05-15 version

First published WD.