The principal goal of this document is to help W3C Working Groups to write
clearer, more implementable, and better testable technical reports. It both
provides a common framework for specifying conformance requirements and
definitions, and also addresses the representation of specifications
(technical reports) as schemata, both of which facilitate the generation of
test materials. The material is presented as a set of organizing guidelines
and verifiable checkpoints. This document is one in a family of Framework
documents of the Quality Assurance (QA) Activity, which includes the other existing or
in-progress specifications: Introduction; Operational Guidelines; and, Test
Guidelines.
This document is a WG version of the QA Framework Specification Guidelines. Unless you want to comment on a very unstable document, you're encouraged to refer to the latest published version instead.
This section describes the status of this document at the time of its
publication. Other documents may supersede this document. The latest status
of this document series is maintained at the W3C.
This document is a W3C Working Draft (WD), made available by the W3C Quality
Assurance (QA) Activity for discussion by W3C members and other interested
parties. For more information about the QA Activity, please see the QA Activity statement.
While keeping most of the principles behind the second published working draft, this third version has re-ordered the guidelines in a more logical way, and takes a more formal approach in the design of checkpoints, where every checkpoint has a set of test assertions. It integrates all the issues resolutions agreed during the QA WG Tokyo face-to-face meeting and the weeks following. A more detailed changelog is available below.
This version supersedes all previous drafts. It is expected that updated versions of this document will be produced regularly, along with other members of the Framework documents family. It is anticipated that the next public version of this document will be the Last Call version (estimated, February 2003).
This document is accompanied by a QA
Framework: Specification Examples & Techniques document, which
illustrate the guidelines and checkpoints with case studies, and explain
how to satisfy the checkpoints.
The QA Working Group Patent
Disclosure page contains details on known patents related to this specification, in conformance with W3C policy
requirements.
Please send comments to www-qa@w3.org,
the publicly
archived list of the QA Interest
Group [QAIG]. Please note
that any mail sent to this list will be publicly archived
and available. Do not send information you would not want to see distributed,
such as private data.
Publication of this document does not imply endorsement by the W3C, its
membership or its staff. This is a draft document and may be updated,
replaced, or made obsolete by other documents at any time. It is
inappropriate to use W3C Working Drafts as reference material or to cite them
as other than "work in progress".
A list of current W3C Recommendations and other technical documents can be
found at http://www.w3.org/TR.
- Introduction
- Scope and Goals
- Class of Product
and Audience
- Motivation and expected benefits
- Relationship to other specifications
- Understanding and using this document
- Terminology
- Checkpoint priorities
- Roadmap to guidelines
- Dimensions of Variability
- Guidelines
- Define Scope.
- Identify what needs to conform and
how.
- Specify conformance policy.
- Address the use of profiles to divide
the technology.
- Address the use of modules to divide the
technology.
- Address the use of functional levels to
divide the technology.
- Identify the
relation between deprecated features and conformance.
- Define discretionary items.
- Allow
extensions or NOT!
- Provide a conformance clause.
- Specify how to make conformance
claims.
- Publish an Implementation
Conformance Statement proforma.
- Support general document conformance
conventions.
- Provide test assertions.
- Conformance
- Normative sections
- Test assertions
- Conformance definition
- Conformance disclaimer
- Definitions
- Acknowledgments
- References
- Normative
- Informative
- Change history
An appendix to this document [SPEC-CHECKLIST] presents all checkpoints in a tabular form, for
convenient reference.
The scope of this document is a set of requirements for W3C Technical
Reports (TRs) that if satisfied, will enhance the clarity, implementability,
and testability of TRs. It decribes what goes into a TR with respect to
conformance and conformance topics, dealing with how a TR establishes,
defines, and presents its conformance policy. This document includes:
- This introduction, which provides context for understanding the
requirements in section 2
- Section 2 explains fourteen guidelines. Each guideline consists of a
list of requirements called "checkpoints", which must be satisfied in
order to conform to this document. There are two general types of
guidelines, those that deal with the document features and conventions of
the specification and those that deal with establishing and defining
conformance policy.
- Section 3 defines explains how to make conformance claims that W3C TRs
satisfy the requirements of section 2. It defines conformance for this
document
- An appendix provides use scenarios or examples of how this document
could be used.
- A second appendix lists all the guidelines and checkpoints into an
Implementation Conformance Statement (ICS) pro-forma for this
specification, which is required to be completed as part of the
conformance claim to this document.
A separate document, entitled "Specification Examples and Techniques 1.0"
(the "ExTech document" from here on) provides suggestions and examples of how
each checkpoint might be satisfied. The techniques in the ExTech document
are informative examples only, and other strategies may be used or required
to satisfy the checkpoints. The ExTech document is expected to be updated
more frequently than the current guidelines. Developers, W3C Working Groups,
users, and others are encouraged to contribute techniques and examples.
This document is applied and conformance (to this document) achieved as
new TRs are being written. New TRs include those in-development or being
revised. As for legacy specification, they may indirectly comply with the
spirit or intent of some checkpoints, without actually satisfying all
requirements in those checkpoints.
The class of product or target of this specification is W3C Technical
Reports. Within this Specification Guidelines document, the term
"specifications' is specifically limited to W3C Technical Reports, even
though these guidelines could be applied to other documents.
The primary target audience of this document is specficiation authors,
however, it is applicable to a broader audience including:
- those who review specifications during their development,
- implementers of specifications,
- those who use specifications to build test materials.
It is a design goal of these guidelines the WGs can apply them in a common-sense and
workable manner.
Good
specifications lead to better implementations and foster the development of
conformance test suites and tools. Conforming implementations lead to
interoperability.
The quality of the specification has direct impact on the quality of
implementations. Quality encompasses utility which refers to the
usefulness of the specification to the intended users and
objectivity which focuses on the whether the specification is
presented in an accurate, clear, complete, and unbiased manner.
Providing requirements and definitions about conformance topics, as well
as guidance in the structure and anatomy of specifications, will foster a
mutual understanding amongst developers of specifications, implementations,
and conformance test materials. Specifically, well-structured specifications
with clear and comprehensive conformance requirements:
- promote a common understanding of conformance and what is required to
claim conformance to a specification,
- facilitate consistent application of conformance within a
specification,
- facilitate consistent application of conformance across related
specifications,
- promote better interoperability and open interchange,
- encourage the use of applicable conformance test suites,
- promote uniformity in the development of conformance test suites,
- enhance test assertion identification and test traceability,
- promote automated test generation directly from the specifications,
This document is part of a family of QA Framework documents designed to
help the WGs improve all aspects of their quality practices. The QA
Framework documents are:
The QA Framework documents are interrelated and complement each other. For
example, the anatomy of a specification is related to the type of test
materials that will be built, hence there is interrelationship between this
document and the Test Guidelines. The reader is strongly encouraged to be
familiar with the other documents in the family.
The Framework as a whole is intended for all Working Groups, as well as
developers of conformance materials for W3C specifications.
Not only are the
Working Groups the consumers of these guidelines, they are also key
contributors. The guidelines capture the experiences, good practices,
activities, and lessons learned of the Working Groups and present them in a
comprehensive, cohesive set of documents for all to use and benefit from. The
objective is to reuse what works rather than reinvent and to foster
consistency across the various Working Group quality activities and
deliverables.
This document does not preclude the need to apply the W3C Manual of
Style [STYLE-MAN] and to
conform to the Publication Rules [PUBRULES ] (member-only). It is intended
to complement those specifications.
This document employs the WAI (Web Accessibility Initiative) model for
representing guidelines or general principles for the development of
conformance materials. See, for example, Web Content Accessibility Guidelines
1.0 [WCAG10]. Each guideline includes:
- the guideline number,
- the statement of the guideline,
- the rationale behind the guideline,
- a collection of one or more checkpoints.
The guidelines are of two general types:
- those that deal solely with the document features and conventions of
the specification — GL1 and GL10 - GL14;
- those that, in addition to documentation aspects, deal with how
specifications should establish and define the conformance policy for the
specification's technology, including ways in which the technology may be
subdivided for conformance purposes — GL2 - GL9.
The checkpoints in each guideline define specification characteristics and requirements needed to fulfill the purpose of the guideline.
Each checkpoint definition includes:
- the checkpoint number,
- the statement of the checkpoint (its "title"),
- the priority of the checkpoint,
- a statement of how to fulfill the checkpoint,
- optional informative notes, clarifying examples, and cross references
to related guidelines or checkpoints.
- a link to the corresponding section in Specification Examples &
Techniques.
Each checkpoint is intended to be specific enough so that someone can
implement the checkpoint as well as verify that the checkpoint has been
satisfied. A checkpoint will contain at least one, and may contain multiple
individual requirements , that use RFC2119 normative keywords. See the "Conformance"
chapter for further discussion of requirements and test assertions.
A separate appendix to this document [SPEC-CHECKLIST] presents all checkpoints in a tabular form, for
convenient reference. This is an Implementation Conformance Statement (ICS) pro-forma for this specification. (See GL12.)
The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY ", and "OPTIONAL" will be used as
defined in RFC 2119 [RFC2119]. When used with the normative RFC2119 meanings, they will be all upper case. Occurrences of these words in lower case comprise normal prose usage, with no normative implications.
Unusual terms in these framework documents are defined when first used.
This document contains a "Definitions" chapter.
Generally useful QA-specific terms will eventually be in the QA Glossary
[QA-GLOSSARY]. If terms are in the QA Glossary, their
definition herein will refer to that QA Glossary entry with a link. The definitions herein
may supplement or build on that generic definition with other information
that is useful or helpful in the specification guidelines context. They will
not contradict the generic definitions.
Some checkpoints are more
critical than others for producing a high
quality, testable standard that is a sound basis for successfully
interoperable products. Therefore each checkpoint is assigned a priority level
based on the checkpoint's impact on the quality of the specifications
produced by the Working Groups.
- [Priority 1]
- Critical/Essential. These checkpoints are considered to be basic requirements for
ensuring an acceptable level quality and testability in the
specification.
- [Priority 2]
- Important/desirable. Satisfying these checkpoints, in addition to the priority 1
checkpoints, should significantly improve the clarity, quality, and
testability, of the standard, as well as its suitability as a basis for
building quality test materials.
- [Priority 3]
- Useful/beneficial. Satisfying these checkpoints, on top of all the others, will further improve the quality, usability, and testability of
the specification.
There are 14 guidelines:
- Scope and use cases (often in Requirements Document)
- Specification categories and product classes
- Profiles (hardware, environment, etc.)
- Modules
- Levels
- Conformance policy
- Deprecation policy
- Discretionary items
- Extensibility
- Conformance clause
- Claims of conformance
- Proforma
- Document conventions
- Test assertions
Each of the latter set of eight guidelines GL2 through GL9 addresses a way
in which the conformance policy of a specification might allow variation
amongst conforming implementations. For example, a specification might allow
implementations to choose between one of two well defined behaviors for a
given functionality, thus two conforming implementations might vary on that
aspect.
For this reason, these eight guidelines are collectively called the
"dimensions of variability (DoV)". The eight dimensions of variability
recognized by this document are:
The above are not necessarily all orthogonal to one another. There are
many possible associations, dependencies, and interrelationships. As a
general policy, these specification guidelines do not attempt to legislate
correct or proper relationships amongst the DoV. Rather, they try to clarify
the nature of each dimension, and require specification to make deliberate
and well documented choices. Some discussion of possible
interrelationships, including examples will be found in the Specification
Examples & Techniques.
The dimensions of variability are one of the principal concepts in this guidelines document to help organize, classify and assess the conformance
characteristics of W3C specifications. The eight DoV get special attention
because, since they are at the core of the definition of a specification's
conformance policy, there is significant potential for negative
interoperability impacts if they are handled carelessly or without careful
deliberation.
As a general principle, variability complicates interoperability.. In
theory, interoperability is best when there are numerous identical,
complete, correct implementations. However, in practice, the net effect of
conformance variability is not necessarily negative in all cases,
when compared to the alternatives. For example profiles — subdivisions of the technology
targeted at specific applications communities — introduce variability
amongst implementations. Some will implement Profile ABC, some will
implement Profile XYZ, and the two might not intercommunicate well if ABC and
XYZ are fairly different. However, if ABC and XYZ are subsets of a large
monolithic specification — too large for many implementors to tackle in
-- and if they are well targeted at actual application sectors, then
subdivision by profiles may actually enhance interoperability.
Different sorts of variability have different
negative and positive impacts. The principal danger is "excessive" variability - variability
which goes beyond that needed for positive interoperability trade-off, and
which unnecessarily complicates the conformance landscape.Specification writers need to
carefully consider and justify any conformance variability allowed, do so by
reference to the project requirements and use cases, and explicitly document
the choices made.
When writing specifications it is critical to understand their primary
purpose and scope. Clearly defined scope helps to keep the specification
content focused and unambiguous.
If the specification describes content requirements (for example, [WCAG10], [UAAG10]), understanding of
their purpose is the key to defining the minimal sufficient set. If the
specification describes a protocol or Application Programmer Interface
(API) — examples
include XML Protocol, DOM --
there should be a clear understanding of the primary use cases. For the
purposes of this document, a use case is a specification mechanism or
technique that captures the ways a specification would be used, including the
set of interactions between the user and the specification as well as the
services, tasks, and functions the specification is required to perform.
Readers of the specification face similar difficulties to writers. To
understand what the document says, the reader needs to know the context of
the author, and the scenarios the author had in mind. In case of protocols
and APIs
specifications developers try to assess whether the specifications cover the
use cases the product needs to cover. Having no use cases described in the
specification invites misuse of the specification itself.
Checkpoints
To fulfill this checkpoint, a specification MUST define what scenarios are in scope and SHOULD identify out of scope scenarios.
Rationale: it helps both the writer and the reader to know what are the limits of what is specified in a normative document.
Examples & Techniques for this checkpoint.
A Usage Scenario is an instance of a use case, representing a single path
through the use case. Thus, there may be a scenario for the main flow through
the use case and other scenarios for each possible variation of flow through
the use case (e.g., representing each option).
To fulfill this checkpoint, a specification MUST include or link to Usage Scenarios, SHOULD have an extensive list of the usage scenarios that the authors have in mind
Rationale: Usage scenarios help to assess what features
are missing and what features are superfluous in the specification.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST include or link to at least one example and SHOULD provide an example for each section that public feedback has shown to be unclear or hard to understand.
Examples & Techniques for this checkpoint.
A formal description is a concrete or abstract representation of an intended behavior in the specification.
To fulfill this checkpoint, a specification MUST provide an example for each identified formal description.
It is hard to understand the formal descriptions of content without
informative interpretations.
For instance, the recent complex specifications like XML Schema [XML-SCHEMA] and XML Protocol have shown the interest of having a "Primer" part or section to illustrate how to use the
specification.
Examples & Techniques for this checkpoint.
Categorizing the specification provides a basis for classifying the
software that may be affected by the specification — and thus, provides the
answer to "what needs to conform?". To answer this question, it helps to look
at the nature of the specification and categorize it. Most specifications can
be classified into one of the following categories:
- foundation or abstract (e.g., Infoset),
- content/data (e.g., MathML, SVG),
- protocols (e.g., SOAP),
- processors (e.g., XSLT,
XML Query),
- APIs (e.g.,
DOM),
- notation/syntax (e.g., XPath),
- set of events (e.g., one part of XForms)
- rules for deriving profiles (e.g., part of SMIL)
The categories indicate what the specification describes. One
specification could potentially fall into more than one category.
From this categorization of specifications, the WG can identify the class of products that
are affected by the specification. Classes of products can be generalized as
either or producers or consumers, or as content itself. Identifying which are
producers and consumers is clear for a protocol-type specification, the two
parties to the dialog are the targets. For a processor-type specification,
the processor is the consumer of an XML vocabulary defined in the
specification. For content-type specifications, there may be one or more
consumers that take the content and 'play' it in some way.
The following is a non-exhaustive list of classes of products for W3C
specifications.
- content (of type, meaning, and format as defined in the
specification)
- producer of content (may be divided into initiators and modifiers)
- player (read-only consumer, conveys content in non- XML way)
- consumer in a one-way pipeline
- responding agent (e.g., server) of API (consumer and
producer)
- processor (consumer of its vocabulary/instructions)
- module that hosts the processor
- producer of instructions/commands to processor
- profile derived from the specification's Rules for Profiles
One specification could define more than one player. For example, MathML
could address the behavior of display of math notation and also a consumer
that parses the MathML as a formula and uses it for mathematical
processing.
The conformance clause identifies the "class of products" (i.e., object of
the claim) that is the target of the specification. In addition to
identifying what conforms (i.e., class of products), it describes how
conformance can be met. This would be a description of the conformance
requirements, conditions and/or criteria for each class of product.
Checkpoints
To fulfill this checkpoint, a specification:
- MUST list the classes of product it addresses
- SHOULD use the above enumerated classes names when they match those of the specification
- MUST define and describe any product class that does not match the enumerated set
Example. Within the SMIL 2.0 Language
Profile ([SMIL20], chapter 13), there are 2
classes of products: documents and basic user agents. Within Mathematical Markup Language (MathML)
2.0 [MATHML20] there are output-compliant,
authoring tools and input-compliant, rendering/reading tools.
Examples & Techniques for this checkpoint.
The conformance requirements indicate the conditions to be satisfied in
order to claim conformance. It is likely that these conformance definitions will reference
normative text within the specification or in other related
specifications.
To fulfill this checkpoint, a specification MUST define the conformance requirements for each class of product.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST identify in the Introductory section which of the
categories of object are specified in the
document as a whole.
Rationale: doing so helps keep the scope of the specification in
focus, both for the reader, and for the author(s) who must define the classes of product and their conformance criteria.
Some specifications define more than one of the enumerated object types.
XForms is an example. It defines: Content, via an XML vocabulary; Content, via
named datatypes;
Syntax, in the form of a set of functions to
supplement the XPath core set; behavior of a processor;
behavior of a user
agent; a set of
events.
Examples & Techniques for this checkpoint.
A look at various W3C Technical Reports shows that the term "conformance"
is often qualified, resulting in more than one type of conformance. It is
important to convey an understanding of what is meant by conformance and
where the products of a class are allowed to have more, less, or different functionality
from one another. If the specification defines behavior for
more than one class of product, there may be a separate conformance policy for each class. Often, the specification will allow discretionary choices,
such as the choice of date-time formats, but require a conforming product to make a choice only within the
allowable range. (See Guideline 8 for
more discussion.)
Sometimes a product developer can choose to implement certain modules.
There may be per-module conformance requirements that apply if and only if
the developer chooses to implement a particular module.
Sometimes a product developer can choose to implement extensions. There
may be conformance requirements for non-interference of extensions. (See
Guideline 9 for more discussion.)
Where all products of a class must be substantially alike, it should be
clear that a "strict conformance" policy is in effect for that product class.
Strict conformance is defined as conformance of an implementation that employs only the
requirements and/or functionality defined in the specification and no more
(e.g., no extensions to the specification are implemented). No discretion is granted to implementers, and any
requirements for handling deprecated features must be followed.
Overall, the intent of the WG
should be clear. In particular, a reader intending to implement a product in
one of the product classes addressed by the specification should know what
the WG wants for interoperability
among products in the class. The developer should understand what forms, if
any, of "product differentiation" are allowed among conformant products. The
specification may need to explain how the rules apply and possibly provide
examples.
Guideline 10, "conformance clause" is related to this guideline. This
Guideline 3 focuses on the establishment and scope of definition of a
conformance policy, while Guideline 10 focuses on where and how to document
it. That is, the verification of these checkpoints will require looking at
the Conformance Clause.
Checkpoints
To fulfill this checkpoint, a specification MUST include a normative section detailing any universal requirements for minimum functionality. It is not applicable if there are not any universal requirements.
Rationale: the reader must be able to recognize any minimum that applies to all conforming products of each class. The test suite can have a core
of universal test cases.
If levels are used (see Guideline 6), the lowest level
may represent the minimum set of requirements. If profiles are used (see Guideline 4), there may be
different minima for each profile. If modules are used (see Guideline 5), there may
be different minima for each module. Furthermore, a module may itself be a
minimum (i.e., required) for a particular class of product.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST state in its conformance section if the conformance requirements are strict or identify the kinds
of variability that are permitted..
Rationale: the reader must be able to recognize when a policy of "strict conformance"
applies. As defined above, this implies
that all conformant products of a class behave essentially the same way.
Strict conformance can apply separately to each class of product addressed
by the specification. If
profiles are used, each profile may have its own conformance boundaries. If
modules are used, each module may have its own conformance boundaries. Some profiles or modules may have a strict
conformance policy while others do not.
Examples & Techniques for this checkpoint.
To full this checkpoint, a specification MUST state in its conformance
section all facets of the requirements where the required features
represent the maximum allowable capabilities.
Rationale: When a strict conformance policy applies, the maximum set of
features is the same as the minimum. When strict conformance does not
apply, implementations may be allowed to exceed the specified capabilities
in ways other than having extensions. (Example: a graphical capability may
be specified with a required resolution or a set of resolution levels that
must be supported. The specification may wish to require that
implementations not create images of higher resolution due to concerns
about excessive size.) The specification should identify the facets in
which an implementation may add more features, if there is a history of
expanding features for those facets.
If profiles are used (see Guideline 4), state
whether extra capabilities of the platform may be exploited. If modules are
used (see Guideline 5), there
may be different provisions for extra features applying to each. If
levels are used (see Guideline 6), state whether the highest level may be
exceeded with additional features or functionality. If
deprecation applies (see Guideline 7), make it
clear whether support of the obsolete features is optional, part of a level,
or required. If levels are used (see Guideline 6), make it clear whether
the highest level may be exceeded with additional features or functionality.
If discretionary choices are allowed (see Guideline 8), make it clear if more than one
may be implemented, when it is technically possible to do so.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST either exclusively use conformance terms as defined in this document or define any other conformance terms used in it and reference them from the conformance clause.
Rationale: it is necessary to define terms that govern application of the conformance
provisions. Ideally, all terms are from QA documents and other existing
literature and need only be cited. If special terms are constructed, such as
to combine modules and levels or modules and discretionary choices, they need to be defined in the specification.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST provide a justification for each Dimension of Variability the specification uses by reference to usage scenarios and requirements.
Rationale: D.o.V. add complexity to a specification; explaining what's the usage of each of them can help implement and test them better, and can help reviewers understanding their necessity.
A profile is a subset of the technology that supports a particular functional objective or a subset of a set of technologies defining how they are required to operate together
(e.g., XHTML plus MathML plus SVG).
Profiles can be based on
hardware considerations associated with target product classes — for
example, SVG Tiny is
aimed at mobile phones — or they may be driven by other functional requirements of their target constituencies — for example, a graphical
profile tailored for technical illustrations in aircraft maintenance
manuals.
The use of profiles to divide the technology is described in the
specification, and may or may not be reflected and paralleled by the
structure and organization of the specification.
Specifications may define individual profiles, or may define rules for
profiles, or both. An individual profile defines the requirements for classes
of products that conform to that profile. Rules for profiles define validity
criteria for profiles themselves — i.e., if others (users, applications, or
other standards) define their own profiles of the standard, then rules for
profiles define the constraints that those profiles must satisfy in order to
be considered valid profiles of the specification.
For example, XHTML Modularization ([XHTML-MOD], section 3) and Synchronized
Multimedia Integration Language (SMIL 2.0), [SMIL20] specifications both define rules for profiles --
what constraints must a profile meet in order to be classified as a "Host
Language Profile" or an "Integration Set Profile." SMIL further defines some
specific profiles, using the modularization. Separate recommendations --
XHTML Basic [XHTML-BASIC] and
XHTML 1.1 [XHTML11] — define specific
profiles based on the XHTML modularization.
Checkpoints
To fulfill this checkpoint, a specification
- MUST state whether conformance of each class of product
in the specification is only defined within the context of profiles, or
whether there can be conforming products outside of the
bounds of profiles.
- MUST describe additional conditions associated with a particular profile or collection of profiles for each class of product
It is not applicable if profiles are not used.
For example, is content required to conform to one of the profiles, or is there a concept of conformance of content independent of conformance to one of the
profiles? Is a producer (of content) conforming if it generates content that
is otherwise valid but does not conform to a profile?
An example of additional conditions on profiles would be to require that only one profile can be implemented at a time.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST define for each profile the minimal required features/support for each class of product. It is not applicable if profiles are not used.
Rationale: because profile places explicit requirements on each class of product that have specific and often limited operating environments these requirements must be defined for each class of product that is affected.
To illustrate, SMIL 2.0 [SMIL20] has a
SMIL 2.0 Language Profile for user agents but also provides a SMIL 2.0 Basic
Profile for wireless and embedded devices. The SMIL 2.0 Language Profile
requires that a user agent implement the BasicAnimation module but not the
SplineAnimation Module. The SMIL 2.0 Basic Profile on the other hand does not
require implementation of any of the Animation modules.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST identify all the relations and interactions between profiles and the other dimensions of variability. It is not applicable if profiles are not used.
Dependency or interrelationship between profiles and modules is common --
XHTML [XHTML-MOD], SMIL [SMIL20], SVG 1.1 [SVG11]. Less often,
deprecated features, levels, discretionary choices, or extensions could
depend on profiles.
Examples & Techniques for this checkpoint.
if the specification allows the creation of derived profiles, to fulfill this checkpoint a specification MUST provide requirements for derived profiles and these requirements MUST be testable. It is not applicable if profiles are not used.
It is RECOMMENDED that requirements for derived profiles impose at least
these two rules on derived profiles: derived profiles SHOULD be specified in
a way that meets all the pertinent checkpoints of this document (QA
Framework: Specification Guidelines); and, derived profiles SHOULD NOT
contradict pre-defined profiles, if there are any in the base specification.
Rationale: well-designed rules for profiles will facilitate that derived profiles are well-specified, and testability will enable an independent tester to verify
conformance of a derived profile to the rules.
Examples & Techniques for this checkpoint.
Modules are non-hierarchical, discrete divisions or functional groupings of the technology.
For example, SMIL 2.0 [SMIL20]
defines a module as follows:
A module is a collection of semantically-related XML elements, attributes, and
attribute values that represents a unit of functionality. Modules are
defined in coherent sets.
Modules generally can be implemented independently of one another — e.g.,
audio vs. video module. That notwithstanding, it is possible for one module's
definition (and therefore implementation) to have explicit dependency upon
another module. It is not only possible, but common to implement multiple
modules.
Checkpoints
To fulfill this checkpoint, a specification MUST
document any mandatory conditions or constraints on their usage. Such conditions include,
- atomicity of modules
- any mandatory module(s);
- any modules that are optional
- dependencies among modules:
- modules that require and build on functionally related modules;
- modules that require modules from other functional areas;
- constraints against combined occurrence of particular pairs of
modules;
- any other — there MAY be conditions and constraints other than just
enumerated, and they MUST be described.
It is not applicable if modules are not used.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST specify any relationships and interaction with other dimensions of variability. It is not applicable if modules are not used.
Rationale: often there is dependency or interrelationship among modules, on the one hand, and profiles or discretionary choices on the other. Modules may have
levels or
deprecated features. Extensions could be defined based on modules.
Examples & Techniques for this checkpoint.
Functional levels — or in common usage, simply "levels" — are used to group functionality into nested subsets, ranging from minimal or core
functionality to full or complete functionally. Level 1 is the minimum or
core of the technology. Level 2 includes all of level 1 plus additional
functionality. This nesting continues until level n, which consists of the
entire technology.
Levels may result from progressive historical development and enrichment
of the technology in a series of specifications, as in the case of CSS and DOM. Levels could also be defined explicitly in a single edition of the specification, but no examples of this are found in W3C specifications. Rather, it is
more common in current W3C practice to use profiles to accomplish this. For
example, SVG Mobile [SVG-MOBILE] defines three
nested profiles — Tiny, Basic, Full — which are each targeted at a specific
graphics hardware community (mobile phone, hand-held computer, desktop
computer).
See Guideline 4 for full
discussion of profiles, including comments on possible profiles-levels
relationships. See Guideline 6
for a full discussion of modules, including possible modules-levels
relationships.
Checkpoints
To fulfill this checkpoint, a specification MUST specify any relationships and interaction with other dimensions of variability. It is not applicable if levels are not used.
Examples & Techniques for this checkpoint.
A deprecated feature is a feature
whose use is discouraged because it has been outdated by newer constructs or
is no longer viable.
After the initial publication of a specification, specification developers
may consider the deprecation of a feature (e.g., function argument, element
or attribute) defined in the specification. Deprecated features may become obsolete and no longer
defined in future versions of the specification. Deprecation of a feature may
warn implementers that the feature was a bad idea and it may be withdrawn in
the future. Specification developers need to consider the effect of
deprecation on all the classes of products that implement the specification
(e.g., authoring tools, user agents) as well as the conformance consequences
on each class of product. For the purpose of backward compatibility, it may
be necessary to specify different requirements for the support of deprecated
features for each class of product. For example, authoring tools (producers)
would not use the feature, but user agents (consumers) would continue to
support it.
Checkpoints
To fulfill this checkpoint, a specification MUST identify in a normative section each deprecated feature. It is not applicable if there is no deprecated feature.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST specify the degree of support required for each deprecated feature for each product class and the conformance consequences of the deprecation. It is not applicable if there is no deprecated features.
For example, a deprecated-features
section of MathML 2.0 ([MATHML20], section
7.2.1.2) describes, about deprecated MathML 1.x features, that
MathML-output-compliant authoring tools may not generate MathML markup
containing deprecated features; whereas MathML-input-compliant
rendering/reading tools must support deprecated features.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST document each deprecated features with a rationale for deprecation. It is not applicable if there is no deprecated features.
Rationale: providing the rationale for deprecating a feature helps implementers and users to understand the motivation for the deprecation, the impact and
consequences on current and future implementations, and perhaps insight into
its eventual disappearance from the specification.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST provide an example for each deprecated feature showing how to avoid using it. It is not applicable if there is no deprecated features.
Rationale: examples are helpful in providing alternatives or better ways to get the same results. By showing what can be done in place of the deprecated feature
will help to get implementers to discontinue use of the deprecated
feature.
Examples & Techniques for this checkpoint.
Discretionary items are defined as
deliberate and explicit grants of discretion by the specification, to the
implementations, that describe or allow optionality of behavior,
functionality, parameter values, error handling, etc.
Discretionary items are often made available in specifications, to give
implementers of the technology the opportunity to decide from alternatives
when building applications and tools. Discretionary items may
be considered necessary because of environmental conditions (e.g., hardware
limitations or software configuration, or external systems), locality (e.g.,
time zone or language), optional choices providing flexibility of
implementation, dependence on other specifications, etc.
Discretionary items come in three basic variants:
- discretionary choices
- a value or behavior may be chosen from a
well-defined enumerated set of two or more possibilities;
- optional features
- a well-defined feature may be supported or not (if
supported, then the requirements are clear and unambiguous)
- implementation dependent values (or features)
- it is open ended and
undefined, what set of values an element or attribute may have, or the
behaviors of a product that implements a feature, etc
Checkpoints
To fulfill this checkpoint, a specification MUST indicate the rationale for the discretionary items by providing a reference or link to its use cases and/or project requirements and SHOULD identify by labeling all discretionary items. It is not applicable for specifications that don't have discretionary items.
Doing this helps readers, implementers and testers to find these discretionary items and understand the need for them.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST describe any permitted variations or constraints for how the implementation dependency is realized by implementations.
Examples of permitted variations or constraints to be addressed include:
- implementation dependent ranges, data, minimum or maximum values,
- environmental resources (e.g., memory or disk limitations),
- environmental values (i.e., language and local settings).
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST indicate whether zero, exactly one, or a multiple of choices/options are allowed to be implemented. It is not applicable for specifications that don't use discretionary items or for implementation dependent values among them.
Examples of constraints include mandating that an implementation implement only one choice, implement every choice, or be allowed to implement none of the choices.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, the specification MUST state that given identical conditions, the effect of a discretionary choice is consistent within a single implementation.
Rationale. Users have an expectation of what to expect and should be able to count on getting the same results under the same conditions with a given implementation.
Examples & Techniques for this checkpoint.
An extension to a specification is a mechanism to incorporate
functionality beyond what is defined in the specification.
Allowing
extensions affects how conformance is defined as well as what conformance
claims can be made. Exercise caution in determining the extent to which
extensions are allowed or not allowed. Since extensions can seriously
compromise interoperability, specification writers should carefully consider
whether extensions should be allowed.
Extensions may be private (often vendor specific) or public (a full
description of the extension is public). Private extensions are usually truly
private, i.e., valid for a specific implementation or are only known by prior
agreement between implementations. Public extensions are extensions in which
the syntax, semantics, identifiers, etc. are defined and published allowing
anyone to implement the extended functionality.
Specifications allow extensions for various reasons. Extensions allow
implementers to include features that they consider to be required by their
customers. Also, extensions, often define new features that may migrate into
future versions of the specifications. However, the use of extensions can
have a severe negative impact on interoperability. Some methods for enabling
extension have less impact on interoperability than other methods. For
example, a specification that allows private extensions (e.g., proprietary)
is highly likely to impede interoperability, whereas a specification than
permits only registered extensions partially mitigates the negative
impacts.
Checkpoints
To fulfill this checkpoint, a specification MUST state that extensions are disallowed and MUST indicate the conditions for when this holds true.
Rationale: if the specification writer wants to enhance interoperability by constraining implementer extensions, wording in the specification must indicate this.
If extensions are not allowed, then it should be clear to the reader that not only are extensions not allowed, but the circumstances under which they are not allowed. This is strict conformance. Strict conformance is often imposed on applications or content (e.g., a software program or document instance). This prohibition of extensions could be applied to a specific profile or module, rather than to the entire specification.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST state
- the conditions under which extensions are allowed,
- the scope
of the extensions,
- their effect on conformance claims,
- any limitations or restrictions on the use of
the extension
and MUST document the rationale for allowing extensions by referencing use cases and/or project requirements. It is not applicable if the specification doesn't allow extensions.
Rationale: readers should be able to understand the motivation for the inclusion of an extension and its intended use. Documenting the scope and rationale for extensions helps assess the impact of extensions on interoperability
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, the specification MUST state that extensions can not negate or change support for required functionality. It is not applicable if extensions are not allowed.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST provide a standard way of defining the extension. It is not applicable if extensions are not allowed.
This helps to ensure predictable handling of extensions, that
is, its recognition as such and the appropriate actions (i.e., to ignore or
to implement).
Examples & Techniques for this checkpoint.
To fulfill this checkpoint a specification MUST require that the syntax and semantics of the extension be publicly documented. It is not applicable if extensions are not allowed.
Rationale: public availability with a full description allows the extension to be implemented by anyone without prior arrangement. This enhances interoperability by allowing the same functionality to be uniformly implemented across different implementations.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST indicate via conformance requirements that implementations provide a mode under which they produce only conforming content. This checkpoint is not applicable if extensions are not allowed.
Rationale: This checkpoint can be used to ensure that there is a way to produce (generate) only conforming content. It is applicable to specifications that identify producer of content as one of its classes of products.
Examples & Techniques for this checkpoint.
A conformance clause is a part or collection of parts of a specification that defines the requirements, criteria, or conditions to be satisfied by an
implementation or application in order to claim conformance. Typically the
conformance clause is a high-level description of what is required of
implementations and applications.
Guideline 3, "conformance policy" is
related to this guideline. Guideline 3 focuses on the establishment and scope
of definition of a conformance policy, while this Guideline 10 focuses (among
other topics) on how and where to document it.
Checkpoints
To fulfill this checkpoint, a specification MUST document its conformance policy and specific conformance requirements.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST document its conformance policy and specific conformance requirements in a dedicated section of the document.
Rationale: having the conformance clause exist as a separate section within the specification makes it clearly identifiable, allowing a reader to find the
overall conformance policy, as well as all specific conformance provisions
from a single starting point.
Examples & Techniques for this checkpoint.
A specification depends on another specification when it relies on or requires functionality (or behavior) from the other specification in order to work (function) correctly. This other specification provides a necessary condition or functionality.
To fulfill this checkpoint, a specification MUST have normative references to any specification it depends on and MUST describe the relationship between the specifications and any conformance implications
Rationale: dependence on other specifications
affects the conformance boundaries of the current specification, and thereby
affects the requirements on conformant products.
For example, SVG 1.0 requires that the class of product
called "user agent" be consistent with the XML 1.0 Recommendation [XML10] and (conditionally) support Cascading Style Sheets, level
2 [CSS2].
Examples & Techniques for this checkpoint.
A specification may differentiate conformance claims by designating
different degrees or types of conformance in order to apply and group
requirements according to modules, profiles, levels, or priorities. When a
conformance claim is linked to functionality, impact and/or incremental
degrees of implementation, the term 'conformance level' is often used to
indicate the varying degrees of conformance. The WG includes in the specification the way they
want people to claim their conformance.
Checkpoints
To fulfill this checkpoint, a specification MUST identify and define all conformance designations.
In current W3C practice, a number of different naming conventions are used
to label conformance, in cases where there is not a single, monolithic
(strict) conformance definition. The naming convention used to label the
conformance can provide useful information. Degrees, for example, implies
incremental importance or difficulty. This Specification Guidelines document
uses "degrees" for example, to refer to three successively more demanding
degrees of conformance (A, AA, AAA).
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST provide specific wording of the claim and the specific wording MUST at least include the specification name, its version, the conformance level satisfied and information about the subject that which is claiming conformance and the date of the claim.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST provide a conformance disclaimer.
Rationale: although it is possible to prove with certainty when something does not conform, the reverse is not necessarily true. Especially for functional
specifications, where a claim goes beyond syntax testing, a claim of
conformance is not a guarantee that the claimant is 100% conforming with the
specification. A disclaimer can help clarify the meaning of a conformance
claim as well as its limitations. For example, this document contains a
conformance disclaimer.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST NOT include any restriction about who can make a claim nor where claims can be published.
Claimants (or relevant assuring parties) are solely responsible for the
validity of their claims, keeping claims up to date, and proper use of the
conformance icons.
Examples & Techniques for this checkpoint.
An Implementation Conformance Statement (ICS) provides a mechanism whereby a supplier of an implementation of the specification provides information
about the implementation in a standardized manner. It is used to indicate
which capabilities and options have been implemented, as well as the
limitations of the implementation. An ICS usually takes the form of a
questionnaire where product implementors report on the conformance of their
product to the named specification.
An ICS is useful in disclosing optional functionality and discretionary
behavior and values. The results of the ICS can be used to identify the
subset of test cases from a conformance test suite that are applicable to the
implementation to be tested. This will allow the implementation to be tested
for conformance against only the relevant requirements.
The basic and detailed information that an ICS provides can also be used
to assess and deduce the interoperability potential of two or more
products.
Checkpoints
To fulfill this checkpoint, a specification MUST either:
- include or reference an Implementation Conformance Statement.
- explain why an ICS is not needed
Rationale: the inclusion of an ICS may not be applicable to all specifications. A specification's ICS can be thought of as providing a minimal list of items supported by the implementation, capturing information the WG deems necessary to support conformance claims.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST include the ICS as part of its conformance claim requirements.
This checkpoint is not applicable, if an ICS is not applicable to the specification.
Rationale: the ICS is coupled with the requirements for making a conformance claim (guideline 11), thus providing specific information about the implementation and substantiating the conformance claim.
Examples & Techniques for this checkpoint.
There is a lot to be said for consistency and clarity within a document -
it facilitates the understanding and comprehension of the document. Authors
and editors of specifications should already be familiar with the W3C
Manual of Style [STYLE-MAN] and
Publication Rules (member-only) [PUBRULES], which help to achieve this. With
respect to conformance, it is important to provide clear and unambiguous
statements, so that the reader knows what is required in order to claim
conformance and what is optional. To achieve this objective, throughout the
document, employ uniformity of structure and style, and consistency of
terminology and phraseology.
Checkpoints
To fulfill this checkpoint, a specification MUST use RFC 2119 keywords to denote whether or not requirements are mandatory, optional, or suggested.
Rationale: Using these keywords helps to identify the
testable statements in a specification.
Examples & Techniques for this checkpoint.
Normative statements are the prescriptive parts of the specification
whereas informative statements are for informational purposes and assist in
the understanding or use of the specification.
To fulfill this checkpoint, a specification MUST distinguish normative text from informative text.
Rationale: it is important that the
reader be able to distinguish between normative and informative statements in
order to know what is required to claim conformance. SMIL 2.0 is an example,
indicating within every subsection whether it is normative or informative,
and even separately labelling pieces of subsections that contain both kinds
of text.
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST use identical wording to express identical provisions, and analogous wording to express analogous provisions .
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST provide at least one navigation mechanism that allows the reader to locate all conformance-related information that is relevant to the specification. The mechanism MUST minimally locate:
- the conformance clause;
- any conformance section;
- unambiguous statements about those DoV that the specification employs, from among the eight defined in this specification;
- requirements for conformance claims.
Rationale: A reader must be able to easily identify and locate all the information necessary to understand the conformance policy and related conformance information without having to read the document from cover to cover. A navigation mechanism adds to the usability of the specification.
Examples & Techniques for this checkpoint.
A test assertion is a statement of behavior, action or
condition that can be measured or tested. It is derived from the
specification's requirements and bridges the gap between the narrative of the
specification and the test cases. Each test assertion is an independent,
complete, testable statement for requirements in the specification. Each test
assertion results in one or more test cases. Multiple test assertions can be
combined to form a test case, in this case one tests multiple facets of a
particular behavior.
Checkpoints
To fulfill this checkpoint, a specification MUST include or reference normatively a list of test assertions stated in it.
In order to enable pointing to test assertions from tests as well as to
give a map of the specification from the point of view of tests, use a
mechanism for making explicit test assertions in the specification.
Rationale: providing test assertions facilitates and promotes the development of test materials. Tests can point directly to the test assertion. Specific benefits include:
- Detailed reporting when testing an implementation using the test;
- Detailed control over what parts of a specification an implementation
supports, grouped by behavior, technical aspect (method/interface) or
other conceptual grouping;
- if the test assertions are included in the specification, they are publicly
reviewed and agreed to;
Examples & Techniques for this checkpoint.
To fulfill this checkpoint, a specification MUST provide a mechanism linking each test assertion to the part of the specification it is stated.
Rationale: this allows both to ensure consistency between the specification and the test assertions list and to facilitate building a test suite framework based on the test assertions list.
Examples & Techniques for this checkpoint.
This section defines conformance of Working Group specifications — i.e.,
technical reports — to the requirements of this QA Framework guidelines
specification. The requirements of this guidelines specification are detailed
in the checkpoints of the preceding "Guidelines" chapter, and apply to the
technical reports produced by Working Groups.
3.1. Normative sections
The following sections are normative in this document:
- all the sentences using a capitalized keyword from RFC 2119 (typically, one paragraph per checkpoint)
- the non-applicability clauses of each checkpoint
- the conformance section
Any other section not explicitely marked as normative is assumed to be informative.
The test assertions of this Specification Guidelines document are found in
the prioritized checkpoints. A checkpoint will contain at least one, and may
contain multiple individual requirements. These requirements are the test
assertions of this specification. A checkpoint is satisfied by satisfying all
of the individual requirements. Failing one individual mandatory requirement means that
the checkpoint is not satisfied. Mandatory requirements are those that use the conformance keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", or "SHALL NOT".
This section defines three degrees of conformance to this guidelines
specification:
- A-Conforming: all Priority 1 checkpoints are
satisfied;
- AA-Conforming: all Priority 1 and Priority 2
checkpoints are satisfied;
- AAA-Conforming: all Priority 1, Priority 2, and
Priority 3 checkpoints are satisfied;
A specification conforms to the QA Framework: Specification
Guidelines at degree X (A,
AA, or AAA) if the Working Group meets at
least all degree X conformance requirements.
To make an assertion about conformance to this document, specify:
- The guidelines title: QA Framework: Specification
Guidelines
- The guidelines dated URI: http://www.w3.org/TR/2002/WD-qaframe-spec-20021108/
- the date of the claim
- the URI of a frozen dated version of the specification that is claiming conformance
- The degree of conformance satisfied: A-Conforming, AA-Conforming, or
AAA-Conforming.
Example:
This specification conforms to W3C's QA Framework: Specification
Guidelines, available at http://www.w3.org/TR/qaframe-spec/,
AA-Conforming.
The list of checkpoints of this specification ([SPEC-CHECKLIST]) is the Implementation Conformance Statement (ICS) pro-forma for this specification. Any specification claiming a degree of conformance with this specification MUST link to a completed I.C.S.
The checkpoints of this guidelines specification present verifiable
conformance requirements about the specifications (technical reports) that
Working Groups produce. As with any verifiable test requirements, it is also
true of these specification requirements that:
- Passing all of the requirements to achieve a given degree of
conformance — A, AA, or AAA — does not guarantee that the subject
specification is well-suited to or will achieve its intended purposes,
nor does it guarantee the quality or suitability of test materials
produced from the specification.
- Failing to achieve conformance degree of at least A-conforming does not
mean that the subject specification is necessarily deficient to its
intended purposes, nor does it mean that it is an unacceptable basis for
the development of quality test materials. It means that the
specification has failed one or more checkpoints that best-practice
experience has shown to improve the testability and usability of
specifications, and to facilitate the timely and successful development
and maintenance of quality test materials based on the specification.
[] This section contains terms used in this
specification, with functional or contextual definitions appropriate for this
specification. See also [QA-GLOSSARY]. Some terms
in this section have been borrowed or adapted from other specifications.
- conditional conformance
- []
- conformance clause
- a part or collection of parts of a specification that defines the
requirements, criteria, or conditions to be satisfied by an
implementation or application in order to claim conformance
- conformance level
- a variety of conformance designation. Other designations include
conformance category, conformance degree, conformance xxx,...
"Conformance level" is discouraged in new specifications, because of
confusion with "functional level".
- dimensions of variability
- the ways in which different products that are conformant to a
specification may vary among themselves. In this Specification
Guidelines document, the dimensions of variability are used to help
organize, classify and assess the conformance characteristics of W3C
specifications
- discretionary items
- deliberate and explicit grants of discretion by the specification, to
the implementations, that describe or allow optionality of behavior,
functionality, parameter values, error handling, etc.
- functional level
- a technology subset that is one of a hierarchy of nested subsets,
ranging from minimal or core functionality to full or complete
functionally.
- implementation conformance statement (ICS)
- a mechanism for providing standardized information about an
implementation of a named specification, usually in the form of a
questionnaire on which product implementers report about the product's
conformance to the specification. An ICS is used to indicate which
requirements, capabilities and options have and have not been
implemented.
- informative text
- text in a specification whose purpose is informational or assistive
in the understanding or use of the specification, and which contains no
test assertions or conformance requirements.
- level
- a commonly used shorthand for functional level.
- module
- a collection of semantically-related elements, attributes, and
attribute values that represents a unit of functionality. Modules are
non-hierarchical, discrete divisions that are defined in coherent
sets.
- normative text
- text in a specification which is prescriptive or contains conformance
requirements.
- profile
- a subset of a technology that is tailored to meet specific functional
requirements of a particular application community. A profile may
address a single technology; or, a profile can also group a set of
technologies (i.e., from different specifications) and define how they
operate together. Profiles may be based on hardware considerations
associated with target product classes, or they may be driven by other
functional requirements of their target communities.
- profiling
- a method for defining subsets of a technology by identifying the
functionality, parameters, options, and/or implementation requirements
necessary to satisfy the requirements of a particular community of
users.
- strict conformance
- conformance of an implementation that employs only the requirements
and/or functionality defined in the specification and no more (i.e., no
extensions to the specification are implemented).
- test assertion
- any sentence or equivalent assemblage of words and phrases that
prescribes the behavior that must be obtained when a stimulus occurs
under a certain set of conditions. (See also QA Glossary
[QA-GLOSSARY].)
- unconditional conformance
- use case
- a specification mechanism or technique that captures the ways a
specification would be used, including the set of interactions between
the user and the specification as well as the services, tasks, and
functions the specification is required to perform.
- usage scenario
- an instance of a use case, that represents a single path through the
use case. Thus, there may be a scenario for the main flow through the
use case and another scenarios for each possible variation of flow
through the use case (e.g., representing each option).
The following QA Working Group and Interest Group participants have
contributed significantly to the content of this document:
- Daniel Dardailler (W3C)
- Dimitris Dimitriadis (Ontologicon)
- Karl DuBost (W3C)
- Peter Fawcett (RealNetworks)
- Kirill Gavrylyuk (Microsoft)
- Dominique Hazael-Massieux (W3C)
- Lofton Henderson (CGM Open)
- Ian Jacobs (W3C)
- David Marston (IBM Research)
- Jack Morrison (Sun Microsystems)
- Lynne Rosenthal (NIST)
- Alex Rousskov (Measurement Factory)
- Mark Skall (NIST)
- Andrew Thackrah (Open Group)
- Olivier Thereaux (W3C)
6.1. Normative
- RFC2119
- Key words for use
in RFCs to Indicate Requirement Levels, March 1997,
available at http://www.ietf.org/rfc/rfc2119.txt.
6.2. Informative
- CSS2
- Cascading Style
Sheets, Level 2, W3C Recommendation, 12 May 1998, available
at http://www.w3.org/TR/REC-CSS2.
- MATHML20
- Mathematical Markup
Language (MathML) Version 2.0, W3C Recommendation, 21
February 2001, available at http://www.w3.org/TR/MathML2/.
- PUBRULES
- W3C Publication
Rules, available at http://www.w3.org/Guide/pubrules.html
(member-only).
- QA-GLOSSARY
- Comprehensive glossary of QA
terminology. (Under construction, at
http://www.w3.org/QA/Glossary.)
- QAF-INTRO
- QA
Framework: Introduction, L. Henderson, D. Dimitriadis, K.
Gavrylyuk, L. Rosenthal, Eds., W3C Working Draft, November 2002, companion
version to this document, available at
http://www.w3.org/TR/2002/WD-qaframe-intro-20021108/ .
- QAF-OPS
- QA
Framework: Operational Guidelines, K. Gavrylyuk, D.
Dimitriadis, L. Henderson, L. Rosenthal, Eds., W3C Working Draft, November
2002, companion version to this document, available at
http://www.w3.org/TR/2002/WD-qaframe-intro-20021108/ .
- QAF-TEST
- QA Framework: Test Guidelines, not yet published, see Table of Seven (at
http://www.w3.org/QA/WG/#docs), for most-recent links to any WG-only or
published version(s).
- QAIG
- Quality Assurance Interest
Group of the W3C QA Activity, which may be found at
http://www.w3.org/QA/IG/.
- QAWG
- Quality Assurance Working
Group of the W3C QA Activity, which may be found at
http://www.w3.org/QA/WG/.
- SMIL20
- Synchronized
Multimedia
Integration Language (SMIL 2.0), W3C Recommendation, 07
August 2001, available at http://www.w3.org/TR/smil20/.
- SPEC-CHECKLIST
- An appendix to this specification guidelines document presents all
checkpoints in tabular form. Available at
http://www.w3.org/TR/2002/WD-qaframe-spec-20021108/qaframe-spec-checklist .
- SPEC-EXTECH
- QA Framework: Specification Examples & Techniques, available at http://www.w3.org/QA/WG/2002/11/qaframe-spec-extech.
- STYLE-MAN
- W3C Manual of
Style, summarizing the style and publication rules for W3C
technical reports, available at http://www.w3.org/2001/06/manual/.
- SVG11
- Scalable Vector Graphics
(SVG) 1.1 Specification, D. Jackson, J. Ferraiolo, J.
Fujisawa, Eds., W3C Candidate Recommendation 30 April 2002, available
at http://www.w3.org/TR/SVG11/.
- SVG-MOBILE
- Mobile SVG Profiles:
SVG Tiny and SVG Basic, T. Capin, Editor, W3C Candidate
Recommendation, 30 April 2002, available at
http://www.w3.org/TR/SVGMobile/.
- TEST-EXTECH
- "QA Framework: Test Examples & Techniques", not yet published,
see Table of Seven (at
http://www.w3.org/QA/WG/#docs), for most-recent links to any WG-only or
published version(s).
- UAAG10
- User Agent Accessibility
Guidelines 1.0, I. Jacobs, J. Gunderson, E. Hansen, Eds.,
W3C Candidate Recommendation, 12 September 2001, available at
http://www.w3.org/TR/UAAG10/.
- W3C-TR
- Location of all published W3C
technical reports, see http://www.w3.org/TR/.
- WCAG10
- Web Content
Accessibility Guidelines 1.0, W. Chisholm, I. Jacobs, G.
Vanderheiden, Eds., W3C Recommendation, 5 May 1999, available at
http://www.w3.org/TR/WCAG10/.
- XHTML-MOD
- Modularization of
XHTML, M. Altheim, F. Boumphrey, S. Dooley, S. McCarron, S.
Schnitzenbaumer, T. Wugofski,Eds., W3C Recommendation, 10 April 2001,
available at http://www.w3.org/TR/xhtml-modularization/.
- XHTML-BASIC
- XHTML
Basic, M. Baker, M. Ishikawa, S. Matsui, P. Stark, T.
Wugofski, T. Yamakami, Eds., W3C Recommendation, 19 December 2000,
available at http://www.w3.org/TR/xhtml-basic/.
- XHTML11
- XHTML 1.1 -
Module-based XHTML, M. Altheim, S. McCarron, Eds., W3C
Recommendation, 31 May 2001, available at
http://www.w3.org/TR/xhtml11/
- XML10
- Extensible Markup
Language (XML) 1.0 (Second Edition), T. Bray, J. Paoli, C.
M. Sperberg-McQueen, E. Maler, Eds., W3C Recommendation, 6 October
2000, available at http://www.w3.org/TR/REC-xml.
- XML-NAMES
- Namespaces in
XML, T. Bray, D. Hollander, A. Layman, Eds., W3C
Recommendation, 14-January-1999, available at
http://www.w3.org/TR/1999/REC-xml-names-19990114/.
- XML-SCHEMA
- XML Schema Part 1:
Structures, W3C Recommendation, 2 May 2001, available at
http://www.w3.org/TR/xmlschema-1/.
- XPATH10
- XML Path Language (XPath)
Version 1.0, W3C Recommendation, 16 November 1999, available
at http://www.w3.org/TR/xpath.
- XPOINTER
- XML
Pointer Language (XPointer) Version 1.0, W3C Candidate
Recommendation, 11 September 2001, available at
http://www.w3.org/TR/2001/CR-xptr-20010911/
- XSLT10
- XSL Transformations (XSLT)
Version 1.0, W3C Recommendation, 16 November 1999, available
at http://www.w3.org/TR/xslt.
- 2002-11-08, third published WD
While keeping most of the principles behind the second published WD, this version has re-ordered the guidelines in a more logical way, and takes a more formal approach in the design of CP, where all CP has a set of test assertions. Besides, repetitions have been diminished through factorization, and non-testable or arguable CP have been removed or marked as to be moved as examples and techniques. Finally, it integrates all the issues resolutions agreed during the QA WG Tokyo face-to-face meeting and the weeks following.
- the introduction integrates explicit scope and goals as required by the GL and has been cleaned of its repetitions
- usage of RFC 2119 has been clarified with the usage of uppercase keywords
- the dimensions of variability have now their own section in the introduction, regrouping the various bits previously scattered around
- ex-GL 14 on granular grammars has been removed, some of its content being integrated in the GL on test assertions
- ex-GL 6 on conformance policy has been moved ahead on GL 3, ex-GL 7 on levels has been grouped with its related GL on profiles and modules
- every CP has now a set of test assertions using RFC 2119 keywords; most of them have an explicit rationale and some of them an example; the style sheet has been modified to highligh the test assertions
- the following CP (using numbering of the previous version) have been removed: 1.4 (
Ensure that every test assertion is covered by an example
), 2.3 (For each class of product, indicate minimal support requirements.
), 3.1 (Choose whether or not to have profiles.
), 3.2 (Include a table of contents entry.
), 4.1 (Choose whether or not to have modules.
), 4.2 (Include a table of contents entry.
), 6.5 (Include a table of contents entry.
), 7.1 (Address whether the specification uses or will use functional levels.
), 7.2 (Include a table of contents entry.
), 9.7 (Include a table of contents entry.
), 10.3 (Include a conformance clause entry in the table of contents
), 10.5 (Identify all dimensions of variability that are not used.
) 11.5 (Include a table of contents entry.
), 13.3 (Follow Web Accessibility Initiative and Internationalization Guidelines.
)
- CP 13.4 (
Provide a fast way to find conformance information
) is new
- overall, an effort has been put into using a consistent terminology across the specification and avoiding unclear or untestable statement
- the verbiage on each DoV has been vastly reduced or factorized in the DoV discussion in the introduction
- 2002-08-26, second published WD
-
Significantly reorganized and revised the first published WD. This
version produced as a series of editor's drafts. The changes below are
reverse chronological (most recent first), so more recent ones may
build on older ones.
- Handle negative disclaimers for DoV's via new
CK10.5.
- First cut at distinguishing specifications from technologies that
they describe (esp in profiles, modules, levels.)
- Add concept of SpecGL test assertions to "Conformance"
chapter.
- Define "use case" and "user scenario", revise GL1 & CK1.2 to
use them accordingly, clarify CK1.3 (Provide examples), add
rigorous CK1.4 (examples--TAs).
- Clean up "atomicity of modules" discussion, add atomicity to
CK4.3
- Harmonize priorities of CK8.1 & 8.2, 9.1 & 9.2.
- All GL14 and GL15 checkpoints to priority 2, except CK15.1;
tighten descriptions.
- All TOC checkpoints
to priority 2, except Conformance Clause (p1).
- Accommodate "Rules for Profiles" in GL2 categories and classes;
new CK3.6 and changes to 2.3.
- Add caveat to "Understanding and using.." (sec. 1.2), about
specifications that pre-date SpecGL.
- Rewrite CK11.1 to replace 'levels' in "conformance levels" w/
degrees, categories, etc (confusion with "functional levels").
- Change "levels" to "degrees" in SpecGL conformance clause.
- Change "discretionary behaviors" to "discretionary items" in GL8,
define three variants in GL8 verbiage.
- Extensive new prose in GL3 (profiles), GL4 (modules), GL7
(levels), describing potential relationships.
- Fixed definition of strict conformance, clarified definition and
usage of "conformance clause".
- Added "Definitions" section (in-spec glossary).
- Enhanced wording of all TOC checkpoints.
- For each of CK3.x (profiles), 4.x (modules), 7.x (levels),
x>1, added a "not applicable" disclaimer if the feature is not
used. (Except for the TOC checkpoints.)
- Change wording of CK7.x, eliminating "chosen", and add some new
explanation to 7.1
- Delete CK5.4 (conformance policy in conformance clause) and move
its verbiage to GL10 (conformance clause) and its checkpoints.
- Fix "Conformance" chapter, add "Checkpoint Priorities"
section.
- Change chapter title of "Navigating through this document" to
"Understanding and using this document"
- New text for Ckpt 3.4
- Added warnings about "excessive variability" to all
dimension-of-variability guidelines.
- Added to several DoV checkpoints a
requirement to explain/justify any variability dimension by use
cases and requirements.
- Fix separation between last two guidelines, "Granular Grammars"
and "Test Assertions".
- Introduce and motivate "dimensions of variability" (DoV) in sec
1.5
- Reordered guidelines (GL), split GL.5 into three new ones, GL.3,
GL.4, and GL.7, customized checkpoints to their topics (profiles,
modules, levels).
- Added new p3 checkpoint ("categorize specification") to (new)
GL.2
- Rewrote (old) GL.3, "conformance flavors", in terms of "specify
conformance policy, (new) GL.5.
- 2002-05-15 version
-
First published WD.