The principal goal of this document is to help W3C Working Groups in
writing clearer, more implementatable, and better testable technical reports.
It both provides a common framework for specifying conformance requirements
and definitions, and also addresses the representation of specifications
(technical reports) as schemata, both of which facilitate the generation of
test materials. The material is presented as a set of organizing guidelines
and verifiable checkpoints. This document is one in a family of Framework
documents of the Quality Assurance (QA) Activity, which includes the other existing or
in-progress specifications: Introduction; Operational Guidelines; and, Test
Guidelines.
This is an Editors draft. It is meant for QA Editors and QAWG
members internal review only. Its contents have not been endorsed or approved
by the QAWG, or by any other entity of W3C.
This is a major reordering and revision of the FPWD version
of 2002-05-15. The reordering and revision was done according to the email
proposal (by DM), which was discussed and endorsed at the Montreal
face-to-face. All guidelines (GL) are re-ordered accordingly. New GL.5
("Conformance policy", formerly GL.3) is rewritten. Old GL.5 is split into
new GL.3, GL.4, GL.7, and checkpoints are rewritten. The text in the
checkpoints of the latter is rough, and there are numerous open issues.
The goal of this series of WG drafts is to start to organize and get a
handle on the dimensions of variability, as a basis for re-opening discussion
on the IG list about how to maximize interoperability.
The initial reorganization has now been refined once according to the
results of the telecons of 2002-06-27, 2002-07-10 and subsequent email
discussion, 2002-07-24 "unofficial" telecon, 2002-07-31, 2002-08-04, and
subsequent email. Several issues remain, in general flagged by "@@".
Following is proposed draft SoTD for next
publication...
This section describes the status of this document at the time of its
publication. Other documents may supersede this document. The latest status
of this document series is maintained at the W3C.
This document is a W3C Working Draft (WD), made available by the W3C Quality
Assurance (QA) Activity for discussion by W3C members and other interested
parties. For more information about the QA Activity, please see the QA Activity statement.
(@@new)This version is the second published WD, and supersedes all
previous WG-only drafts as well as the first published WD. It is expected
that updated WD versions of this
document will be produced regularly, along with other members of the
Framework documents family. Future progression of this document beyond
Working Draft is possible, but has not yet been determined.
This is a major reordering and revision of the first published WD
version, 2002-05-15. The reordering and revision was done according to
the email
proposal -- which was discussed and endorsed at the Montreal face-to-face
(@@link) -- in response to issues raised on the discussion list about the
first published WD. Note that significant comments about old Guideline (GL)
12 (now GL14 and GL15) -- Theme 2, specification anatomy -- have not been
addressed in this version. It is anticipated that this is the next major
undertaking of the QAWG.
Rather, version has focused on a complete reorganization and restructuring
of the guidelines (GL) of Theme 1, Conformance Content. Old GL1 - GL11 are
restructured into 13 new guidelines, organized around a central concept of
specification "dimensions of variability" (DoV). The QAWG has done this with
the specific goal of presenting it to the W3C, to solicit feedback on:
- is this a useful organization that QAWG should pursue, develop, and
refine?
- are there dimensions that we can eliminate, or major dimensions that we
have missed (and that can't be fit into an existing dimension)?
- can we start to make and integrate judgements about goodness and
badness within particular dimensions or combinations of dimensions?
About #3, in this version the QAWG has included checkpoints that require
explanation and justification of the use of DoVs within a specification, and
has warned of the dangers of excessive variability, but has generally avoided
specific value judgements. There are several specific goodness issues about
which the QAWG desires feedback:
- (...let's put 'em in the issues list, as well as point from
here...)
- (levels-profiles, levels-modules, and minimality of levels across other
DoV)
- (@@enumeration of others?)
A future version of this document will be accompanied by a "Specification
Examples & Techniques" document, which will illustrate the guidelines and
checkpoints with case studies, and explain how to satisfy the checkpoints.
Please send comments to www-qa@w3.org,
the publicly
archived list of the QA Interest
Group [QAIG]. Please note
that any mail sent to this list will be publicly archived
and available. Do not send information you wouldn't want to see distributed,
such as private data.
Publication of this document does not imply endorsement by the W3C, its
membership or its staff. This is a draft document and may be updated,
replaced, or made obsolete by other documents at any time. It is
inappropriate to use W3C Working Drafts as reference material or to cite them
as other than "work in progress".
A list of current W3C Recommendations and other technical documents can be
found at http://www.w3.org/TR.
1. Introduction
1.1 Motivation for this guidelines document
1.2 Understanding and using this document
1.3 Checkpoint priorities
1.4 Terminology
1.5 Major Themes
1.6 Variability, complexity, and roadmap of guidelines
2. Guidelines
G 1. Define user scenarios.
G 2. Identify what needs to conform and
how.
G 3. Address the use of profiles to divide
the specification.
G 4. Address the use of modules to divide the
specification.
G 5. Specify conformance policy.
G 6. Clarify the
relation between deprecated features and conformance.
G 7. Address the use of levels to divide the
specification.
G 8. Define discretionary items.
G 9. Allow
extensions or NOT!
G 10. Provide a conformance clause.
G 11. Specify how to make conformance
claims.
G 12. Publish an Implementation
Conformance Statement proforma.
G 13. Support general document conformance
conventions.
G 14. Use granular grammars to author the
specification.
G 15. Include test assertions.
3. Conformance
3.1 Conformance definition
3.2 Conformance disclaimer
4. Definitions
5. Acknowledgments
6. References (non-normative)
7. Change history
An appendix to this document [SPEC-CHECKLIST] presents all checkpoints in a tabular form, for
convenient reference.
1. Introduction
The quality of the specification has direct impact on the quality of
implementations. Quality encompasses utility which refers to the
usefulness of the specification to the intended users and
objectivity which focuses on the whether the specification is
presented in an accurate, clear, complete, and unbaised manner.
This document defines a common framework for specifying conformance
requirements and definitions, and for representing the structure of the
document as schemata, both of which facilitate the generation of test
materials. The primary goal is to assist W3C Working Groups (WGs) by
providing guidelines and verifiable checkpoints for writing clearer, more
implementable, and better testable specifications (technical reports). Good
specifications lead to better implementations and foster the development of
conformance test suites and tools. Conforming implementations lead to
interoperability.
These guidelines were developed so that WGs can apply them in a
common-sense and workable manner. This document identifies the conformance
requirements and statements to be included or addressed in specifications as
well as addressing the anatomy of the specification. Conformance requirements
are the expressions that convey the criteria to be fulfilled in an
implementation of a specification. The conformance requirements are stated in
a conformance clause or statements within the specification. The anatomy of
the specification pertains to the method for writing the specification using
schemata. A specification represented by an XML grammar with sufficient
granularity of its information facilitates the generation of test materials
by providing the ability to point to, extract, query, manipulate and/or
automatically generate test materials. Given the symbiotic connection between
specification and test materials, this document also addresses the intermixed
quality practices of specification authoring and test material production and
maintenance within the W3C process.
This document is part of a family of QA Framework documents designed to
improve the quality of W3C specifications as well as their implementations by
solidifying and extending current quality practices within the W3C. The QA
Framework documents are:
The process for developing testable technical reports and specifications
is affected by QA activities beyond those that are explicitly provided in
this document. Specifically, the QA Framework documents are interrelated and
complement each other. For example, the anatomy of a specification is
dependent on the type of test materials that will be built - hence there is
interrelationship between this document and the Test Guidelines. Links
between applicable guidelines in this document and the other Framework
documents will be given.
The guidelines are intended for all Working Groups as well as developers
of conformance materials for W3C specifications. Not only are the Working
Groups the consumer of these guidelines they are also key contributors. The
guidelines capture the experiences, good practices, activities, and lessons
learned of the Working Groups and present them in a comprehensive, cohesive
set of documents for all to use and benefit from. The objective is to reuse
what works rather than reinvent and to foster consistency across the various
Working Group quality activities and deliverables.
1.1 Motivation for this guidelines document
The ultimate goal of a specification is to be implemented. In order to be
implemented, a specification needs to be written such that developers and
users can comprehend what is expected of them and build correct, robust,
interoperable software and content. Correct utilization of specifications
leads to portability and interoperability. In particular, it is easier to
implement clear specifications, where the possibility of misinterpretation
has been eliminated, safeguarding the quality of implementations. In
addition, if care is given to the specifications, interoperability between
W3C technologies is easier to track and assert.
Given that the W3C invests time and resources into producing
specifications that are eventually implemented, and especially given that
some of those specifications are interrelated, it makes sense to enhance and
streamline the specification authoring process. This should be done for three
main reasons:
- More straightforward specifications are easier to understand and
implement
- They foster the development of conformance materials
- It is easier to construct W3C-generic testing frameworks if both the
test framework and what is being tested, i.e. implementations and
specifications, follow generally accepted and implemented rules.
The W3C needs to ensure that the deliverables of its WGs are clear,
implementable and sound technical specifications with testable requirements
and that can be used to easily generate Quality Assurance materials (tests
and suites). A specification is testable if there exists some finite
cost-effective process with which a person or software can check that the
requirement has been met.
The benefits of producing clear, testable specifications include:
- Straightforward to assess and assert conformance
- Easy to generate tests and test suites
- Given that specification authoring is standardized, it will be
straightforward to test interdependencies between specifications (for
example it is easier to point to relevant parts of other specifications
with regard to technical details where specifications share common
features) as well as test interoperability between implementations of
different W3C specifications.
The benefits derived by writing granular specifications that provide
detailed control over the information in the specification, include
- Information extraction directly from the specification
- Generating a test harness directly from specifications
- Traceability
- Generating graphical representations of and interfaces to
specifications
- Efficient and effective test development
In particular, information from the specifications is easier to extract if
specification authoring involves a standardized set of specification schemas
where both technology and intended behaviour is specified. Tests can be
generated directly from the specifications, where needed, to generate primers
for test suites. In this way, a fairly broad coverage of test cases can be
achieved in a straightforward way. An added bonus is that a graphical
representation of the specification can be generated in parallel to the
normative HTML version. Specification granularity allows for more concise
reporting when running test suites by pointing directly to the relevant part
of the specification. Specification granularity allows for generating tests
that can be validated against the specification itself, thus making it easier
in those cases where different interpretations of the normative text are
given.
Use of a formal (or formal-like) specification language may enable
'validation' of the' specification as well as auto generation of tests or
test components.
Synchronizing writing specifications with building test materials provides
a feedback loop. Testing can help find ambiguities, inconsistencies, and
holes in a specification.
In particular
- Ambiguities and misinterpretations are more easily caught and can be
resolved earlier in the specification writing process
- Given the possibility to automatically generate primers for test
suites, coverage can be tracked fairly easily and therefore enhanced.
Corner cases are easy to identify and write tests for.
Interaction between WGs and other bodies in specification authoring is
related to test suite production and maintenance. The reasons to develop
specifications, test assertions, and test materials together are:
- help to uncover inconsistencies, ambiguities, gaps, and non-testable
statements in the specification by developing the test assertions in
parallel with the specification.
- ensure consistency between the specification and assertions,
- allow test assertions to be reviewed and accepted by the specifications
developers and the public,
- provide a common set of assertions from which test developers can
develop test materials
- encourage the early development of conformance tests that can be used
by implementers during the development of their implementation,
- achieve confidence in the resulting test materials as a measure of
conformance.
1.2 Understanding and using this document
Since this document is a member of a family of documents, the reader is
strongly encouraged to be familiar with the other documents in the family.
There is an interrelationship between the Guidelines in this document and the
Guidelines in the other Framework documents. In particular, Guidelines 1, 2,
3 and 5 in the QA Framework: Operational Guidelines [QAF-OPS
] are especially relevant and interrelated
to the Guidelines in this document.
This document describes what goes into the specification with respect to
conformance and conformance topics, followed by rules for specification
anatomy that should facilitate test development as well as produce better,
more testable specifications. It does not preclude the need to apply the
W3C Manual of
Style
[STYLE-MAN] and to conform to the
Publication
Rules
[PUBRULES
] (member-only).
This document employs the WAI (Web Accessibility Initiative) model for
representing guidelines or general principles for the development of
conformance materials. See, for example,
Web Content Accessibility Guidelines
1.0
[WCAG10]. Each guideline includes:
- The guideline number
- The statement of the guideline
- The rationale behind the guideline
- A list of checkpoint definitions.
The checkpoint definitions in each guideline define the processes and
operations that need to be implemented in order to accomplish the guideline.
Each checkpoint definition includes:
- The checkpoint number
- The statement of the checkpoint.
- The priority of the checkpoint.
- Optional informative notes, clarifying examples, and cross references
to related guidelines or checkpoints.
Each checkpoint is intended to be specific enough so that someone can
implement the checkpoint as well as verify that the checkpoint has been
satisfied. A checkpoint will contain at least one, and may contain multiple
individual requirements that must all be satisfied in order to satisfy the
checkpoint. These requirements are the test assertions of this Specification
Guidelines document.
[@@new]Caveat. The checkpoints are written in anticipation that they are
being applied to new, in-development specifications. It is not expected that
existing specifications that pre-date these guidelines will be able to
satisfy all checkpoints as they are stated. It is expected that legacy
specifications may indirectly comply with the spirit or intent of some
checkpoints, without actually conforming to the test assertions in those
checkpoints.
A separate appendix to this document [SPEC-CHECKLIST] presents all checkpoints in a tabular form, for
convenient reference.
1.3 Checkpoint priorities
Satisfaction of the checkpoints in this guidelines specification are key
requirements to produce a high quality, testable standard that is a suitable
basis for successfully interoperable products. Some checkpoints are more
crictical than others. Therefore each checkpoint is assigned a priority level
based on the checkpoint's impact on the quality of the specifications
produced by the Working Groups.
- [Priority 1]
- These checkpoints are considered to be basic requirements for
ensuring an acceptable level quality and testability in the
specification.
- [Priority 2]
- Satisfying these checkpoints, in addition to the priority 1
checkpoints, should significantly improve the clarity, quality, and
testability, of the standard, as well as its suitability as a basis for
building quality test materials.
- [Priority 3]
- Satisfying these checkpoints, on top of all the others, is a
significant step toward ensuring the highest levels of quality and
testablity of the specification.
1.4 Terminology
The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" will be used as
defined in RFC 2119 [RFC2119].
Unusual terms in these framework documents are defined when first used.
This document contains a "Definitions" chapter.
Most generally useful QA-specific terms will eventually be in the QA Glossary
[QA-GLOSSARY].
1.5 Major Themes
The guidelines address two general themes: specifying conformance
requirements in the specification; and addressing the anatomy of the
specification.
Theme 1, Conformance content: general
requirements and definitions concerning conformance and related issues. It is
intended to contribute towards mutual understanding amongst developers of
specifications and conformance test suites and tools. It does not define
specific conformance requirements for any specific specification - this is
the responsibility of the WG chartered to develop the specification. For the
developers of specifications, it helps them to develop conformance
requirements and to create testable, unambiguous specifications. A
conformance clause:
- promotes a common understanding of conformance and what is required to
claim conformance to a specification
- facilitates consistent application of conformance within a
specification
- facilitates consistent application of conformance across related
specifications,
- promotes interoperability and open interchange,
- encourages the use of applicable conformance test suites,
- promotes uniformity in the development of conformance test suites
Theme 2, Anatomy: invest in writing granular specification with careful
modeling of the information conveyed. This allows for:
- detailed control over information extraction from the spec
- generating tests directly from the specs
- easy graphic interface to specification
1.6 Variability, complexity, and roadmap of guidelines
There are 15 guidelines that are presented in a sequence that runs
approximately from the general to the specific. The later (higher-numbered)
guidelines are likely to depend upon decisions made in accordance with
earlier guidelines.
The first four guide the establishment of strategic objectives and address
the overall motivation for the creation of the specification. The next two
bring the QA concerns into the plan. Guidelines 7 through 9 are about
additional specification tools to adjust the boundaries between conformant
and non-conformant implementations. The final six guidelines pertain to
particular documentation features.
- Scope and use cases (often in Requirements Doc)
- Specification categories and product classes
- Profiles (hardware, environment, etc.)
- Modules
- Conformance policy
- Deprecation policy
- Levels
- Discretionary items
- Extensibility
- Conformance clause
- Claims of conformance
- Proforma
- Document conventions
- Granular grammar
- Test assertions
[@@tech-spec: in GL3,4,7 at least, DH proposal to substitute "technology"
for "specification" in ckpt statement and appropriate points GL verbiage.
Here we would discuss the distinction, e.g., "It is necessary to clearly
distinguish between the specification -- the W3C Technical Report -- and the
technology that it standardizes. For example, in GL3, profiling is a method
to partition the technology into groups, and profiles are technology groups.
Profiling and profiles of the technology may or may not lead to division of
the specification into multiple parts or units, concurrent or subsequent.
This document will strive to keep the distinction clear, even though in
common usage concepts such as profiles typically are used to refer to
both.@@]
In dealing with Theme 1, considerable complexity is
introduced by a basic reality of diverse W3C specifications: multiple
dimensions of variability (DoV) can be accommodated in a specification. The
WG may determine that conformant products are allowed to vary in one or more
ways. Guidelines 2 through 9 cover the "dimensions of variability" that a
specification may permit. This document recognizes the following dimensions
of variability:
- product classes (Guideline 2)
- profiles (Guideline 3)
- modules (Guideline 4)
- conformance policy (Guideline 5)
- deprecation (Guideline 6)
- levels (Guideline 7)
- discretionary choices (Guideline 8)
- extensibility (Guideline 9)
The above are not necessarily all orthogonal to one another -- some of the
possible associations and relationships are noted in the individual
guidelines.
The dimensions of variability comprise a tool used in this guidelines
document to help organize, classify and assess the conformance
characteristics of W3C specifications. While variability complicates
interoperability, its net effect is not necessarily negative in all cases,
when compared to the alternatives -- e.g., profiles, compared to a large
monolithic specification. Different sorts of variability have different
negative and positive impacts. In any case, specification writers need to
carefully consider and justify any conformance variability allowed, do so by
reference to the project requirements and use cases, and explicitly document
the choices made.
Guideline 1. Define user scenarios.
When writing specifications it is critical to understand their primary
purpose and scope. If the specification describes a protocol or Application
Programmer Interface (API) -- examples XML
Protocol, DOM -- there
should be a clear understanding of the primary user scenarios. If the
specification describes content requirements (for example, [WCAG10], [UAAG10]), understanding of
their purpose is the key to defining the minimal sufficient set. Clearly
defined scope helps to keep the specification content focused and
unambiguous.
When reading the specification, we face similar difficulties. To
understand what the document says, the reader needs to know the context of
the author, and the scenarios the author had in mind. In case of protocols
and APIs specifications developers try to assess whether the specifications
covers the user scenarios the product needs to cover. Having no user
scenarios described in the specification generates misuses of the
specification itself.
Checkpoint 1.1. Define
the scope of the specification.
[Priority 1]
This is very basic but powerful requirement. The specification must
clearly define what scenarios are in scope and what are out of scope in order
to be interpreted, implemented, and tested correctly.
Checkpoint 1.2. Include Use Cases.
[Priority 2]
A Use Case is a description of the user scenario given in formally defined
terms. Once included in the specification Use Cases become normative, unless
otherwise specified. (@@how? what MUST requirement do they convey? and on
what class of product do they impose it? it seems like a use case is a
requirement on the specification itself. New issue #72 logged)
The specification should have extensive list of the orthogonal Use Cases the
authors have in mind. Priorities MAY be assigned to the Use Cases, describing
how important the particular scenario is for the specification. Use Cases in
their turn may help to assess what features are missing and what features are
superfluous in the specification.
Checkpoint 1.3. Include examples.
[Priority 1]
It is good practice to illustrate each behavior or requirement in the
specification by short and precise examples. (per SM comment, too vague to
verify "each behavior or requirement". This checkpoint is going to need some
refined verbiage in order to make it verifiable.)
Checkpoint 1.4. Include an interpretation section.
[Priority 2]
It is hard to understand the formal descriptions of content without
informative interpretations. The recent complex specifications like XML Schema [XML-SCHEMA] and XML Protocol have shown an necessity
to have a "Primer" part or section to illustrate how to use the
specification. Such a section does not substitute for the numerous examples
needed for each requirement or behavior described in the specification, but
illustrates how the specification fulfills the User Scenarios.
Guideline 2. Identify what needs to conform and
how.
Categorizing the specification provides a basis for classifying the
software that may be affected by the specification -- and thus, provides the
answer to "what needs to conform?". To answer this question, it helps to look
at the nature of the specification and categorize it. Most specifications can
be classified into one of the following categories:
- foundation or abstract (e.g., Infoset),
- content/data [@@Dom comment] (e.g., MathML, SVG),
- protocols (e.g., SOAP),
- processors [@@Dom comment] (e.g., XSLT, XML Query),
- APIs (e.g., DOM),
- notation/syntax (e.g., XPath),
- set of events (e.g., one part of XForms)
- rules for deriving profiles (e.g., part of SMIL) [@@new]
The categories indicate what the specification describes. One
specification could potentially fall into more than one category.
From this categorization of specifications, the WG can identify the "class
of products" that are affected by the specification. Classes of products can
be generalized as either or producers or consumers, or as content itself.
Identifying which are producers and consumers is very clear for a
protocol-type specification, the two parties to the dialog are the targets.
For a processor-type specification, the processor is the consumer of an XML
vocabulary defined in the specification. For content-type specifications,
there may be one or more consumers that take the content and 'play' it in
some way.
The following is a non-exhaustive list of classes of products for W3C
specifications.
- content (of type, meaning, and format as defined in the
specification)
- producer of content (may be divided into initiators and modifiers)
- player (read-only consumer, conveys content in non-XML way)
- consumer in a one-way pipeline
- responding agent (e.g., server) of API (consumer and producer)
- processor (consumer of its vocabulary/instructions)
- module that hosts the processor
- producer of instructions/commands to processor
- profile derived from the spec's Rules for Profiles [@@new]
One specification could define more than one player. For example, MathML
could address the behavior of display of math notation and also a consumer
that parses the MathML as a formula and uses it for mathematical
processing.
The conformance clause identifies the "class of products" (i.e., object of
the claim) that is the target of the specification. In addition to
indentifying what conforms (i.e., class of products), it describes how
conformance can be met. This would be a description of the conformance
requirements, conditions and/or criteria for each class of product.
Checkpoints
Checkpoint 2.1. Identify all classes of product.
[Priority 1]
This checkpoint can be broken into several individual requirements:
- The specification must be clear about those classes of product for
which it is defining conformance.
- For those product classes which match one of the above enumerated classes, use the above
terminology to describe the class. This requirement is not applicable for
specifications which pre-date these Specification Guidelines.
- If the specification addresses a product class that doesn't match the
enumerated set, then the specification must define and describe the
class.
Example. Within the
SMIL 2.0 Language
Profile
([SMIL20], chapter 13), there are 2
classes of products: documents and basic user agents. Within
Mathematical Markup Language (MathML)
2.0
[MATHML20] there are output-compliant,
authoring tools and input-compliant, rendering/reading tools.
Example. Schema Part 1 could be said to define an abstract notion of
"schema validity" but this checkpoint can only be satisfied if the
Recommendation says explicitly whether it is setting expectations of an XML
parser or of a "schema validator" that could be standalone. Schema Part 2
defines data types, so it is a specification of type 2 above -- content/data
-- and it is used as a foundation by other specifications (e.g., XPath 2.0)
as well as being part of the schema validator and hence parser requirements.
The specification could define the behavior of a data-input tool that rejects
data not fitting the schema, but it probably wouldn't because the tool would
be expected to use a parser module to validate the data. To satisfy this
checkpoint, the Schema Part 2 specification has to make clear whether it is
to be taken as an independent specification of a parser (that
produces data from arbitrary strings) or a foundation to be used by
other specifications.
For specifications that are used as components of other specifications and
rely on these other specifications to define conformance,
- indicate that conformance criteria for independent implementations are
not defined within the specification.
Example. XPath (
XML Path
Language 1.0
, [XPATH10]) relies on
specifications that use XPath -- such as XPointer (
XML Pointer
Language
, [XPOINTER]) and XSLT (
XSL Transformations 1.0
, [XSLT10]) -- to specify criteria for conformance of
implementations of XPath.
Checkpoint 2.2. For each class of product, define
the conformance requirements.
[Priority 1]
The conformance requirements indicate the conditions to be satisfied in
order to claim conformance. In addition to specifying the requirements for
claiming conformance, it may be appropriate to specify that which is not a
requirement. It is likely that these conformance definitions will reference
normative text within the specification or in other related
specifications.
Checkpoint 2.3. For each class of product, indicate minimal
support requirements.
[Priority 3]
For each class of product, describe the minimum set of elements,
attributes, etc., that are required to claim conformance. If there are no
minimum support requirements -- i.e., the minimal support requirement
includes all of the functionality of the specification -- explicitly state
that. This is helpful in providing the reader a full understanding of the
conformance requirements, especially when some of the classes of products
have minimal requirements whereas others do not.
If there are distinct minimal requirements, then -- implicitly or
de-facto, if not explicitly -- the specification has levels (minimal = level
1) or modules (minimal = core module or core module set). [@@new]If the
specification has profiles, then any specification-wide, cross-profile
distinct minimal requirements would represent something that all the profiles
have in common. This effectively becomes a "rule for profiles", i.e., a
conformance constraint on profiles that are valid by some definition in the
specification.
Checkpoint 2.4. Identify which of the categories of object
are specified in the document as a whole.
[Priority 3]
In the Introductory section of the specification, identify which of the
types of object (see the enumeration of
categories in the initial text of this guideline) are specified in the
document as a whole. Doing so helps keep the scope of the specification in
focus, both for the reader, and for the author(s) who must define the classes
of product and their conformance criteria.
Some specifications define more than one of the enumerated object types.
XForms is an example. It defines: Content, via an XML vocabulary; Content,
via named datatypes;
Syntax, in the form of a set of functions to
supplement the XPath core set; behavior of a
processor; behavior of
a user agent; a set of
events.
Guideline 3. Address the use of profiles to divide
the specification.
[@@tech-spec: sub "technology" for "specification" in ckpt statement and
appropriate points in the next paragraph? E.g., "Address the use of profiles
to divide the technology". And, "Dividing the technology into functional
groupings ...tbd..."@@]
Dividing a specification into functional groupings can facilitate its
implementation by enabling a developer to build to portions of the
specification rather than implementing the entire specification. Some of the
dimensions of variability, but not all, can be used to divide the
specification: product classes, profiles, modules, and (to some extent)
levels divide the specification, while extensibility separates specified
behavior from other product features.
A profile is a subset of the specification that supports a particular
functional objective. Profiles can also be used to group a set of standards
and define how they are required to operate together (e.g., XHTML plus MathML
plus SVG). Profiles may be based on hardware considerations associated with
target product classes -- for example, SVG Tiny is aimed at mobile phones --
or they may be driven by other functional requirements of their target
constituencies.
Modules are independent divisions or functional groupings of the
specification, that can be implemented independently (e.g., audio vs. video
module). See Guideline 4 for the
full discussion of the topic. Within W3C specifications, a very common way to
define profiles is to modularize the specification, and gather together
several modules to define a profile.
Levels are used to group functionality into nested subsets. See Guideline 7 for a full discussion of
the topic. A profile can require a particular level in its entirety. A
profile can also be defined as a subset of a level or levels. Hierarchy is an
essential attribute of levels. While it is not an essential attribute of
profiles, nevertheless profiles can be hierarchical in the sense that Profile
B can be defined as all of Profile A plus additional capabilities.
Specifications may define individual profiles, or may define rules for
profiles, or both. An individual profile defines the requirements for classes
of products that conform to that profile. Rules for profiles define validity
criteria for profiles themselves -- i.e., if others (users, applications, or
other standards) define their own profiles of the standard, then rules for
profiles define the constraints that those profiles must satisfy in order to
be considered valid profiles of the specification.
For example, XHTML Modularization ([XHTML-MOD],
section 3) and SMIL 2.0 [SMIL 2.0] specifications both
define rules for profiles -- what constraints must a profile meet in order to
be classified as a "Host Language Profile" or an "Integration Set Profle."
SMIL further defines some specific profiles, using the modularization.
Separate recommendations -- XHTML Basic and XHTML 1.1 (links needed here) --
define specific profiles based on the XHTML modularization.
The SMIL Recommendations (e.g., [SMIL20]) illustrate
several of these concepts at once. Modularization is an approach in which
markup functionality is specified as a set of modules that contain
semantically-related XML elements, attributes and attribute values. Profiling
is a method for defining subsets of a specification by identifying the
functionality, parameters, options, and/or implementation requirements
necessary to satisfy the requirements of a particular community of users. In
SMIL, it is the creation of an XML-based language through combining these
modules that provides the functionality required by a particular application
community. Based on a modularization framework, SMIL defines rules for
profiles -- e.g., "SMIL
Host Language Conformance" (section 2.4.1, [SMIL20]) -- as well as defining some specific profiles --
e.g., the SMIL
Language Profile and the SMIL Basic Profile
(sections 13 and 14, [SMIL20]).
It should noted that profiles are not applicable for all W3C
Recommendations. For example, subsetting of XML Namespace (@@link) would make
no sense.
Exercise caution - profiles represent one of the dimensions of variability. Excessive
variability fragments and confuses the marketplace, which inhibits
interoperability. Furthermore, dividing the conformance policy of a single
specification in two or more ways (i.e., by two or more dimensions of
variability) creates complexity -- complexity generally retards
interoperability, while simplicity generally facilitates it.
Checkpoints
Checkpoint 3.1. Choose whether or not to have
profiles.
[Priority 1]
It is possible for a specification to have one or more of profiles,
modules, levels, or none of these. If profiles are defined and supported in
the specification, explicitly say so. Furthermore, the specification must
explain why profiles are necessary, by reference to use cases and/or project
requirements. Otherwise, explicitly state that profiles are not supported.
Checkpoint 3.2. Include a table of contents entry.
[Priority 1]
The reader should be able to easily identify and locate information about
profiles. A link from the table of contents (TOC) provides this ability. The
form of TOC linkage can vary, depending on the style of the specification --
some have only a high-level TOC, some have a detailed TOCs, and some have
both. This checkpoint could be satisfied by TOC links to specific
profile-related text from a detailed TOC. For a high-level TOC, it could be
satisfied by a link to a conformance clause section that either contains or
points to profile information. It could be satisfied by both of the two
previous when both types of TOC are present.
Checkpoint 3.3. If profiles are chosen, indicate whether
or not their use is mandatory for conformance.
[Priority 1]
If profiles are defined, then
- it must be clearly stated whether conformance of each classe of product
in the specification is only defined within the context of profiles, or
whether on the other hand there can be conforming products outside of the
bounds of profiles.
For example, must content conform to one of the profiles, or is there a
concept of conformance of content independent of conformance to one of the
profiles? Is a producer (of content) conforming if it generates content that
is otherwise valid but doesn't conform to a profile?
Note. If there is a "full" profile defined -- for example, incorporating
all of the defined functionality of the specification, including
extensibility features -- then any valid content, as well as any correct
producers and fully capable consumers, might seem to be automatically using
that profile. However, profiles (e.g., of content) often include
self-identification requirements, and these would have to be observed for
conformance of valid content to even a "full" profile.
- If there are additional conditions associated with a particular profile
or collection of profiles, they must be described.
For example: if one and only one profile can be implemented at a time.
(@@do we have any example to clarify what this means?)
- A specification must address this checkpoint's requirements
individually for each class of product defined in the specification.
This checkpoint is not applicable if profiles are not chosen.
Checkpoint 3.4. If profiles are chosen, define any minimal
requirements.
[Priority 2]
A profile places requirements on each class of product that is affected by
the profile's definition.
- For each profile, define the minimal required features/support for each
class of product.
For example, for a user-agent product class, a profile must define the
minimal language support requirements for that target's user agents. This
may, for example, be done by requiring specific modules (in a modularized
specification), levels (if the specification is level-structured), or
specific lists elements and attributes.
To illustrate, SMIL 2.0 has a SMIL 2.0 Language Profile for user agents
but also provides a SMIL 2.0 Basic Profile for wireless and embedded devices.
The SMIL 2.0 Language Profile requires that a user agent implement the
BasicAnimation module but not the SplineAnimation Module. The SMIL 2.0 Basic
Profile on the other hand does not require implementation of any of the
Animation modules.
This checkpoint is not applicable if profiles are not chosen.
Checkpoint 3.5. If profiles are chosen, define their
relationships and interaction with other dimensions of variability.
[Priority 2]
Dependency or interrelationship between profiles and modules is common --
XHTML, SMIL, SVG Mobile (@@links). Less often, deprecated features, levels,
discretionary choices, or extensions could depend on profiles.
This checkpoint is not applicable if profiles are not chosen.
Checkpoint 3.6. [@@new ckpt] If profiles are chosen, address
rules for profiles.
[Priority 2]
If the specification anticipates and allows that groups may define new
profiles in the future, provide rules for derived profiles that will enable
the derived profiles to be well-specified. Derived profiles should be
specified in a way that meets all the pertinent checkpoints of this document.
Derived profiles should not clash with pre-defined profiles, if there are any
(@@in the base specification?). Checkpoints from Guideline 9 (extensions) can
be adapted into rules for profiles. Any rules for derived profiles in the
base specification must be testable, so that an independent tester can verify
conformance of a derived profile to the rules.
This checkpoint is not applicable if profiles are not chosen.
Guideline 4. Address the use of modules to divide the
specification.
[@@tech-spec: sub "technology" for "specification" in ckpt statement and
appropriate points in the next paragraph? E.g., "Address the use of modules
to divide the technology". And, "Dividing the technology into functional
groupings ...tbd..."@@]
Dividing a specification into functional groupings can facilitate its
implementation by enabling a developer to build to portions of the
specification rather than implementing the entire specification. Some of the
dimensions of variability, but not all, can be used to divide the
specification: product classes, profiles, modules, and (to some extent)
levels divide the specification, while extensibility separates standardized
behavior from other product features.
Modules are non-hierarchical, discrete divisions or functional groupings
of the specification. For example, SMIL 2.0 [SMIL20]
defines modules as follows:
A Module is a collection of semantically-related XML elements,
attributes, and attribute values that represents a unit of functionality.
Modules are defined in coherent sets.
Modules generally can be implemented independently of one another -- e.g.,
audio vs. video module. That notwithstanding, it is possible for one module's
definition (and therefore implementation) to have explicit dependency upon
another module. It is not only possible, but common to implement multiple
modules.
Modules and profiles. Within W3C, a common relationship between modules
and profiles is to build profiles by assembling modules in different
collections. This approach is used in both XHTML and SMIL 2.0, and in fact is
the driving motivation behind their modularization. In both, it is required
that modules be atomic when included in profile definitions -- profiles
cannot subset modules. Conversely, there are examples where profiles subset
(some) modules, and are defined by a collection of modules and pieces of
modules (e.g., the Tiny and Basic profiles of the SVG Mobile CR subdivide
some of the modules of SVG 1.1).
Atomicity of modules within profiles represents a clean design, and a
reflects that the modularization has been well tailored to the goal of
building profiles from modules. This is a recommended design constraint for
specifications that modularize for the purpose of buidling profiles. The
rules for profiles in XHTML and SMIL in fact require atomicity of modules, if
the resulting profiles are to qualify for classifications such as "Host
Language Conformant" or "Integration Set Conformant". See Guideline 3 for full discussion of
profiles.
Modules and levels. Levels are specifically designed to represent
hierarchical steps, for example DOM level 1, DOM level 2, and DOM level 3.
See Guideline 7 for full
discussion of levels. Levels in W3C specifications most commonly reflect
progressive historical development of the specification, rather than
all-at-once design and standardization. In current W3C practice, nested
profiles have often been used in cases of one-time definition of functional
levels within a specification. The nested Tiny, Basic, Full profiles of SVG
Mobile are an example that could, in principle, have been defined as three
levels.
The relationship of modules and levels is less obvious than the
relationship of modules and profiles. In principle, at least, two
relationships can be envisioned:
- In principle, levels could divide modules. For example, level 1 and
level 2 could be defined as different resolution/precision levels within
audio and video modules. Note that this modules-levels relationship would
likely result from all-at-once design of the levels at the same time as
the modules. I.e., it would not likely happen with levels that result
from historical enrichment. In such cases -- e.g., see XHTML's discussion
of evolution of modules -- an "enriched" module is given a different name
and effectively designated as a new, different module. There are no clear
examples of this potential relationship yet in W3C specifications.
- In principle, levels could contain modules. This could result from
historical evolution of a modularized standard -- during level-2
standardizations, some new modules are defined, which when added to the
modules of level 1 would define level 2.
[@@Issue about goodness criteria, from 7/24
telecon. I.e., levels could divide modules or whole modules could be
grouped into levels during progressive (historical) development of the
standard. Is one good and the other bad? Is either one to be encouraged or
discouraged?]
Exercise caution - modules represent one of the dimensions of variability. Excessive
variability fragments and confuses the marketplace, which inhibits
interoperability. Furthermore, dividing the conformance policy of a single
specification in two or more ways creates complexity -- complexity generally
retards interoperability, while simplicity generally facilitates it.
Checkpoints
Checkpoint 4.1. Choose whether or not to have
modules.
[Priority 1]
It is possible for a specification to have profiles, modules, levels, or
none of these. If modules are defined and supported in the specification,
explicitly document it. Furthermore, the specification must explain why
modules are necessary, by reference to use cases and/or project requirements.
Otherwise, explicitly state that modules are not supported.
Checkpoint 4.2. Include a table of contents entry.
[Priority 1]
The reader should be able to easily identify and locate modules
information. A link from the table of contents (TOC) provides this ability.
The form of TOC linkage can vary, depending on the style of the specification
-- some have only a high-level TOC, some have a detailed TOCs, and some have
both. This checkpoint could be satisfied by TOC links to specific
modules-related text from a detailed TOC. For a high-level TOC, it could be
satisfied by a link to a conformance clause section that either contains or
points to modules information. It could be satisfied by both of the two
previous when both types of TOC are present.
Checkpoint 4.3. If modules are chosen, indicate any
mandatory conditions or constraints on their usage.
[Priority 1]
If there are additional conditions or constraints associated with modules,
they must also be described. Such conditions include,
- any mandatory module(s);
- any modules that are optional;
- dependencies among modules:
- modules that require and build on functionally related modules;
- modules that require modules from other functional areas;
- constraints against combined occurrence of particular pairs of
modules;
- any other -- there may be conditions and constraints other than just
enumerated, and they must be described.
The conditions or constraints normally will be tailored according to class
of product.
This checkpoint is not applicable if modules are not chosen.
Checkpoint 4.4. If modules are chosen, define their
relationships and interaction with other dimensions of variability.
[Priority 2]
Often there is dependency or interrelationship among modules, on the one
hand, and profiles or discretionary choices on the other. Modules may have
levels (although no such examples are known amongst W3C standards), or
deprecated features. Extensions could be defined based on modules. [@@Issue
about goodness criteria, from 7/24
telecon. I.e., levels could divide modules or whole modules could be
grouped into levels during progressive (historical) development of the
standard. Is one good and the other bad? Is either one to be discouraged?]
This checkpoint is not applicable if modules are not chosen.
Guideline 5. Specify conformance policy.
A look at various W3C Technical Reports shows that the term "conformance"
is often qualified, resulting in more than one type of conformance. It is
important to convey an understanding of what is meant by conformance and
where the products of a class are allowed to have more or less functionality
than defined in the specification. If the specification defines behavior for
more than one class of product, there may be a separate conformance policy
for each class. Often, the specification will allow discretionary choices,
such as the choice of one or more natural languages for verbal content and
messages, but require a conforming product to make a choice only within the
allowable range. (See Guideline 8 for more discussion.)
Sometimes a product developer can choose to implement certain modules.
There may be per-module conformance requirements that apply if and only if
the developer chooses to implement a particular module.
Where all products of a class must be substantially alike, it should be
clear that a "strict conformance" policy is in effect for that product class.
Strict conformance is defined as conformance of an implementation that
employs only the requirements and/or functionality defined in the
specification and no more (i.e., no extensions to the specification are
implemented).
Sometimes a product developer can choose to implement extensions. There
may be conformance requirements for non-interference of extensions. (See
Guideline 9 for more discussion.)
Overall, the intent of the WG should be clear. In particular, a reader
intending to implement a product in one of the product classes addressed by
the specification should know what the WG wants for interoperability among
products in the class. The developer should understand what forms, if any, of
"product differentiation" are allowed among conformant products. The
specification may need to explain how the rules apply and possibly provide
examples.
Guideline 10, "conformance clause" is related to this guideline. This
Guideline 5 focuses on the establishment and scope of definition of a
conformance policy, while Guideline 10 focuses on where and how to document
it. That is, the verification of these checkpoints will require looking at
the Conformance Clause.
[@@Stern words about excessive variability? 6/27 telcon said "yes" for
GL.3 - GL.8. Was that a hasty "yes"? If not, what does "beware excessive
variability" mean in the context of "Specify conformance policy". (E.g.,
"strict is invariably better for interoperability"?) ]
Checkpoint 5.1. Make it clear where there are universal requirements
for minimum functionality.
[Priority 1]
The reader should be able to recognize a minimum that applies to all
conforming products of each class. If levels are used (see Guideline 7), the
lowest level represents the minimum set of requirements. If profiles are used
there may be different minima for each profile. If modules are used, there
will normally be different minima for each module. Furthermore, a module may
itself be a minimum (i.e., required) for a particular class of product.
Checkpoint 5.2. Make it clear when conformance requirements are
strict.
[Priority 1]
The reader should be able to recognize when a policy of "strict
conformance" applies. As defined above, this implies that all conformant
products of a class behave the same way. If profiles are used, each profile
may have its own conformance boundaries. If modules are used, each module may
have its own conformance boundaries.
Either use the definition provided above (@@link) (or in the QA
Glossary
[QA-GLOSSARY]), modify it, or
provide your own definition (@@really!?). It is strongly recommended that the
definition given above be used to ensure consistency across WGs and promote a
common understanding of conformance requirements.
Checkpoint 5.3. Make it clear where requirements stop and
product-specific extra features begin.
[Priority 1]
The reader should be able to recognize conformance requirements as
distinct from allowable extra functionality. If profiles are used (see
Guideline 3), make it clear whether extra capabilities of the platform may be
exploited. If modules are used (see Guideline 4), there may be
different provisions for extra features applying to each. If deprecation
applies (see Guideline 6), make it clear whether support of the obsolete
features is optional, part of a level, or required. If levels are used (see
Guideline 7), make it clear whether the highest level may be exceeded with
additional features or functionality. If discretionary choices are allowed
(see Guideline 8), make it clear if more
than one may be implemented, when it is technically possible to do so.
Checkpoint 5.4. If
special conformance terms are used, include a definition in the
specification.
[Priority 1]
It is necessary to define terms that govern application of the conformance
provisions. Ideally, all terms are from QA documents and other existing
literature and need only be cited. If special terms are constructed, such as
to combine modules and levels or modules and discretionary choices, these
terms should be defined in the specification. The conformance clause should
reference all locally-defined terms that affect conformance.
Guideline 6. Clarify the
relation between deprecated features and conformance.
After the initial publication of a specification, specification developers
may consider the deprecation of a feature (e.g., function argument, element
or attribute) defined in the specification. A deprecated feature is a feature
whose use is discouraged because it has been outdated by newer constructs or
is no longer viable. Deprecated features may become obsolete and no longer
defined in future versions of the specification. Deprecation of a feature may
warn implementers that the feature was a bad idea and it may be withdrawn in
the future. Specification developers need to consider the effect of
deprecation on all the classes of products that implement the specification
(e.g., authoring tools, user agents) as well as the conformance consequences
on each class of product. For the purpose of backward compatibility, it may
be necessary to specify different requirements for the support of deprecated
features for each class of product. For example, authoring tools (producers)
would not use the feature, but user agents (consumers) would continue to
support it.
Checkpoint 6.1. Identify and clearly indicate each deprecated
feature.
[Priority 1]
Any editions or versions of a specification after the first edition or
version must
- identify and clearly indicate each deprecated feature.
Checkpoint 6.2. For each class of product, specify the
level of support required for each deprecated feature and the conformance
consequences of the deprecation.
[Priority 1]
Define what it means for a feature to be deprecated and how this affects
conformance. For example, a deprecated-features
section of MathML 2.0 ([MATHML20], section
7.2.1.2) describes, about deprecated MathML 1.x features, that
MathML-output-compliant authoring tools may not generate MathML markup
containing deprecated features; whereas MathML-input-compliant
rendering/reading tools must support deprecated features.
Checkpoint 6.3. Include an explanation for the
deprecation.
[Priority 3]
Providing the rationale for deprecating a feature helps implementers and
users to understand the motivation for the deprecation, the impact and
consequences on current and future implementations, and perhaps insight into
its eventual disappearance from the specification.
Checkpoint 6.4. Include examples to illustrate how to avoid
using deprecated features.
[Priority 3]
Examples are helpful in providing alternatives or better ways to get the
same results. By showing what can be done in place of the deprecated feature
will help to get implementers to discontinue use of the deprecated
feature.
Checkpoint 6.5. Include a table of contents entry.
[Priority 2]
The reader must be able to easily identify and locate information about
deprecated features. A link from the table of contents (TOC) provides this
ability. The form of TOC linkage can vary, depending on the style of the
specification -- some have only a high-level TOC, some have a detailed TOC,
and some have both. This checkpoint could be satisfied by TOC links to
specific deprecation-related text from a detailed TOC. For a high-level TOC,
it could be satisfied by a link to a conformance clause section that either
contains or points to deprecation information. It could be satisfied by both
of the two previous when both types of TOC are present. In the case that
there are numerous deprecated features, consider a link to a table or list,
which links to the individual locations in the text.
Guideline 7. Address the use of levels to divide the
specification.
[@@tech-spec: sub "technology" for "specification" in ckpt statement and
appropriate points in the next paragraph? E.g., "Address the use of levels
to divide the technology". And, "Dividing the technology into functional
groupings ...tbd..."@@]
Dividing a specification into functional groupings can facilitate its
implementation by enabling a developer to build to portions of the
specification rather than implementing the entire specification. Some of the
dimensions of variability, but not all, can be used to divide the
specification: product classes, profiles, modules, and (to some extent)
levels divide the specification, while extensibility separates specified
behavior from other product features.
Levels are used to group functionality into nested subsets, ranging from
minimal or core functionality to full or complete functionally. Level 1 is
the minimum or core of the specification. Level 2 includes all of level 1 and
also additional functionality. This nesting continues until level n, which
consists of the entire specification. Levels may result from progressive
historical development and enrichment of the specification, as in the case of
CSS and DOM.
[@@issue: Scope of minimality of levels. Levels represent the minimal or
core requirement within the levels DoV. Do they also represent a minimal or
core requirement in the 3-DoV subspace spanned by the profiles-modules-levels
DoVs?]
Levels may also be defined explicitly in a single edition of the
specification, including in the first edition. No examples of this are found
in W3C specifications. Rather, it is more common in current W3C practice to
use profiles to accomplish this. For example, SVG Mobile defines three nested
profiles -- Tiny, Basic, Full -- which are each targetted at specific
graphics hardware communities (mobile phone, hand-held, desktop).
See Guideline 3 for full
discussion of profiles, including comments on possible profiles-levels
relationships. See Guideline 4
for a full discussion of modules, including possible modules-levels
relationships.
Exercise caution - levels represent one of the dimensions of variability. Excessive
variability fragments and confuses the marketplace, which inhibits
interoperability. Furthermore, dividing the conformance policy of a single
specification in two or more ways creates complexity -- complexity generally
retards interoperability, while simplicity generally facilitates it.
Checkpoints
Checkpoint 7.1. Address whether or not the specification will
use levels.
[Priority 1]
It is possible for a specification to have profiles, modules, levels, or
none of these. If levels are defined and supported in the specification,
explicitly say so. Furthermore, the specification must explain why levels are
necessary, by reference to use cases and/or project requirements. Otherwise,
explicitly state that levels are not supported.
A specification may come to have levels in a number of ways:
- levels may be defined explicitly in a single (first) edition of the
specification, as a way to divide the specification (e.g., for
conformance policy purposes). There are no W3C examples of this.
- or, levels may be planned in the initial edition of the specification
(which defines level 1), as a mechanism associated with anticipated
future, richer editions of the specification. DOM is an example.
- or, levels may be unanticipated initially, but may be decided later as
a means to handle evolving, richer requirements and future editions. CSS
is example?
In all cases, to satisfy this checkpoint, a specification must address the
use of levels.
Checkpoint 7.2. Include a table of contents entry.
[Priority 1]
The reader must be able to easily identify and locate levels information.
A link from the table of contents (TOC) provides this ability. The form of
TOC linkage can vary, depending on the style of the specification -- some
have only a high-level TOC, some have a detailed TOC, and some have both.
This checkpoint could be satisfied by TOC links to specific levels-related
text from a detailed TOC. For a high-level TOC, it could be satisfied by a
link to a conformance clause section that either contains or points to levels
information. It could be satisfied by both of the two previous when both
types of TOC are present.
Checkpoint 7.3. If levels are used, define their relationships
and interaction with other dimensions of variability.
[Priority 2]
Levels can be dependent on, or apply to, modules. Less often, there can be
a relationship between levels, on the one hand, and profiles or deprecated
features on the other.
This checkpoint is not applicable if levels are not used.
Guideline 8. Define discretionary items.
Discretionary items are often made available in specifications, to give
implementers of the technology the opportunity to decide from alternatives
when building applications and tools. Discretionary items are defined as
deliberate and explicit grants of discretion by the specification, to the
implementations, that describe or allow optionality of behavior,
functionality, parameter values, error handling, etc. Discretionary items may
be considered necessary because of environmental conditions (e.g., hardware
limitations or software configuration, or external systems), locality (e.g.,
time zone or language), optional choices providing flexibility of
implementation, dependence on other specifications, etc.
(@@new)Discretionary items come in three basic variants:
- discretionary choices -- a value or behavior may be chosen from a
well-defined enumerated set of two or more possibilities;
- optional features -- a well-defined feature may be supported or not (if
supported, then the requirements are clear and unambiguous)
- implementation dependent values (or features) -- it is open ended and
undefined, what set of values an element or attribute may have, or the
behaviors of a product that implements a feature, etc
One type of discretionary item is implementation dependent values.
Implementation dependent values are used when it is not possible or not
desired to define the behavior or values of a function. Implementation
dependent means that an implementation may determine the effect, rather than
having the effect mandated by the specification. Details in a specification
may deliberately be omitted (i.e., not specified), so as to provide freedom
to adapt implementations to different environments and different
requirements.
Exercise caution - discretionary behaviors represent one of the dimensions of variability. Excessive
variability fragments and confuses the marketplace, which inhibits
interoperability. Furthermore, dividing the conformance policy of a single
specification in two or more ways creates complexity -- complexity generally
retards interoperability, while simplicity generally facilitates it.
Implementation-dependent values is a form of discretionary behaviors which
can have particularly bad interoperability impacts -- it is not a recommended
practice.
(@@[LR] another checkpoint? -- "if impl dependent is allowed, document how
it affects interoperability of implementations.)
Checkpoints
Checkpoint 8.1. Explicitly state the cases and conditions
where discretionary choices are allowed.
[Priority 2]
Although there may be many individual discretionary choices sprinkled
throughout the specification, they often arise from broader ideas about how
the specified software should operate. To help readers, implementers and
testers to find these discretionary items, it is recommended that where ever
possible these items be collected and presented as a group. For example, in
XSLT [XSLT10] most discretionary items offer two choices,
representing design philosophies to escalate an error or to continue
processing. Thus, XSLT could
address discretion as a choice between two groups of items.
Furthermore, the specification must explain why discretionary choices are
necessary, by reference to its use cases and/or project requirements.
Checkpoint 8.2. Indicate implementation dependencies and address
allowable differences between implementations, where applicable.
[Priority 1]
Examples of allowable differences to be addressed include:
- implementation dependent ranges, data, minimum or maximum values,
- environmental resources (e.g., memory or disk limitations),
- environmental values (i.e., language and local settings).
Checkpoint 8.3. Describe alternative approaches and the
conditions under which an implementation is considered to be
conforming.
[Priority 1]
Specifications may describe several different ways to accomplish its
operation (e.g., a choice of file formats, protocols, or encodings). In such
a case, enumerate the approaches and specify if there are limitations on the
number of approaches or combination of approaches that can be implemented.
Some possible ways to define conformance when allowing alternative approaches
include mandating that an implementation:
- implement only one approach,
- implement every approach,
- be allowed to implement none of the approaches.
Note that if the specification does not include the different approaches,
this becomes an implementation detail.
Checkpoint 8.4. Include a statement regarding consistent
handling of a discretionary item within an implementation.
[Priority 2]
The effect of each individual discretionary item should be consistent
within a single implementation. For example, a browser's rendering of a
XSL-FO (XSL Formatting Object) should be the same for every invocation
regardless of the document instance.
Checkpoint 8.5. Include a table of contents entry.
[Priority 2]
The reader must be able to easily identify and locate information about
discretionary items. A link from the table of contents (TOC) provides this
ability. The form of TOC linkage can vary, depending on the style of the
specification -- some have only a high-level TOC, some have a detailed TOC,
and some have both. This checkpoint could be satisfied by TOC links to
specific discretionary items text from a detailed TOC. For a high-level TOC,
it could be satisfied by a link to a conformance clause section that either
contains or points to discretionary-itemsinformation. It could be satisfied
by both of the two previous when both types of TOC are present. In the case
that there are numerous discretionary features, consider a link to a table or
list, which links to the individual locations in the text.
Guideline 9. Allow
extensions or NOT!
An extension to a specification is a mechanism to incorporate
functionality beyond what is defined in the specification. Allowing
extensions affects how conformance is defined as well as what conformance
claims can be made. Exercise caution in determining the extent to which
extensions are allowed or not allowed. Since extensions can seriously
compromise interoperability, specification writers should carefully consider
whether extensions should be allowed.
Extensions may be private (often vendor specific) or may be public (a full
description of the extension is public). Private extensions are usually truly
private, i.e., valid for a specific implementation or are only known by prior
agreement between implementations. Public extensions are extensions in which
the syntax, semantics, identifiers, etc. are defined and published allowing
anyone to implement the extended functionality.
Specifications allow extensions for various reasons. Extensions allow
implementers to include features that they consider to be required by their
customers. Also, extensions, often define new features that may migrate into
future versions of the specifications. However, the use of extensions can
have a severe negative impact on interoperability. Some methods for enabling
extension have less impact on interoperability than other methods. For
example, a specification that allows private extensions (e.g., proprietary)
is highly likely to impede interoperability, whereas a specification than
permits only registered extensions partially mitigates the negative
impacts.
Checkpoints
Checkpoint 9.1. If extensions are disallowed, explicitly
state it.
[Priority 3]
If extensions are not allowed, then make sure it is clear to the reader
that not only are extensions not allowed, but the circumstances under which
they are not allowed. The implementations of the specification precisely
implement the specification. This is strict conformance. Strict conformance
is often imposed on applications or content (e.g., a software program or
document instance). This prohibition of extensions could be applied to a
specific profile or module, rather than to the entire specification.
Checkpoint 9.2. If extensions are allowed, explicitly state
it.
[Priority 1]
State the conditions under which extensions are allowed, the applicability
of the extensions, their effect on conformance claims, their anticipated
effect on interoperability, and any limitations or restrictions on the use of
the extension. Because of potentially serious negative impacts on
interoperability, and to help product developers understand the motivation
for their inclusion and their intended use, the specification must also
explain the rationale for allowing extensions, by reference to the use cases
and/or project requirements.
Checkpoint 9.3. If extensions are allowed, make it clear that
the extensions do not negate support for required functionality.
[Priority 1]
An extension does not change the fact that an implementation needs to
support all required functionality in the specifications exactly as
specified; nor does it cause the non-conformance of functionality defined in
the specification. The specification can include statements such as:
- Each implementation shall fully support all required functionality of
the specification exactly as specified.
- Extensions shall not re-define semantics for existing functions.
- The use of extensions shall not contradict nor cause the
non-conformance of functionality defined in the specification.
Checkpoint 9.4. If extensions are allowed, use a standard
mechanism to define the extension.
[Priority 3]
One mechanism to allow extensions within a specification is to provide a
standard way of defining the extension or a "standard way of being
non-standard". This helps to ensure predictable handling of extensions, that
is, its recognition as such and the appropriate actions (i.e., to ignore or
to implement). The nature of the extension dictates the method for defining
the extension. It may be possible to define a generic function or mechanism
that indicates external functionality. This external function may take the
form of an escape or control character or may be an identifier, the presence
of indicates that an extension follows. Another method, especially when
extending a list of numeric parameters is to use a scheme where positive
values represent standardized values and negative values are reserved for
private extensions.
Checkpoint 9.5. If extensions are allowed, register or publish
them.
[Priority 3]
Registration is a procedure that allows extensions to be acknowledged and
made available to the public. Registration provides for a degree of rigor and
technical review for any proposed extension. Typically the WG would be
responsible for processing the registration of an extension, thus ensuring
adequate quality of a proposed extension and a technical description
sufficient to be uniformly implementable. Often registered extensions may
migrate into a later version of the specification.
(@@Ed note. This was raised as an issue in QAWG
project review, but we never got to it on the agenda.)
Checkpoint 9.6. If extensions are allowed, require that
implementations include a way to operate without the extension.
[Priority 3]
If an implementation contains extensions, require it to have a mode under
which the implementation can be directed to produce only conforming files
(documents) or to operate in a strictly conforming manner.
Checkpoint 9.7. Include a table of contents entry.
[Priority 2]
The reader must be able to easily identify and locate information about
extensions and extensibility. A link from the table of contents (TOC)
provides this ability. The form of TOC linkage can vary, depending on the
style of the specification -- some have only a high-level TOC, some have a
detailed TOC, and some have both. This checkpoint could be satisfied by TOC
links to specific extensibility section(s) from a detailed TOC. For a
high-level TOC, it could be satisfied by a link to a conformance clause
section that either contains or points to extensibility information. It could
be satisfied by both of the two previous when both types of TOC are
present.
Guideline 10. Provide a conformance clause.
A conformance clause is a part or collection of parts of a specification
that defines the requirements, criteria, or conditions to be satisfied by an
implementation or application in order to claim conformance. Typically the
conformance clause is a high-level description of what is required of
implementations and applications.
Guideline 5, "conformance policy" is related to this guideline. Guideline
5 focuses on the establishment and scope of definition of a conformance
policy, while this Guideline 10 focuses (among other topics) on how and where
to document it.
Checkpoint 10.1. Include a conformance clause.
[Priority 1]
Every specification must document its conformance policy and specific
conformance requirements somewhere in the specification. As used in this
checkpoint, "clause" does not necessarily imply a specific single document
section or location (see next checkpoint).
Checkpoint 10.2. Create a separate conformance section.
[Priority 2]
Having the conformance clause exist as a separate section within the
specification makes it clearly identifiable, allowing a reader to find the
overall conformance policy, as well as all specific conformance provisions
from a single starting point.
Checkpoint 10.3. Include a
conformance clause entry in the table of contents.
[Priority 2]
It should be possible for the reader to start at the table of contents and
find all aspects of the conformance requirements, including the overall
policy.
Checkpoint 10.4. Make normative reference to specifications on which
the current specification depends. [Priority 2]
Often a specification is dependent on other specifications or portions of
specifications. For example, SVG 1.0 requires that the class of product
called "user agent" be consistent with the
XML 1.0 Recommendation
[XML10
] and (conditionally) support
Cascading Style Sheets, level
2
[CSS2]. Dependence on other specifications
affects the conformance boundaries of the current specification, and thereby
affects the requirements on conformant products. To ensure clarity and
understanding of these implications, describe the relationship between the
specifications and any conformance implications. Linking from the prose to
the reference is described in the linking parts of the
Manual of Style ([STYLE-MAN], section 11.5.1).
(@@yes? do we intend to make this a requirement of satisfying the checkpoint?
Or recommended? Or?)
Guideline 11. Specify how to make conformance
claims.
A specification may differentiate conformance claims by designating
different degrees or types of conformance in order to apply and group
requirements according to modules, profiles, levels, or priorities. When a
conformance claim is linked to functionality, impact and/or incremental
degrees of implementation, the term 'conformance level' is often used to
indicate the varying degrees of conformance. The WG includes in the specification the way they
want people to claim their conformance.
Checkpoints
Checkpoint 11.1. Identify and define all conformance
designations.
[Priority 1]
[@@new]In current W3C practice, a number of different naming converntions
are used to label conformance, when there is other than a single, monolithic
conformance definition. The naming convention used to label the conformance
can provide useful information. Degrees, for example, implies incremental
importance or difficulty. This Specification Guidelines document uses
"degrees" for example, to refer to three successively more demanding degrees
of conformance (A, AA, AAA).
Commonly used conformance designations include categories, degrees, and
levels. Use of "conformance levels" is discouraged in new specifications,
because of the potential for confusion with "functional levels".
Each specification must
- clearly define the conformance naming convention, and identify and
define all conformance designations.
Functional (technology) specifications commonly use groupings of
functionality to distinguish amongst conformance designations. For example,
SVG Mobile defines three
conformance categories as three nested profiles -- Tiny, Basic, and Full --
based on the modularization
description of SVG
1.1 (Scalable Vector Graphics 1.1, [SVG11], section 1.1.2). Each of the profiles is equivalent
to a functional level that subsumes the next lower level (if any), providing
additional functionality and/or complexity.
The conformance designations of the guidelines documents of WAI and the QA
Framework, by comparison, reflect the a target object's satisfaction of
successively larger sets of prioritized checkpoints. A basic conformance
designation (A) is associated with the Priority 1 checkpoints. The other
designations, for AA (double-A) and AAA (triple-A), are associated with the
addition respectively of all Priority 2 and all Priority 3 checkpoints.
Examples: Web Content Accessibility Guidelines 1.0 [WCAG10]), in which the ; and, QA Framework:
Specification Guidelines (this document, see Conformance section.)
Checkpoint 11.2. Provide specific wording of the claim.
[Priority 3]
A well-formed conformance claim includes: date of the claim, specification
name, date and version, URI of the specification, conformance level
satisfied, and information about the subject (that which is claiming
conformance). Information about the subject refers to information such as,
the name of the software or software component, version information, and
operating environment.
Checkpoint 11.3. Provide a conformance disclaimer.
[Priority 3]
Although it is possible to prove with certainty when something does not
conform, the reverse is not necessarily true. Especially for functional
specifications, where a claim goes beyond syntax testing, a claim of
conformance is not a guaruntee that the claimant is 100% conforming with the
specification. A disclaimer can help clarify the meaning of a conformance
claim as well as its limitations. For example, this document contains a
conformance disclaimer.
Checkpoint 11.4. Impose no restrictions about who can make a
claim or where claims can be published.
[Priority 1]
Claimants (or relevant assuring parties) are solely responsible for the
validity of their claims, keeping claims up to date, and proper use of the
conformance icons. As of the publication of this document, W3C does not act
as an assuring party, but it may do so in the future, or it may establish
recommendations for assuring parties. Claimants are expected to modify or
retract a claim if it may be demonstrated that the claim is not valid.
Checkpoint 11.5. Include a
table of contents entry.
[Priority 2]
The reader should be able to easily identify and locate the information on
how to make conformance claims. A link from the table of contents (TOC)
provides this ability. (@@new) The form of TOC linkage can vary, depending on
the style of the specification -- some have only a high-level TOC, some have
a detailed TOC, and some have both. This checkpoint could be satisfied by TOC
links to specific conformance-claims text from a detailed TOC. For a
high-level TOC, it could be satisfied by a link to a conformance clause
section that either contains or points to conformance-claims information. It
could be satisfied by both of the two previous when both types of TOC are
present.
Guideline 12. Publish an Implementation
Conformance Statement proforma.
An Implementation Conformance Statement (ICS) or questionnaire is useful
in clarifying and declaring optional functionality and discretionary behavior
and values. The results of the ICS can be used to identify the subset of test
cases from a conformance test suite that are applicable to the implementation
to be tested. This will allow the implementation to be tested for conformance
against only the relevant requirements.
The basic and detailed information that an ICS provides can also be used
to assess and deduce the interoperability potential of two or more
products.
Checkpoints
Checkpoint 12.1. Include
an Implementation Conformance Statement proforma as part of the
specification.
[Priority 3]
If an ICS is included as part of the specification, indicate whether it is
a normative or informative part of the specification.
Checkpoint 12.2. Require
the ICS be completed as part of the conformance claim.
[Priority 3]
An ICS provides specific information about the implementation and can be
helpful in substantiating the conformance claim.
Guideline 13. Support general document conformance
conventions.
There is a lot to be said about consistency and clarity within a document
- it leads to the understanding and comprehensiveness of the document.
Authors and editors of specifications should already be familiar with the
W3C Manual of Style [STYLE-MAN] and
Publication Rules [PUBRULES
],
which help to achieve this. With respect to conformance, it is important to
provide clear and unambiguous statements, so that the reader knows what is
required in order to claim conformance and what is optional. To achieve this
objective, throughout the document, employ uniformity of structure and style
and consistency of terminology and phraseology.
Checkpoints
Checkpoint 13.1. Use
conformance key words.
[Priority 1]
Use RFC 2119 key words to denote whether or not requirements are
mandatory, optional, or suggested. Using these keywords helps to identify the
testable statements in a specification.
Checkpoint 13.2. Distinguish normative and informative
text.
[Priority 2]
Normative statements are the prescriptive parts of the specification
whereas informative statements are for informational purposes and assist in
the understanding or use of the specification. It is important that the
reader be able to distinguish between normative and informative statements in
order to know what is required to claim conformance and what is optional.
SMIL 2.0 is a good example, indicating within every subsection whether it is
normative or informative, and even separately labelling pieces of subsections
that contain both kinds of content.
Checkpoint 13.3. Follow Web
Accessibility Initiative and Internationalization Guidelines.
[Priority 1]
Applying the principles of the Web Content Accessibility Guidelines
1.0 [WCAG10] and Internationalization Guidelines
(@@reference? what do you mean to point to here?) not only ensures
accessibility and internationalization, but also contributes to the
principles of conformance - clear, unambiguous, testable statements. For
example, use simpler words to express your ideas, markup text with structural
elements, add markup to distinguish common words from keywords.
Checkpoint 13.4. Use
the same words to express the same ideas.
[Priority 1]
Use identical wording to express identical provisions and analogous
wording to express analogous provisions.
Guideline 14. Use granular grammars to author the
specification.
Using a W3C endorsed grammar language (DTD or XML Schema) provides
control over the information conveyed in the specification. This allows for
and facilitates automatic generation of test materials, more detailed
reporting of test results as well as specification coverage.
Examples include the XML and DOM Working Groups, that both use the
xmlspec
DTD
.
Using a granular grammar to author a specification draws further on
separation of content and presentation and simplifies the task of generating
readable versions of the specification, maintaining the possibility to
generate test materials from the granular version of the specification. In
addition, using a grammar to author the specification makes it easier to
logically group the constituents of the specification, thus adding control
over interpretation and implementation.
Using a grammar greatly simplifies controlling conformance with W3C
publication rules and guidelines/checkpoints. Also, having written the
specification using a grammar, automated validation with regard to these
aspects is simplified.
Finally, using an advanced schema for specification authoring allows for
greater control over coverage of the specification in the Test Suite.
Checkpoint 14.1. Use W3C endorsed grammar where applicable.
[Priority 1]
One example is the DTD noted above. Other
examples will be given.
Checkpoint 14.2. Specify intended behavior in the specification
using markup.
[Priority 1]
There are different types of technologies that are specified, and this
checkpoint applies in different ways to those. For API specifications, such
as DOM, the intended behavior is the algorithm output; the specification
contains wording on what the expected result of applying a particular method
on an interface is. On user-centric specifications, such as WAI guidelines
specifications, the intended behavior is not so much a question of testing
algorithms, but rather how user agents should behave given different input or
how markup should be designed. In this case, intended behavior is more a
question of parsing documents. On processor specifications, such as XML, the
intended behavior is, for example, preserving or expanding entity references,
and the intended behavior is a particular state after processing a
document.
Checkpoint 14.3. Supply prose description of intended behavior
together with each test assertion.
[Priority 1]
Following the guideline below, group test assertions and statements of
intended behavior in the specification, or group the pointer to the test
assertions together with the statement of intended behavior in the
specification (in case the test assertion is, for example, given in an
appendix).
Guideline 15. Include test assertions.
Some specifications include test assertions as part of the specification,
and more should. A test assertion is a statement of behavior, action or
condition that can be measured or tested. It is derived from the
specification's requirements and bridges the gap between the narrative of the
specification and the test cases. Each test assertion is an independent,
complete, testable statement for requirements in the specification. Each test
assertion results in one or more test cases. Multiple test assertions can be
combined to form a test case, in this case one tests multiple facets of a
particular behaviour. Including test assertions as part of the specification
facilitates and promotes the development of test materials. Tests can point
directly to the test assertion in the specification. Specific benefits
include:
- Test assertions (that are part of a specification) are publicly
reviewed and agreed to;
- Detailed control over what parts of a specification an implementation
supports, grouped by behaviour, technical aspect (method/interface) or
other conceptual grouping;
- Detailed reporting when testing an implementation using the test;
Checkpoints
Checkpoint 15.1. Supply test assertions in the markup of the
specification, if applicable using a set of predefined tags used in the
specification markup language.
[Priority 1]
In order to enable pointing to test assertions from tests as well as to
give a map of the specification from the point of view of tests, use a
mechanism for making explicit test assertions in the specification (@@markup
particulars are being investigated as of 2002-07-05; ED)
Checkpoint 15.2. Tag
test assertions according to the above.
[Priority 1]
Where applicable, use existing markup for making explicit the test
assertions for the specification. If possible, use markup provided in
(@@[SpecGL ExTech. Link needed]), otherwise construct test assertion markup
acording to (@@[SpecGL ExTech and/or Test GL (ExTech). Link needed]).
This section defines conformance of Working Group specifications -- i.e.,
technical reports -- to the requirements of this QA Framework guidelines
specification. The requirements of this guidelines specification are detailed
in the checkpoints of the preceding "Guidelines" chapter, and apply to the
technical reports produced by Working Groups.
3.1 Conformance definition
[@@revised] This section defines three degrees of conformance to this
guidelines specification:
-
A-Conforming: all Priority 1 checkpoints are
satisfied;
-
AA-Conforming: all Priority 1 and Priority 2
checkpoints are satisfied;
-
AAA-Conforming: all Priority 1, Priority 2, and
Priority 3 checkpoints are satisfied;
A specification conforms to the QA Framework: Specification
Guidelines at degree X (A,
AA, or AAA) if the Working Group meets at
least all degree X conformance requirements.
To make an assertion about conformance to this document, specify:
- The guidelines title: QA Framework: Specification
Guidelines
- The guidelines URI: http://www.w3.org/TR/qaframe-spec/
- The degree of conformance satisfied: A-Conforming, AA-Conforming, or
AAA-Conforming.
Example:
This specification conforms to W3C's QA Framework: Specification
Guidelines, available at http://www.w3.org/TR/qaframe-spec/,
AA-Conforming.
3.2 Conformance disclaimer
The checkpoints of this guidelines specification present verifiable
conformance requirements about the specifications (technical reports) that
Working Groups produce. As with any verifiable test requirements, it is also
true of these specification requirements that:
- Passing all of the requirements to achieve a given degree of
conformance -- A, AA, or AAA -- does not guarantee that the subject
specification is well-suited to or will achieve its intended purposes,
nor does it guarantee the quality or suitability of test materials
produced from the specification.
- Failing to achieve conformance degree of at least A-conforming does not
mean that the subject specification is necessarily deficient to its
intended purposes, nor does it mean that it is an unacceptable basis for
the development of quality test materials. It means that the
specification has failed one or more checkpoints that best-practice
experience has shown to improve the testability and usability of
specifications, and to facilitate the timely and successful development
and maintenance of quality test materials based on the specification.
- conditional conformance
- conformance clause
- a part or collection of parts of a specification that defines the
requirements, criteria, or conditions to be satisfied by an
implementation or application in order to claim conformance
- conformance level
- (@@deprecated term, due to confusion with level and functional level.
Same basic meaning as conformance category, conformance degree,
conformance
- dimensions of variability
- discretionary items
- deliberate and explicit grants of discretion by the specification, to
the implementations, that describe or allow optionality of behavior,
functionality, parameter values, error handling, etc.
- functional level
- level
- shorthand for functional level.
- profile
- a subset of a specification or set of specifications that is tailored
to meet specific functional requirements of a particular application
community. A profile may address a single specification; or, a profile
can also group a set of standards and define how they operate together.
Profiles may be based on hardware considerations associated with target
product classes, or they may be driven by other functional requirements
of their target communities.
- profiling
- a method for defining subsets of a specification by identifying the
functionality, parameters, options, and/or implementation requirements
necessary to satisfy the requirements of a particular community of
users.
- strict conformance
- conformance of an implementation that employs only the requirements
and/or functionality defined in the specification and no more (i.e., no
extensions to the specification are implemented).
- test assertion
- any sentence or equivalent assemblage of words and phrases that
prescribes the behavior that must be obtained when a stimulus occurs
under a certain set of conditions. (See also QA Glossary [QA-GLOSSARY].)
- unconditional conformance
- use case
- user scenario
The following QA Working Group and Interest Group participants have
contributed significantly to the content of this document:
- Daniel Dardailler (W3C)
- Dimitris Dimitriadis (Ontologicon)
- Karl DuBost (W3C)
- Peter Fawcett (RealNetworks)
- Kirill Gavrylyuk (Microsoft)
- Dominique Hazael-Massieux (W3C)
- Lofton Henderson (CGM Open)
- Ian Jacobs (W3C)
- David Marston (IBM Research)
- Jack Morrison (Sun Microsystems)
- Lynne Rosenthal (NIST)
- Mark Skall (NIST)
- Andrew Thackrah (Open Group)
- Olivier Thereaux (W3C)
-
CSS2
-
Cascading Style Sheets,
Level 2
, W3C Recommendation, 12 May 1998, available at
http://www.w3.org/TR/REC-CSS2.
-
MATHML20
-
Mathematical Markup
Language (MathML) Version 2.0
, W3C Recommendation, 21
February 2001, available at http://www.w3.org/TR/MathML2/.
-
PUBRULES
- W3C Publication
Rules, available at http://www.w3.org/Guide/pubrules.html
(member-only).
-
QA-GLOSSARY
- Comprehensive glossary of QA
terminology. (Under construction, at
http://www.w3.org/QA/Glossary.)
-
QAF-INTRO
-
QA Framework:
Introduction
, L. Henderson, D. Dimitriadis, K. Gavrylyuk, L.
Rosenthal, Eds., W3C Working Draft, May 2002, companion version to this
document, available at [...]
-
QAF-OPS
-
QA Framework: Operational
Guidelines
, K. Gavrylyuk, D. Dimitriadis, L. Henderson, L.
Rosenthal, Eds., W3C Working Draft, May 2002, companion version to this
document, available at [...].
-
QAF-TEST
-
QA Framework: Test Guidelines, not yet published.
-
QAIG
-
Quality Assurance Interest
Group of the W3C QA Activity, which may be found at
http://www.w3.org/QA/IG/.
-
QAWG
-
Quality Assurance Working
Group of the W3C QA Activity, which may be found at
http://www.w3.org/QA/WG/.
-
RFC2119
-
Key words for use
in RFCs to Indicate Requirement Levels
, March 1997,
available at http://www.ietf.org/rfc/rfc2119.txt.
-
SMIL20
-
Synchronized
Multi
media
Integration Language (SMIL 2.0)
, W3C Recommendation, 07
August 2001, available at http://www.w3.org/TR/smil20/.
-
SPEC-CHECKLIST
- An appendix to this specification guidelines document presents all checkpoints in tabular form.
Available at
http://www.w3.org/TR/2002/WD-qaframe-spec-20020515/qaframe-spec-checklist.html
-
SPEC-EXTECH
-
QA Framework: Specification Examples & Techniques,
not yet published.
-
STYLE-MAN
-
W3C Manual of
Style
, summarizing the style and publication rules for W3C
technical reports, available at http://www.w3.org/2001/06/manual/.
-
SVG11
-
Scalable Vector Graphics
(SVG) 1.1 Specification
, W3C (last call) Working Draft, 15
February 2002, available at http://www.w3.org/TR/SVG11/.
-
TEST-EXTECH
- "QA Framework: Test Examples and Techniques", not yet published.
-
UAAG10
-
User Agent Accessibility
Guidelines 1.0
, I. Jacobs, J. Gunderson, E. Hansen, Eds.,
W3C Candidate Recommendation, 12 September 2001, available at
http://www.w3.org/TR/UAAG10/.
-
W3C-TR
- Location of all published W3C
technical reports, see http://www.w3.org/TR/.
-
WCAG10
-
Web Content
Accessibility Guidelines 1.0
, W. Chisholm, I. Jacobs, G.
Vanderheiden, Eds., W3C Recommendation, 5 May 1999, available at
http://www.w3.org/TR/WCAG10/.
-
XHTML-MOD
- ...tbd...
-
XML10
-
Extensible Markup
Language (XML) 1.0 (Second Edition)
, W3C Recommendation, 6
October 2000, available at http://www.w3.org/TR/REC-xml.
-
XML-SCHEMA
-
XML Schema Part 1:
Structures
, W3C Recommendation, 2 May 2001, available at
http://www.w3.org/TR/xmlschema-1/.
-
XPATH10
-
XML Path Language (XPath)
Version 1.0
, W3C Recommendation, 16 November 1999, available
at http://www.w3.org/TR/xpath.
-
XPOINTER
-
XML
Pointer Language (XPointer) Version 1.0
, W3C Candidate
Recommendation, 11 September 2001, available at
http://www.w3.org/TR/2001/CR-xptr-20010911/
-
XSLT10
-
XSL Transformations (XSLT)
Version 1.0
, W3C Recommendation, 16 November 1999, available
at http://www.w3.org/TR/xslt.
- 2002-08-xx version
-
Significantly reorganized and revised the first published WD. This
version produced as a series of editor's drafts. The changes below are
reverse chronological (most recent first), so more recent ones may
build on older ones.
- Add caveat to "Understanding and using..", about specifications
that pre-date SpecGL.
- Rewrite CK11.1 to replace 'levels' in "conformance levels" w/
degrees, categories, etc (confusion with "functional levels").
- Change "levels" to "degrees" in SpecGL conformance clause.
- Change "discretionary behaviors" to "discretionary items" in GL8,
define three variants in GL8 verbiage.
- Extensive new prose in GL3 (profiles), GL4 (modules), GL7
(levels), describing potential relationships.
- Fixed definition of strict conformance, clarified definition and
usage of "conformance clause".
- Added "Definitions" section (in-spec glossary).
- Enhanced wording of all TOC checkpoints.
- For each of CK3.x (profiles), 4.x (modules), 7.x (levels),
x>1, added a "not applicable" disclaimer if the feature is not
used. (Except for the TOC checkpoints.)
- Change wording of CK7.x, eliminating "chosen", and add some new
explanation to 7.1
- Delete CK5.4 (conformance policy in conformance clause) and move
its verbiage to GL10 (conformance clause) and its checkpoints.
- Fix "Conformance" chapter, add "Checkpoint Priorities"
section.
- Change chapter title of "Navigating through this document" to
"Understanding and using this document"
- New text for Ckpt 3.4
- Added warnings about "excessive variability" to all
dimension-of-variability guidelines.
- Added to several DoV checkpoints a requirement to explain/justify
any variability dimension by use cases and requirements.
- Fix separation between last two guidelines, "Granular Grammars"
and "Test Assertions".
- Introduce and motivate "dimensions of variability" (DoV) in sec
1.5
- Reordered guidelines (GL), split GL.5 into three new ones, GL.3,
GL.4, and GL.7, customized checkpoints to their topics (profiles,
modules, levels).
- Added new p3 checkpoint ("categorize specification") to (new)
GL.2
- Rewrote (old) GL.3, "conformance flavors", in terms of "specify
conformance policy, (new) GL.5.
- 2002-05-15 version
-
First published WD.