Abstract
A lot of effort goes into writing a good specification. It takes more than
knowledge of the technology to make a specification precise, implementable
and testable. It takes planning, organization, and foresight about the
technnology and how it will be implemented and used. The goal of this
document is to help W3C editors write better specifications, by making a
specification easier to interpret without ambiguity and clearer as to what is
required in order to conform. It focuses on how to define and specify
conformance for a specification. Additionally, it addresses how a
specification might allow variation among conforming implementations. The
document is presented as a set of guidelines or principles, supplemented with
good practices, examples, and techniques.
Status of This Document
This section describes the status of this document at the time of its
publication. Other documents may supersede this document. A list of current
W3C publications and the latest revision of this technical report can be
found in the W3C technical
reports index at http://www.w3.org/TR/.
This document is a W3C Working Draft (WD), made available by the W3C Quality
Assurance (QA) Activity for discussion by W3C members and other interested
parties. For more information about the QA Activity, please see the QA Activity
statement.
This is the first published version of this document after the decision to
completely redesign the QA Framework (QAF), resolved by the QA Working Group (QAWG) at its 2004 Technical Plenary
face-to-face. It is a lighter-weight, less authoritarian, more
user-friendly and useful version of the previously published Candidate
Recommendation version of the Specification Guidelines. Because
of the extensive nature of the changes, it has been returned to Working
Draft.
This draft accurately reflects the intended overall structure and content
of the new, lightweight Specification Guidelines. However it is
incomplete in some details (generally indicated by "@@" or "TBD"), especially in the examples -- lots more
examples are intended. Also, some sections more resemble outlines than
completed prose. Because this is a significant departure from the
last-published Specification Guidelines, QAWG wants review and comment on the
overall direction as soon as possible. Incomplete details will be fleshed out
in the next publication.
This Specification Guidelines document will be coordinated
with the other parts of the QA Framework, The QA Handbook [QA-HANDBOOK] and
QA Framework: Test Guidelines. Synchronization at uniform level
of maturity will occur no later than Last Call. The first publications of the
various parts will occur somewhat more independently.
The QAWG expects this
specification to become a Recommendation.
Publication as a Working Draft does not imply endorsement by the W3C
Membership. This is a draft document and may be updated, replaced or
obsoleted by other documents at any time. It is inappropriate to cite this
document as other than work in progress.
This document has been produced under the
24 January 2002 CPP as amended by the
W3C Patent Policy
Transition Procedure. An individual who has actual knowledge of
a patent which the individual believes contains Essential Claim(s)
with respect to this specification should disclose the information
in accordance with section 6 of the W3C Patent Policy.
Patent disclosures relevant to this specification may be found on the
Working
Group's patent disclosure page.
Comments on this document may be emailed to www-qa@w3.org, the publicly archived list
of the QA Interest Group
(QAIG).
Commenters please note that your comments will be publicly
archived and available, do not send information that should not be
distributed, such as private data. If you are able to give your feedback by
12 June 2004, the QAWG can
consider it at its next face-to-face meeting. The comments will still be collected after the F2F in June 2004.
Table of contents
Introduction
Scope and Goals
This document is a guide for W3C editors and authors. It provides
guidelines for writing better specifications. This document differs from
other W3C process and publication related documents in that it looks at
specifications from a conformance viewpoint. It address the most basic and
critical topics with respect to conformance, including how to convey what is
required in order to conform and how to allow variation among conforming
impelmentations. The goal is to make specifications precise, easier to
interpret without ambiguity or confusion, and clearer as to what is required
in order to conform. Good specifications lead to better implementations and
foster the develoment of test suites and tools. Conforming implemenations
lead to interoperablity.
Motivation
Everyone benefits from having well-written specifications. Editors may
have less rework and thus, fewer errata. Implementers can implement sooner
and have a better chance to conform to the specification. Test developers are
able to derive unambigious requirements. Users benefit by having
interoperable solutions. W3C gains by having recommendations produced with
higher quality and reduced maintenance.
Why Specification Guidelines?
It is not an easy task to write accurate, clear, complete, unbiased
specifications. It requires planning, organization, and foresight about the
technology, how it will be implemented and used, and how technology decisions
affect conformance. This document provides a collection of principles, good
practices, examples, and techniques that leads the reader though the
decisions that need to be made in order to write precise requirements and
establish, define, and specify conformance for specifications. Realizing that
editors and authors are busy, under pressure to get the specification
published, and already have a reading list of W3C documents (a good place to
start, W3C
Editor's Home Page [EDITOR]
this document can be used as a checklist or reminder of things to consider, a
how-to guide with examples and techniques which can be adapted, a reference
for understanding conformance concepts and terminology, or all of these.
About this document
This document is a practical guide to writing a specification,
presenting editors with topics to consider. Consequently, much of the
document is non-normative recommendations labeled Good
Practices. There are however, a limited number of normative
requirements, labeled Principles, that must be implemented
in order to conform to this document. The overall objective of these
principles and good practices is to facilitate the creation of a conformance
clause in every specification. A conformance
template is provided to assist editors satisfy the requirements of this
document and end up with a conformance clause. Note that for some
specifications (e.g., QAH, WebArch) where conformance is not an issue (e.g.,
no normative content), the conformance clause may be an explanation of why
there is no "conformance to this document" which may not be a separate
section but incorporated into another section.
The topics presented are inclusive (self-contained) providing information
needed to understand and successfuly apply the principle, although related
information and advanced topics may be referenced.
If in a hurry - start at the first section, Specifying confomrance -- this
is the expected outcome of adhering to this document. It serves as a roadmap
to other parts of this document, which help you achieve Secifying
Conformance.
Audience of this
document
The primary audience of this document is editors and authors, however, it
is applicable to a broader audience including:
- those who review specifications during their development
- implementers of specifications
- builders of test materials, including conformance test suites and
tools.
This document makes no distinction between the terms editors and authors
and will refer to them collectively as editors.
Structure of this document
This document is organized in a series of main topics like Conformance or Extensibility. In each of this main topics, a few principles are developed and explained. Theses principles are accompanied by techniques and examples. The techniques illustrate basic (and non exhaustive) questions or methods that might help you create a specification. The examples are explanations or extract of specifications which specifically address the point which has been made in the principle.
explain what is normative and what are the conformance requirements
The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY ", and "OPTIONAL" will be used as defined in RFC 2119 [RFC2119]. When used with the normative RFC2119 meanings, they will be all uppercase. Occurrences of these words in lowercase comprise normal prose usage, with no normative implications.
This document can be used as a stand alone document or as part of a family
of QA Framework documents designed to help the WGs improve all aspects of
their quality practices. The QA Handbook [QA-HANDBOOK] is a non-normative handbook about the process and operational aspects of the quality practices of W3C's Working Groups (WG). @@Add reference to the Test GL@@
Guidelines
Clear presentation of what is meant by conformance to the specification is
ultimately crucial to successful interoperability of implementations. The
conformance policy and language in the specification not only determine its
testability, but also shape the overall look of its implementations landscape
-- are there numerous fragmented flavors or a few strong, interoperating
variations?
Conformance is the fulfillment of a product, process, or
service of specified requirements. These requirements are specified in a
standard or specification as either part of a conformance clause or in the
body of the specification. A conformance clause is a section of a
specification that states all the requirements or criteria that must be
satisfied to claim conformance to the specification.
With a good conformance clause, a specification comes close to full
conformance to this Specification Guidelines. Therefore:
Good Practice: In the conformance clause, define how
normative language is expressed.
What does it mean? There are a number of common language
styles for expressing the detailed conformance requirements of a
specification. One of the most popular is RFC 2119 [RFC 2119] -- keywords like
must, may, and should not
both signal the presence of conformance requirements and express their
relative "manditoriness". Other styles include imperative voice statements
(as in the statement of the Good Practice item), physical segregation via
styling, labelling, etc. What matters is that the style be consistent,
unambiguous, and (ideally) quickly recognized.
Why care? As before, conformance of implementations
is defined by and measured against normative content, which includes
thedetailed conformance requirements.
The central message of this section -- "have a good conformance clause"
--has many ancilliary details. Because the conformance clause is the
foundation for defining and measuring conformance, it is also the basis for
assessing conformance claims. One detail worthy of attention is "valid
conformance claims".
[Ed note. valid? or well-formed? Maybe:well-formed
if it meets the minimal bullet list of contents below.; valid ifit has an ICS
that supports the claim. Or ... avoid the question fornow?]
Rather than live with the infinite varieties of creative conformance
claims that will arise in a vacuum, the specification can be proactive:
Good Practice: Provide an Implementation Conformance
Statement (ICS) proforma.
What does it mean? An Implementation Conformance
Statement (ICS) provides standardized information about the conformance of an
implementation to the specification. It indicates which capabilities and
options have been implemented, as well as the limitations of the
implementation. An ICS typically takes the form of a questionnaire for
implementor to complete.
This Good Practice suggests that the specification itself include an ICS
proforma. (Caveat. The ICS concept may be inapplicable to some types of
specifications.)
Why care? An ICS provides detail about conformance. The
detail can, for example, be used to identify the subset of a conformance test
suite that is applicable to the implementation to be tested. An ICS can also
be especially valuable in optionality choices in the implementation, and
documenting the presence of extensions.
Good Practice: Require an Implementation Conformance
Statement (ICS) as part of valid conformance claims.
What does it mean? This simply puts together the previous
two good practices. Not only could the specification provide an ICS proforma
for implementors, but it could require it to be linked from its standardized
conformance claim template.
@@should we include discussion on - the meaning/value of a conformance claim
may change as the spec and tests evolve
B. The Nature of the Specification
B.1. Scope
The path to a quality specification begins with its scope. It is critical
to convey what the specification is about by describing its intent and
applicability. As the specification develops, its a good idea to revisit the
scope to make sure it still reflects the intent of specification or if it
needs to be modified.
Principle: Define the scope. Describe what the specification is about.
In addition to containing the subject of the specification, a more
complete scope contains:
- applicability of the specification,
- purpose, objectives, and goals,
- types of products or services (i.e., classes of products) to
which the specification applies,
- relationship to other specifications or technologies,
- what is not in scope, i.e., what the specification is not about,
- limitations on the application of the specification.
Note that it is not necessary to write a separate statement for each of
these items, rather, they can be combined.
Why care? It helps to keep the specification content
focused. It also will help readers know the limits or boundaries of the
specification and whether it is of interest to them.
Examples
Examples: of good Scopes, bad scopes?
Good Practice: Be direct, providing simple,
understandable statements of facts. Don't specify requirements
Technique
Use statements such as: This document (@@check other Rec's and use examples from them)
- specifies a method of ...
- specifies the characteristics of ...
- defines ...
- establishes a system for...
- establishes general prinicples for...
- gives guidelines for ...
- is a guide for...
Good Practice: Use examples or use cases to illustrate
Examples
Providing examples, use cases or both helps to clarify the scope for the
reader.
@@Here are different ways to say basically the same thing.
- What is the target of the spec what will implement it
- What will be the object of the conformance claim
Foo defines conformance for: (fill in the rest)
Examples of 'what': protocol, API, content
Advanced Topic: Classes of Product, DoV
B.3 Make a list of normative (and non-normative) references
Good Practice: Start now and keep adding to it as you go.
C.1 Define your terms
Define conformance concepts, designations
Examples: valid, well-formed, foo-conformant, document conformance (CC/PP)
consumer conformance (CC/PP)
Good Practice: being consistent and using the same term when you mean the
same thing don't be a thesaurus.
C.2 Identify normative and informative parts
(Remove or keep - redundant with A1, GP. not much new
to say here)
Distinguish between normative and non-nomative content
C.3 What is mandatory
(Remove or keep - redundant with A1, GP.)
Indicate what requirements are mandatory as opposed to optional,
recommended, good ideas, etc.
Expressions that conveys the criteria to be fulfilled in an implementation
of the specification
Examples: use RFC keywords, assertive voice, gold stars.
Good Practice: being able to find these requirements easily navigation
D. Managing Variability
TBD
Variability affects interoperability
D.1 Subdivide
TBD
Subdivide into related groups of conformance requirements
Examples: modules, levels, profiles
@@ not sure what to do with these questions, they lead
you to doing the right thing.
- What are the division?
- What are the ways conforming implementations differ?
- How does an implementation conform to the subdivision?
- Are there mandatory conditions or constraints
Examples of reasons to divide:
- no one will implement the whole spec easier to implement a subset
- incremental functionality
- target specific a type of device (SVG tiny)
- target specific audience
D.2 Optionality and Options
Options in specifications provide implementations the freedom to make
choices about
- whether or not to support a well-defined feature,
- which value or behavior to choose from a well-defined enumerated set of
possibilities,
- implementation dependent values or features.
These choices also called dscretionary items, give implementers
of the technology the opportunity to decide from alternatives when building
their applications and tools.They describe or allow optionality of behavior,
functionality, parameter values, error handling, etc. They may be considered
necessary because of environmental conditions (e.g., hardware limitations or
software configuration), locality differences (e.g., language or time zones),
dependencies on other technologies, or the need for flexibility.
Although there are perceived benefits to providing options, there is also
a downside: options increase the variations that can exist among
implementations. The greatest way to undo the utility of a specification is
with too many options. Bascially, different implementations may specify
different combinations of options. This makes comparisons between
implementations difficult as well as complicates conformance testing and
increases the chance of non-interoperable implementations.
Example or story from [XSLT10] illustrating the problem? @@need example
Good Practice: Determine the need for each option.
Make sure there is a real need for the option.
What does it mean?Examine the reason for the option - is
it to address a real, existing need? Should it really be an option or made a
mandatory part of the specification? Be careful not to provide options in
anticipation for something that sounds like a good idea but will probably
never be realized - ask the implementers if they ever plan to need this. As
the specification progresses, if the option is not implemented, consider
removing it.
Technique
Develop a set of tests and analyze the results
to see what is implemented and how it is implemented.
Examples
Tests were used to determine if options were needed. As part of
its exit criteria for Candidate Recommendation, a Working Group created a set
of tests to 'test the specification'. The tests were able to show where there
was need for optionality (e.g., diversity among implementations and
flexibility justified) and where it was possible to narrow the choices (e.g.,
implementations used a much narrower set of values than what was
permitted).
Good Practice: Indicate any limitations or constraints
What does it mean? Provide as much information as possible to narrow the allowable choices and to increase predictability.
For implementation dependent values or
features, if possible, provide a range or set of permitted values rather than
leaving it completely open.
Technique
For options, especially enumerated lists, make sure that the number of
choices/options that can be implemented is known. Specifically, can zero,
exactly one, or several of the allowable choices be implemented? Does this
number depend on other parts of the specification or other chosen options?
Examples
Example: any specs that provide min/max values,
ranges, etc.
Examples of constraints include mandating that an implemenation implement
only one choice, implement every choice (allowing the user to decide), be
allowed to implement none of the choices, or be required to implement a
specified values and as many additional values as they wish from an
open-ended list.
Good Practice: Address the impact of the option
What does it mean?Think about the implications of both
implementing the option and of not implementing the option
Discuss the implications of either
using the option or not. It may be helpful to provide rationale for choosing
one option over another or for not using the option at all. Consider the
unintended consequence of the option - on other options, other parts of the
specification, on other specifications.
Technique
Questions to consider include:
- What affect will implementing the option have on a conforming
implementation, on interoperability?
- Is there any affect or negative consequence if the option is not
implemented?
- Must the option be implemented?
- Is there a default value for the option?
- Are there any dependencies, that is, the option can only be implemented
if certain conditions are true?
- Are the options mutually exclusive, that is, can one option override
and void another?
Good Practice: Make options easy to find. Use tags to label options.
Why care? To make it easier for implementers, test
creaters and users to discover the choices and variability allowed. For test
creaters, having a name for each item and a link target for each one, allows
systematic reference to individual items from test cases as well as elsewhere
in the specificaton.
D.3 Extensibility and Extensions
To accomodate changes in technology and information on the Web, a
specification can be designed for extensibility. A specification is
extensible when it provides a mechansim to allow any party to create
extensions. Extensions incorporate additional features beyond what
is defined in the specification. However, extensions can compromise
interoperabiltiy if there are too many differences between implementations.
The impact of extensions can be mitigated through features specifically
designed to allow new functionality. These features provide a 'standard way
to be non-standard' by including hooks, conformance rules, or other
mechanisms by which new functionality may be added in a conforming way.
Principle: Deal with the likelihood of extensions. Consider whether some parts of the specification will benefit from
extensibility. If so, define a mechanism to allow for the extension.
Good Practice: State the conditions under which
extensions are prohibited. This doesn't have to be complex, it may be a simple statement (e.g., extensions are not allowed). The statement is often associated with the
conformance clause.
Why care? Extensions increase
variability between implementations. Defining a mechanism will help to ensure
extensions are defined in a consistent manner leading to predictable handling
of extensions, including the ability to take appropriate actions (e.g., do
the extension, ignore, or take a fall-back behavior). By planning for
extensions and designing a specification for extensibility, there is less
chance that extensions will interfere with conformance to the base
specification and a better chance at achieving interoperability. On the other
hand, there may be areas in a specification that would not benefit from
extensibility and extensions are strictly forbidden (e.g., permissible
characters in XML 1.0 names for elements and attributes, most of the built-in
datatypes in Schema Part 2). This is called strict conformance. Strict
conformance is defined as conformance of an implementation that employs
only the requirements and/or functionality defined in the specification and
no more.
Reasons for designing extensibility into a specification include:
- providing a stable, useful framework to manage the rapid pace of
technology change,
- reducing the chances that extensions will interfere with conformance and
increasing the chances for achieving interoperablity,
- providing the ability to 'test-drive' new functionality that may
migrate into future specifications,
- enabling implementers to include functionality that they consider to be
required by their customers.
Technique
Technology and
application needs will dictate the best strategy for enabling
extensibility.
Many specifications include extensibility mechanism along with their
technology (XSLT, WSDL, SOAP, SVG). Whereas other specifications are
predominately about the extensibility mechanisms (e.g., Component Extension
(CX) API, HTTP Extension Framework) from which specifications are based
(e.g., CC/PP Exchange Protocol).
When designing for extensibility, it can get complicated. Questions to
consider that can impact design decisions include, but definately not limited
to:
- Is the specification contain a section for extension?
- Does the conformance section address the use of extension?
- Is there a well defined mechanism to create extension?
- Does this mechanism prevent the creation of extension conflicting with the semantics of the specification?
- When needed, is the scope of extensions is defined? (e.g., Are extensions authorized only for certain parts of the technology?)
- As an exercise, create a fake extension to illustrate problems that could arise interacting with the specification?
- As an example, create a fake extension as an example for developpers what would be the right way to do it?
- Can extensions interact with other extensions?
- Can extensions be combined?
- Can the extension mechanism be propagated? (e.g., Using the extension
mechanism of Specification UNO, Specification TOO is created. Can this
same mechanism be used create Specification TOOTOO, an extension of
Specification TOO?)
- Can extensions apply to a specification's profiles or modules and not
the core part of the specification?
Examples
Mechanism defined within the specification, extension indicator,
error handling instructions
XSLT 1.0 [XSLT10] provides extension mechanisms that allow an XSLT stylesheet to determine
whether the XSLT processor by which it is being processed:
- has implementations of particular extensions available,
- and to specify what should happen if those extensions are not available.
It defines two boolean functions: function-available (QName) and element-available (QName) that must be present in every implementation. These functions inform the XSLT processor that there is an extension, therefore the XSLT processor can return a value of false and provides handling
instructions (e.g., signal an error, perform fallback and not signal
error), if the extension is not available.
WSDL 2.0 [WSDL 2.0] defines binding extension elements which are used to provide
information specific to a particular binding. It is a two-part extensibility
model based on namespace-qualified elements and attributes. It provides the
syntax and semantics for signaling extensions. Extension elements can be
marked as mandatory, meaning that they MUST be processed correctly by the
WSDL processor - i.e., either agree to fully abide by all the rules and
semantics signaled by the extention or immediate cease processing (fault).
@@ someone should check this, does it make
sense?
Mechanism based on another specification's extension mechanism
CC/PP Exchange [CC/PP-EXCHANGE]
protocol using HTTP Extension Framework defines a HTTP extension to exchange
CC/PP descriptions effectively. The CC/PP exchange protocol conforms to
HTTP/1.1 and is a generic extension mechamism for HTTP/1.1 designed to
interoperate with existing HTTP applications. The CC/PP exchange protocol
uses an extension declaration to indicate that an extension has been applied
to a message and possibly to reserve a part of the header namespace. It
provides rules for which of the HTTP
Extension Framework extension declaration strengths and extension
delcaration scopes to use and defines the syntax and semantics of the header
fields.(@@ someone should check that I captured this
correctly)
OWL is a vocabulary extension
of RDF. OWL imposes
additional semantic conditions on RDF called semantic extensions of RDF.
These semantic extensions confom to the semantic conditions for simple
interpretations described in the RDF Semantics Recommendation. The OWL
semantics is consistent with RDF semantics, but OWL when understood as RDF is
'incomplete' versus the same OWL when understood as OWL. Thus, by
understanding OWL more is learned and nothing that is learned contradicts
what is learnt by RDF alone.
Mechanism defined as conformance rules
XHTML
Modularization defines a set of conformance rules to allow for the
extension of XHTML's layout and presentation capabiltiies. This method uses
the extensibility of XML without breaking the XHTML standard. It defines the
extension by using XML DTD or Schema which is uniquely identifiable, does not
redefine behavior of the technology it extends, and the syntactic and
semantic requirements are described and documented. Additionally, it cn have
some organization and naming constraints defined by the technology it
extends.
CSS 3 module: Syntax has an vendor specific extension mechanism perfectly described. The CSS WG has established a strict set of rules to define extensions which are:
A proprietary name should have a prefix consisting of:
- an underscore ("_") or a dash ("-"),
- the (possibly abbreviated) name of the company, organization, etc. that created it,
- an underscore or a dash.
Good Practice: Define error handling for unknown
extensions (@@is this only applicable to
agents?). When defining extensibility, include error handling instructions for when
an extension is not available or understood.
What does it mean?There are typically two approaches: (
from WebArch)
A good way to handle these two approaches is to have a way in the syntax
to distinguish which behavior is expected, e.g., mustUnderstand attribute in
Soap
1.2
Examples
Example: a hypothetical XHTML specification could suggest that user
agents, when they come across an unknown extension, take some fall-back
behavior, but recognizse the input as 'legal' rather than 'illegal'.
Applicability: Extensible specifications such as
languages and protocols. Unlikely candidates include: content specifications
(?) @@remove or keep Applicability?
D.4 Deprecation
The need for deprecation comes about when there are features (e.g.,
function argument, element or attribute) defined in the specification that
are no longer viable or have become outdated by newer constructs. These
deprecated features should be avoided and may become obsolete and no
longer defined in future versions of the specification.
Principle: Identify each deprecated feature. If there are deprecated features from a previous specification version,
let the reader know about them.
Why care? It provides the user
and implementer a 'user beware' warning, letting them know that use of the
feature is discouraged.
It helps the reader find the deprecated features when presented as a
collection in one place rather than distributed throughout the document.
Technique
Provide a list
of the features or a list of links to the features. Make this list easty to
find. Make it part of the Conformance Section of the document or put it in an
Appendix.
Let the reader know that there are deprecated features in the
specification. Don't blend the discussion of a deprecated feature into the
specification so that it appears as just another feature, without a means to
easily find it.
Examples
Avoid doing. Section 2.2 .2 Use of IRIs as Namespace
Names in Namespaces XML 1.1 discusses the deprecation of relative IRI
references. Difficult to find, luckily this is a very short Rec.
Principle: Specify the degree of support required for
each deprecated feature and its conformance consequence. Consider the effect of deprecation on all classes of products that implement the specification (e.g., authoring tools, user agents).
Why care? Deprecating a feature
has a direct affect on implementations - e.g., disallow use of the feature or
continue to support it for backward compatibility reasons.
For the purposes of backward compatibility, it may be necessary to specify
different requirements of support for each class of product - e.g., prohibit
its use in authoring tools, but require continued support by user agents.
Technique
Define what it
means for a feature to be deprecated. Do this for each class of product. Also
clarify the relation between the deprecated feature and Foo-conformance.
Examples
MathML2.0
section 7.2.1.2 describes deprecated MathML 1.x features in terms of
MathML-output-conformant authoring tools, MathML-input-confomrant
rendering/reading tools, and MathML-roundtrip-conrormant processors.
Good Practice: Indicate workarounds. Explain and give examples of how to avoid the deprecated feature
What does it mean? Examples are helpful in providing
alternatives or better ways to get the same results. Showing what can be done
in place of the deprecated feature will help get implementers and users to
discontinue use of the feature. Additional information to provide insight
into why the deprecation is also helpful.
Examples
Example: of alternative.
Namespaces XML
1.1 deprecation of IRI references includes a link to the deprecation ballot results,
providing background information on the proposal to deprecate, what this
means with respect to conformance to XML 1.0 and Namespaces as well as the
affect on other specifications (i.e., DOM, XPath).
Good Practice: Identify obsolete features
For historical purposes, a list of obsolete features is helpful. Providing
an alternative is also helpful. This is usually done as an informative
list.
Technique
List obsolete
features in the Change section or separate appendix of a specification.
Examples
HTML 4.01,
Appendix A: Changes lists the elements that are obsolete and suggests an
alternative element for use. "The following elements are obsolete: LISTING,
PLAINTEXT, and XMP. For all of them, authors should use the PRE element
instead."
D5. Error Handling?
@@ should we address this? does it belong
here?
E. Do Quality Control
As all editors know, the work is not finished when the writing is
completed. In fact, various review and checks of all W3C documents are
required prior to their publication and advancement. There are many things
that can be done to improve the quality and clarity of the document, aside
from the W3C process and publication rules and before Last Call reviews. Many
of these things can be an integrated part of the specification development.
Ensuring a quality document prior to external reviews can save time and
energy in that the Working Group may get fewer comments and issues to
resolve.[@@ this seems a bit pedantic?)
Example: One Working Group's document entered CR prior to a through
quality review, resulting in a huge number of issues to resolve and the
eventual retreat back to Working Draft for major revisions. Many of the
comments could have been prevented, such as: inconsistency and incompleteness
of the document (e.g., links to supplementary materials which did not exist
or were not complete, overlapping and conflicting requirements), undefined
terminology, and unimplementable requirements.
Example: When defining a language (such as XHTML) some requirements are
defined in a natural language (e.g., English) while others are defined using
a formal specification. Often there is overlap between the two approaches -
especially with XML Languages where a DTD or an XML Schema can be used for
two purposes: to define the syntactic requirements set by the language and/or
to allow validating documents conforming to the said language. This overlap
can lead to contradictions between the prose and the formal language, i.e.,
the prose says more than the formal language, formal language is more
constraining than the prose, or the formal language says something different
than the prose.
Good Practice: Check everything first. Do a systematic and thorough review. Don't forget to examine related
specifications and tools where correlation and consistency is essential.
What does it mean?Don't rush to publication or publish
too soon. First make sure that everything is finished and ready to go. This
is obvious, but often forgotten in the desire to meet deliverable dates.
Checking everything seems simple, especially if you have a monolithic
specification. But with the modularization of specifications as well as
interactions between specifications, there is a need to think globally. As an
illustration, consider this situation: If you submit a web page address to
the W3C MarkUp Validator, it could happen that the result page tells you that
your document is "Valid XHTML 1.0 Strict" and provides a link to the XHTML
1.0 Recommendation. If you follow this link you will encounter a problem:
There is no definition of what it means for a document to be identified as
"Valid XHTML 1.0 Strict".
Technique
Get colleagues to review the document.
Do a
walk-through, where experts do a detailed review of the content. Perhaps,
establish a team, where each person focuses on a different aspect of the
specification's correctness. For example, each person checks a different part
of the specification or do it by expertise - the grammarian checks for
consistency, grammar, spelling, readability, the test builder checks
requirements for precision and implementability, the conformant checks that
conformance criteria exists.
There's a need to split this section and redistribute it in the following good practices.
Why care? Catching problems as you go saves time and effort later on. It helps to mimize problems such as:
- incorrect requirements, definitions, statements, etc.,
- imprecise and ambiguous requirments due to the nature of human
language,
- unimplementable or untestabie requirements,
- specification creep - losing focus and spreading beyond the intended
targets,
- integration affects - relationships between various pieces of the
technology (e.g., different modules or a family of specifications).
Good Practice: Write sample
code or tests.
Technique
A test driven specification can provide an understanding of a
concept as well as help enhance clarity and usability as well as keep the
focus. It provides an illustration of correct usage. It can be used to
facilitate issue resolution as well as deployment. An added benefit is that
it provides a jump-start on testing efforts, including conformance testing
and providing tests to implementers as they build their software.
Examples
WebOnt or RDF experience - writing tests to drive the
specification. OWL
Test Cases.
CSS editors are encouraged (required?) to provide tests with all new content.
Good Practice: Write test assertions
Technique
Test assertions facilitate the development of consistent, complete
specifications and promote the early development of conformance tests. This
exercise helps to uncover incosistencies, ambiguities, gaps, and non-testable
statements in the specification. It can provide early feedback to the editors
regarding areas that need attention. An added benefit is that the assertions
can be used as input to test development efforts.
When trying to write test assertions for all its conformance
requirements, one Working Group found that informative information was needed
to complete the normative requirement. Consequently, the conformance
requirements were rewritten.
Examples
SOAP
version 1.2 Test Assertions
Assertion x1-conformance-part2
Location of the assertion: SOAP 1.2 Part 1, Section 1.2
Text from the specification: The implementation of an Adjunct MUST
implement all the pertinent mandatory requirements expressed in the
specification of the Adjunct to claim conformance with the Adjunct.
Comments: This statement applies to all assertions in part 2 and as such
will not be tested separately.
HTML
4.01:
Assertion 6.14-1
Reference: Section 6.14
(must) Script data ( %Script; in the DTD) can be the content of the
SCRIPT element and the value of intrinsic event attributes. User agents
must not evaluate script data as HTML markup but instead must pass it on as
data to a script engine.
Tests: 6_14-BF-01.html
XML:
Section: Documents
Type: Well_Formed
Purpose: A well formed document must have one or more elements one
Level 1
@@ need to figure this out. Much of this document is
non-normative - the goal is to get people to think about these aspects when
writing a specification, in some cases there isn't anything else to do other
than to be aware of the topic. For topics that don't apply are won't be part
of the specification, it would be silly to require a statement in their
specification that they don't do something - although there are some topics
that you do want to do this. Thus, it would be difficult to have a meaningful
test of SpecLite Principles - basically it is a checklist asking if they read
the Principle and considered it. What is key, is that there is a conformance
clause containing at least a minimal set of information (we need to determine
this).
Do this by -- completing the conformance template (which
needs to be developed) would signify conformance to this document
Having a conformance section in the specification that
addresses all the Principles in this document, would signify
conformance
Acknowledgments
The following QA Working Group and Interest Group participants have
contributed significantly to the content of this document:
- Jeremy Caroll (HP)
- Patrick Curran (Sun Microsystems),
- Dimitris Dimitriadis (Ontologicon),
- Karl Dubost (W3C),
- Dominique Hazaël-Massieux (W3C),
- Lofton Henderson (CGM Open),
- Björn Höhrmann,
- Susan Lesch (W3C),
- David Marston (Lotus Development Corp),
- Lynne Rosenthal (NIST),
- Mark Skall (NIST),
- Andrew Thackrah (Open Group),
- Olivier Théreaux (W3C).
References
Normative References
- QA-GLOSSARY
- Comprehensive glossary of QA terminology.
(Under construction, at http://www.w3.org/QA/glossary.)
- RFC2119
- IETF RFC 2119: Key words for use in RFCs to Indicate Requirement Levels, S. Bradner, March 1997. Available at
http://www.ietf.org/rfc/rfc2119.txt.
- CC/PP-EXCHANGE
- Latest version available at
http://www.w3.org/TR/NOTE-CCPPexchange .
- EDITOR
- W3C
Editor's Home Page, available at
http://www.w3.org/2003/Editors/ .
- QA-HANDBOOK
- The QA Handbook, Lofton Henderson, W3C Working Draft 10 May 2004, (work in progress). http://www.w3.org/TR/2004/WD-qa-handbook-20040510/ . Latest version available at http://www.w3.org/TR/qa-handbook/
- RFC2360
- IETF
Guide for Internet Standards Writers, June 1998, available
at http://www.ietf.org/rfc/rfc2360.txt.
- XSLT10
- XSL
Transformations (XSLT) Version 1.0 , J. Clark, Editors,
W3C Recommendation, 16 November 1999,
http://www.w3.org/TR/1999/REC-xslt-19991116 . Latest version available at
http://www.w3.org/TR/xslt .
Glossary
- Class of Products
- generic name for the group of products or services that would
implement, for the same purpose, the specification, (i.e., target of
the specification). A specification may identify several classes of
products.
- Conformance
- fulfillment by a product, process, systems, or service of a specified
set of requirements.
- Conformance clause
- a section of the specification that defines the requirements,
criteria, or conditions to be satisfied by an implmentation in order to
claim conformance.
- Deprecated Feature
- an existing feature that has become outdated by a newer construct and
is no longer viable.
- Dimensions of Variability (DoV)
- the ways in which different products that are conformant to a
specification may vary among themselves.
- Discretionary Items
- deliberate and explicit grants of discretion by the specification to
the implementations, that describe or allow optionality of behavior,
functionality, parameter values, error handling, etc.
- Extension
- The ability to incorporate additional functionality beyond what is
defined in the specification. It broadens the possibility of the
technology.
- Extensible
- The ability of a specification to accept extensions in a define way.
A specification is extensible if it provides a mechanism for any party
to create extensions
- Implementation
- An implementation is a realization of a technology in accordance to the principles defined in the technical specifications for this technology. This implementation can be a document, product, application, process, service, system, or other entity.
- Strict Conformance
- conformance of an implementation that employs only the requirements
and/or functionality defined in the specification and no more (i.e., no
extensions to the specification are implemented).
- Testability
- A proposition is testable if there is such a procedure that assesses the truth value of a proposition with a high confidence level. Whether the confidence level is measurable and what confidence level is high enough depends on the proposition and its framework. (@@ better definition?)
- Test Assertion
- a statement of behavior, action, or condition that can be measured or
tested. It is derived from the specification's requirements.