A lot of effort goes into writing a good specification. It takes more than knowledge of the technology to make a specification precise, implementable and testable. It takes planning, organization, and foresight about the technnology and how it will be implemented and used. The goal of this document is to help W3C editors write better specifications, by making a specification easier to interpret without ambiguity and clearer as to what is required in order to conform. It focuses on how to define and specify conformance for a specification. Additionally, it addresses how a specification might allow variation among conforming implementations. The document is presented as a set of guidelines or principles, supplemented with good practices, examples, and techniques.
This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.
This document is a W3C Working Draft (WD), made available by the W3C Quality Assurance (QA) Activity for discussion by W3C members and other interested parties. For more information about the QA Activity, please see the QA Activity statement.
This is the first published version of this document after the decision to completely redesign the QA Framework (QAF), resolved by the QA Working Group (QAWG) at its 2004 Technical Plenary face-to-face. It is a lighter-weight, less authoritarian, more user-friendly and useful version of the previously published Candidate Recommendation version of the Specification Guidelines. Because of the extensive nature of the changes, it has been returned to Working Draft.
This draft accurately reflects the intended overall structure and content of the new, lightweight Specification Guidelines. However it is incomplete in some details (generally indicated by "@@" or "TBD"), especially in the examples -- lots more examples are intended. Also, some sections more resemble outlines than completed prose. Because this is a significant departure from the last-published Specification Guidelines, QAWG wants review and comment on the overall direction as soon as possible. Incomplete details will be fleshed out in the next publication.
This Specification Guidelines document will be coordinated with the other parts of the QA Framework, The QA Handbook [QA-HANDBOOK] and QA Framework: Test Guidelines. Synchronization at uniform level of maturity will occur no later than Last Call. The first publications of the various parts will occur somewhat more independently.
The QAWG expects this specification to become a Recommendation.
Publication as a Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.
This document has been produced under the 24 January 2002 CPP as amended by the W3C Patent Policy Transition Procedure. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) with respect to this specification should disclose the information in accordance with section 6 of the W3C Patent Policy. Patent disclosures relevant to this specification may be found on the Working Group's patent disclosure page.
Comments on this document may be emailed to email@example.com, the publicly archived list of the QA Interest Group (QAIG). Commenters please note that your comments will be publicly archived and available, do not send information that should not be distributed, such as private data. If you are able to give your feedback by 12 June 2004, the QAWG can consider it at its next face-to-face meeting. The comments will still be collected after the F2F in June 2004.
This document is a guide for W3C editors and authors. It provides guidelines for writing better specifications. This document differs from other W3C process and publication related documents in that it looks at specifications from a conformance viewpoint. It address the most basic and critical topics with respect to conformance, including how to convey what is required in order to conform and how to allow variation among conforming impelmentations. The goal is to make specifications precise, easier to interpret without ambiguity or confusion, and clearer as to what is required in order to conform. Good specifications lead to better implementations and foster the develoment of test suites and tools. Conforming implemenations lead to interoperablity.
Everyone benefits from having well-written specifications. Editors may have less rework and thus, fewer errata. Implementers can implement sooner and have a better chance to conform to the specification. Test developers are able to derive unambigious requirements. Users benefit by having interoperable solutions. W3C gains by having recommendations produced with higher quality and reduced maintenance.
It is not an easy task to write accurate, clear, complete, unbiased specifications. It requires planning, organization, and foresight about the technology, how it will be implemented and used, and how technology decisions affect conformance. This document provides a collection of principles, good practices, examples, and techniques that leads the reader though the decisions that need to be made in order to write precise requirements and establish, define, and specify conformance for specifications. Realizing that editors and authors are busy, under pressure to get the specification published, and already have a reading list of W3C documents (a good place to start, W3C Editor's Home Page [EDITOR] this document can be used as a checklist or reminder of things to consider, a how-to guide with examples and techniques which can be adapted, a reference for understanding conformance concepts and terminology, or all of these.
This document is a practical guide to writing a specification, presenting editors with topics to consider. Consequently, much of the document is non-normative recommendations labeled Good Practices. There are however, a limited number of normative requirements, labeled Principles, that must be implemented in order to conform to this document. The overall objective of these principles and good practices is to facilitate the creation of a conformance clause in every specification. A conformance template is provided to assist editors satisfy the requirements of this document and end up with a conformance clause. Note that for some specifications (e.g., QAH, WebArch) where conformance is not an issue (e.g., no normative content), the conformance clause may be an explanation of why there is no "conformance to this document" which may not be a separate section but incorporated into another section.
The topics presented are inclusive (self-contained) providing information needed to understand and successfuly apply the principle, although related information and advanced topics may be referenced.
If in a hurry - start at the first section, Specifying confomrance -- this is the expected outcome of adhering to this document. It serves as a roadmap to other parts of this document, which help you achieve Secifying Conformance.
The primary audience of this document is editors and authors, however, it is applicable to a broader audience including:
This document makes no distinction between the terms editors and authors and will refer to them collectively as editors.
This document is organized in a series of main topics like Conformance or Extensibility. In each of this main topics, a few principles are developed and explained. Theses principles are accompanied by techniques and examples. The techniques illustrate basic (and non exhaustive) questions or methods that might help you create a specification. The examples are explanations or extract of specifications which specifically address the point which has been made in the principle.
explain what is normative and what are the conformance requirements
The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY ", and "OPTIONAL" will be used as defined in RFC 2119 [RFC2119]. When used with the normative RFC2119 meanings, they will be all uppercase. Occurrences of these words in lowercase comprise normal prose usage, with no normative implications.
This document can be used as a stand alone document or as part of a family of QA Framework documents designed to help the WGs improve all aspects of their quality practices. The QA Handbook [QA-HANDBOOK] is a non-normative handbook about the process and operational aspects of the quality practices of W3C's Working Groups (WG). @@Add reference to the Test GL@@
Clear presentation of what is meant by conformance to the specification is ultimately crucial to successful interoperability of implementations. The conformance policy and language in the specification not only determine its testability, but also shape the overall look of its implementations landscape -- are there numerous fragmented flavors or a few strong, interoperating variations?
Conformance is the fulfillment of a product, process, or service of specified requirements. These requirements are specified in a standard or specification as either part of a conformance clause or in the body of the specification. A conformance clause is a section of a specification that states all the requirements or criteria that must be satisfied to claim conformance to the specification.
With a good conformance clause, a specification comes close to full conformance to this Specification Guidelines. Therefore:
Principle: Include a conformance clause.
What does it mean? The conformance section of a specification -- commonly called the Conformance Clause -- is a high-level description of what is required of implementers and application developers. It, in turn, may refer to other parts of the specification for some details. Ideally, it is the root source from which readers can find any conformance-related information.
For starters, the conformance clause needs to answer the all-important question: what may conform and how? It may, for conformance purposes, partition the technology into functional subsets, such as profiles, levels, or other structures. Additionally it may specify the permissibility of extensions, options, and alternative approaches and how they are to be handled.
Why care? A well done conformance clause brings most of the benefit of these specification guidelines for the least effort. Almost all of SpecGL's significant advice -- normative Principles and recommended Good Practices alike -- are satisfied with a good conformance clause.
Simple, complete the conformance clause template and put the result into the specification.
To be honest, answering the questions in the conformance clause template may not be a simple matter, and may lead the Working Group into thorny issues. However, these are questions that must be answered if the specification is to be successful, i.e., if it is to foster multiple high quality interoperating implementations.
For some specifications, the conformance landscape may be plain and simple, and the conformance clause template may be almost all that is needed [Example?]. For others, the conformance landscape will be complex or convoluted, and the advanced details and topics of these specification guidelines -- topics like multiple intersecting Dimensions of Variability (@@ add DoV appendix?)-- may be invaluable [Example, like JC's OWL comments?].
The scope of content in a good conformance clause overlaps most of the other remaining topics in these specification guidelines. Possible repetition notwithstanding, a handful of points -- conformance clause Good Practices -- are worth mentioning here up front. (@@huh, this needs explaining?)
Good Practice: Define the specification's conformance model in the conformance clause.
What does it mean? The conformance model is the conceptual framework in which conformance is defined. It consists of and is defined by addressing at least these three topics:
Good Practice: Specify in the conformance clause how to distinguish normative from informative content.
What does it mean? Normative content is the prescriptive part of the specification, whereas informative content is for informational purposes and assists in the understanding and use of the specification. Content includes all sorts of different forms -- not only descriptive prose, but illustrations, examples, use cases, formulae and other formalisms, etc.
Why care? Conformance of implementations is defined by and measured against normative content.
This Good Practice is aimed principally at the high level partitioning of information in the specification. The next, related Good Practice considers the use of language at a fine-grained level:
Good Practice: In the conformance clause, define how normative language is expressed.
What does it mean? There are a number of common language styles for expressing the detailed conformance requirements of a specification. One of the most popular is RFC 2119 [RFC 2119] -- keywords like must, may, and should not both signal the presence of conformance requirements and express their relative "manditoriness". Other styles include imperative voice statements (as in the statement of the Good Practice item), physical segregation via styling, labelling, etc. What matters is that the style be consistent, unambiguous, and (ideally) quickly recognized.
Why care? As before, conformance of implementations is defined by and measured against normative content, which includes thedetailed conformance requirements.
The central message of this section -- "have a good conformance clause" --has many ancilliary details. Because the conformance clause is the foundation for defining and measuring conformance, it is also the basis for assessing conformance claims. One detail worthy of attention is "valid conformance claims".
[Ed note. valid? or well-formed? Maybe:well-formed if it meets the minimal bullet list of contents below.; valid ifit has an ICS that supports the claim. Or ... avoid the question fornow?]
Rather than live with the infinite varieties of creative conformance claims that will arise in a vacuum, the specification can be proactive:
Principle: Provide the wording for conformance claims.
[@@ Or is this more like a GP?]
What does it mean? Most of those items are self-explanatory -- unambiguous identification of the specification (version), of the product, etc. "Degree of conformance" pertains where the specification's conformance model is not a single, monolithic category. For example, conformance might be defined in terms of variable collections of functional modules. (The topic of conformance variability is dealt with in much more detail later.)
A well-formed conformance claim should include at least this information:
The specification should require that conformance claims contain at least this information.
[Ed note. WAI specs have some good examples.]
Good Practice: Provide an Implementation Conformance Statement (ICS) proforma.
What does it mean? An Implementation Conformance Statement (ICS) provides standardized information about the conformance of an implementation to the specification. It indicates which capabilities and options have been implemented, as well as the limitations of the implementation. An ICS typically takes the form of a questionnaire for implementor to complete.
This Good Practice suggests that the specification itself include an ICS proforma. (Caveat. The ICS concept may be inapplicable to some types of specifications.)
Why care? An ICS provides detail about conformance. The detail can, for example, be used to identify the subset of a conformance test suite that is applicable to the implementation to be tested. An ICS can also be especially valuable in optionality choices in the implementation, and documenting the presence of extensions.
Good Practice: Require an Implementation Conformance Statement (ICS) as part of valid conformance claims.
What does it mean? This simply puts together the previous two good practices. Not only could the specification provide an ICS proforma for implementors, but it could require it to be linked from its standardized conformance claim template.
The path to a quality specification begins with its scope. It is critical to convey what the specification is about by describing its intent and applicability. As the specification develops, its a good idea to revisit the scope to make sure it still reflects the intent of specification or if it needs to be modified.
Principle: Define the scope. Describe what the specification is about.
In addition to containing the subject of the specification, a more complete scope contains:
Note that it is not necessary to write a separate statement for each of these items, rather, they can be combined.
Why care? It helps to keep the specification content focused. It also will help readers know the limits or boundaries of the specification and whether it is of interest to them.
Examples: of good Scopes, bad scopes?
Good Practice: Be direct, providing simple, understandable statements of facts. Don't specify requirements
Use statements such as: This document (@@check other Rec's and use examples from them)
Good Practice: Use examples or use cases to illustrate
Providing examples, use cases or both helps to clarify the scope for the reader.
@@Here are different ways to say basically the same thing.
Foo defines conformance for: (fill in the rest)
Examples of 'what': protocol, API, content
Advanced Topic: Classes of Product, DoV
Good Practice: Start now and keep adding to it as you go.
Define conformance concepts, designations
Examples: valid, well-formed, foo-conformant, document conformance (CC/PP) consumer conformance (CC/PP)
Good Practice: being consistent and using the same term when you mean the same thing don't be a thesaurus.
(Remove or keep - redundant with A1, GP. not much new to say here)
Distinguish between normative and non-nomative content
(Remove or keep - redundant with A1, GP.)
Indicate what requirements are mandatory as opposed to optional, recommended, good ideas, etc.
Expressions that conveys the criteria to be fulfilled in an implementation of the specification
Examples: use RFC keywords, assertive voice, gold stars.
Good Practice: being able to find these requirements easily navigation
Variability affects interoperability
Subdivide into related groups of conformance requirements
Examples: modules, levels, profiles
@@ not sure what to do with these questions, they lead you to doing the right thing.
Examples of reasons to divide:
Options in specifications provide implementations the freedom to make choices about
These choices also called dscretionary items, give implementers of the technology the opportunity to decide from alternatives when building their applications and tools.They describe or allow optionality of behavior, functionality, parameter values, error handling, etc. They may be considered necessary because of environmental conditions (e.g., hardware limitations or software configuration), locality differences (e.g., language or time zones), dependencies on other technologies, or the need for flexibility.
Although there are perceived benefits to providing options, there is also a downside: options increase the variations that can exist among implementations. The greatest way to undo the utility of a specification is with too many options. Bascially, different implementations may specify different combinations of options. This makes comparisons between implementations difficult as well as complicates conformance testing and increases the chance of non-interoperable implementations.
Example or story from [XSLT10] illustrating the problem? @@need example
Good Practice: Determine the need for each option. Make sure there is a real need for the option.
What does it mean?Examine the reason for the option - is it to address a real, existing need? Should it really be an option or made a mandatory part of the specification? Be careful not to provide options in anticipation for something that sounds like a good idea but will probably never be realized - ask the implementers if they ever plan to need this. As the specification progresses, if the option is not implemented, consider removing it.
Develop a set of tests and analyze the results to see what is implemented and how it is implemented.
Tests were used to determine if options were needed. As part of its exit criteria for Candidate Recommendation, a Working Group created a set of tests to 'test the specification'. The tests were able to show where there was need for optionality (e.g., diversity among implementations and flexibility justified) and where it was possible to narrow the choices (e.g., implementations used a much narrower set of values than what was permitted).
Good Practice: Indicate any limitations or constraints
What does it mean? Provide as much information as possible to narrow the allowable choices and to increase predictability.
For implementation dependent values or features, if possible, provide a range or set of permitted values rather than leaving it completely open.
For options, especially enumerated lists, make sure that the number of choices/options that can be implemented is known. Specifically, can zero, exactly one, or several of the allowable choices be implemented? Does this number depend on other parts of the specification or other chosen options?
Example: any specs that provide min/max values, ranges, etc.
Examples of constraints include mandating that an implemenation implement only one choice, implement every choice (allowing the user to decide), be allowed to implement none of the choices, or be required to implement a specified values and as many additional values as they wish from an open-ended list.
Good Practice: Address the impact of the option
What does it mean?Think about the implications of both implementing the option and of not implementing the option
Discuss the implications of either using the option or not. It may be helpful to provide rationale for choosing one option over another or for not using the option at all. Consider the unintended consequence of the option - on other options, other parts of the specification, on other specifications.
Questions to consider include:
Good Practice: Make options easy to find. Use tags to label options.
Why care? To make it easier for implementers, test creaters and users to discover the choices and variability allowed. For test creaters, having a name for each item and a link target for each one, allows systematic reference to individual items from test cases as well as elsewhere in the specificaton.
To accomodate changes in technology and information on the Web, a specification can be designed for extensibility. A specification is extensible when it provides a mechansim to allow any party to create extensions. Extensions incorporate additional features beyond what is defined in the specification. However, extensions can compromise interoperabiltiy if there are too many differences between implementations. The impact of extensions can be mitigated through features specifically designed to allow new functionality. These features provide a 'standard way to be non-standard' by including hooks, conformance rules, or other mechanisms by which new functionality may be added in a conforming way.
Principle: Deal with the likelihood of extensions. Consider whether some parts of the specification will benefit from extensibility. If so, define a mechanism to allow for the extension.
Good Practice: State the conditions under which extensions are prohibited. This doesn't have to be complex, it may be a simple statement (e.g., extensions are not allowed). The statement is often associated with the conformance clause.
Why care? Extensions increase variability between implementations. Defining a mechanism will help to ensure extensions are defined in a consistent manner leading to predictable handling of extensions, including the ability to take appropriate actions (e.g., do the extension, ignore, or take a fall-back behavior). By planning for extensions and designing a specification for extensibility, there is less chance that extensions will interfere with conformance to the base specification and a better chance at achieving interoperability. On the other hand, there may be areas in a specification that would not benefit from extensibility and extensions are strictly forbidden (e.g., permissible characters in XML 1.0 names for elements and attributes, most of the built-in datatypes in Schema Part 2). This is called strict conformance. Strict conformance is defined as conformance of an implementation that employs only the requirements and/or functionality defined in the specification and no more.
Reasons for designing extensibility into a specification include:
Technology and application needs will dictate the best strategy for enabling extensibility.
Many specifications include extensibility mechanism along with their technology (XSLT, WSDL, SOAP, SVG). Whereas other specifications are predominately about the extensibility mechanisms (e.g., Component Extension (CX) API, HTTP Extension Framework) from which specifications are based (e.g., CC/PP Exchange Protocol).
When designing for extensibility, it can get complicated. Questions to consider that can impact design decisions include, but definately not limited to:
It defines two boolean functions:
function-available (QName) and
element-available (QName) that must be present in every implementation. These functions inform the XSLT processor that there is an extension, therefore the XSLT processor can return a value of false and provides handling
instructions (e.g., signal an error, perform fallback and not signal
error), if the extension is not available.
WSDL 2.0 [WSDL 2.0] defines binding extension elements which are used to provide information specific to a particular binding. It is a two-part extensibility model based on namespace-qualified elements and attributes. It provides the syntax and semantics for signaling extensions. Extension elements can be marked as mandatory, meaning that they MUST be processed correctly by the WSDL processor - i.e., either agree to fully abide by all the rules and semantics signaled by the extention or immediate cease processing (fault). @@ someone should check this, does it make sense?
CC/PP Exchange [CC/PP-EXCHANGE] protocol using HTTP Extension Framework defines a HTTP extension to exchange CC/PP descriptions effectively. The CC/PP exchange protocol conforms to HTTP/1.1 and is a generic extension mechamism for HTTP/1.1 designed to interoperate with existing HTTP applications. The CC/PP exchange protocol uses an extension declaration to indicate that an extension has been applied to a message and possibly to reserve a part of the header namespace. It provides rules for which of the HTTP Extension Framework extension declaration strengths and extension delcaration scopes to use and defines the syntax and semantics of the header fields.(@@ someone should check that I captured this correctly)
OWL is a vocabulary extension of RDF. OWL imposes additional semantic conditions on RDF called semantic extensions of RDF. These semantic extensions confom to the semantic conditions for simple interpretations described in the RDF Semantics Recommendation. The OWL semantics is consistent with RDF semantics, but OWL when understood as RDF is 'incomplete' versus the same OWL when understood as OWL. Thus, by understanding OWL more is learned and nothing that is learned contradicts what is learnt by RDF alone.
XHTML Modularization defines a set of conformance rules to allow for the extension of XHTML's layout and presentation capabiltiies. This method uses the extensibility of XML without breaking the XHTML standard. It defines the extension by using XML DTD or Schema which is uniquely identifiable, does not redefine behavior of the technology it extends, and the syntactic and semantic requirements are described and documented. Additionally, it cn have some organization and naming constraints defined by the technology it extends.
CSS 3 module: Syntax has an vendor specific extension mechanism perfectly described. The CSS WG has established a strict set of rules to define extensions which are:
A proprietary name should have a prefix consisting of:
- an underscore ("_") or a dash ("-"),
- the (possibly abbreviated) name of the company, organization, etc. that created it,
- an underscore or a dash.
Principle: Prevent extensions from breaking conformance
Extensions must not contradict or negate conformance to the specification. If it conformed without the extension, conformance should hold true with the extension.
Include statements in the specification such as:
In XSLT 1.0, extension attributes (from other namespaces) can be present on the official XSLT elements, but they are prohibited from changing the specified behavior within the detectability of conforming behavior. Thus, an extension attribute can cause the element to perform faster but cannot change the result.
Good Practice: Define error handling for unknown extensions (@@is this only applicable to agents?). When defining extensibility, include error handling instructions for when an extension is not available or understood.
What does it mean?There are typically two approaches: ( from WebArch)
Must ignore can be further refined, requiring that the unknown item be passed through unchanged to the next downstream process, while other tehcnologies simply discard it.
A good way to handle these two approaches is to have a way in the syntax to distinguish which behavior is expected, e.g., mustUnderstand attribute in Soap 1.2
Example: a hypothetical XHTML specification could suggest that user agents, when they come across an unknown extension, take some fall-back behavior, but recognizse the input as 'legal' rather than 'illegal'.
Applicability: Extensible specifications such as languages and protocols. Unlikely candidates include: content specifications (?) @@remove or keep Applicability?
The need for deprecation comes about when there are features (e.g., function argument, element or attribute) defined in the specification that are no longer viable or have become outdated by newer constructs. These deprecated features should be avoided and may become obsolete and no longer defined in future versions of the specification.
Principle: Identify each deprecated feature. If there are deprecated features from a previous specification version, let the reader know about them.
Why care? It provides the user and implementer a 'user beware' warning, letting them know that use of the feature is discouraged.
It helps the reader find the deprecated features when presented as a collection in one place rather than distributed throughout the document.
Provide a list of the features or a list of links to the features. Make this list easty to find. Make it part of the Conformance Section of the document or put it in an Appendix.
Let the reader know that there are deprecated features in the specification. Don't blend the discussion of a deprecated feature into the specification so that it appears as just another feature, without a means to easily find it.
HTML 4.01, Appendix A: Changes includes a section called "Deprecated elements" which provides a linked list to the elements.
Avoid doing. Section 2.2 .2 Use of IRIs as Namespace Names in Namespaces XML 1.1 discusses the deprecation of relative IRI references. Difficult to find, luckily this is a very short Rec.
Principle: Specify the degree of support required for each deprecated feature and its conformance consequence. Consider the effect of deprecation on all classes of products that implement the specification (e.g., authoring tools, user agents).
Why care? Deprecating a feature has a direct affect on implementations - e.g., disallow use of the feature or continue to support it for backward compatibility reasons.
For the purposes of backward compatibility, it may be necessary to specify different requirements of support for each class of product - e.g., prohibit its use in authoring tools, but require continued support by user agents.
Define what it means for a feature to be deprecated. Do this for each class of product. Also clarify the relation between the deprecated feature and Foo-conformance.
MathML2.0 section 184.108.40.206 describes deprecated MathML 1.x features in terms of MathML-output-conformant authoring tools, MathML-input-confomrant rendering/reading tools, and MathML-roundtrip-conrormant processors.
Good Practice: Indicate workarounds. Explain and give examples of how to avoid the deprecated feature
What does it mean? Examples are helpful in providing alternatives or better ways to get the same results. Showing what can be done in place of the deprecated feature will help get implementers and users to discontinue use of the feature. Additional information to provide insight into why the deprecation is also helpful.
Example: of alternative.
Namespaces XML 1.1 deprecation of IRI references includes a link to the deprecation ballot results, providing background information on the proposal to deprecate, what this means with respect to conformance to XML 1.0 and Namespaces as well as the affect on other specifications (i.e., DOM, XPath).
Good Practice: Identify obsolete features
For historical purposes, a list of obsolete features is helpful. Providing an alternative is also helpful. This is usually done as an informative list.
List obsolete features in the Change section or separate appendix of a specification.
HTML 4.01, Appendix A: Changes lists the elements that are obsolete and suggests an alternative element for use. "The following elements are obsolete: LISTING, PLAINTEXT, and XMP. For all of them, authors should use the PRE element instead."
@@ should we address this? does it belong here?
As all editors know, the work is not finished when the writing is completed. In fact, various review and checks of all W3C documents are required prior to their publication and advancement. There are many things that can be done to improve the quality and clarity of the document, aside from the W3C process and publication rules and before Last Call reviews. Many of these things can be an integrated part of the specification development. Ensuring a quality document prior to external reviews can save time and energy in that the Working Group may get fewer comments and issues to resolve.[@@ this seems a bit pedantic?)
Example: One Working Group's document entered CR prior to a through quality review, resulting in a huge number of issues to resolve and the eventual retreat back to Working Draft for major revisions. Many of the comments could have been prevented, such as: inconsistency and incompleteness of the document (e.g., links to supplementary materials which did not exist or were not complete, overlapping and conflicting requirements), undefined terminology, and unimplementable requirements.
Example: When defining a language (such as XHTML) some requirements are defined in a natural language (e.g., English) while others are defined using a formal specification. Often there is overlap between the two approaches - especially with XML Languages where a DTD or an XML Schema can be used for two purposes: to define the syntactic requirements set by the language and/or to allow validating documents conforming to the said language. This overlap can lead to contradictions between the prose and the formal language, i.e., the prose says more than the formal language, formal language is more constraining than the prose, or the formal language says something different than the prose.
Good Practice: Check everything first. Do a systematic and thorough review. Don't forget to examine related specifications and tools where correlation and consistency is essential.
What does it mean?Don't rush to publication or publish too soon. First make sure that everything is finished and ready to go. This is obvious, but often forgotten in the desire to meet deliverable dates.
Checking everything seems simple, especially if you have a monolithic specification. But with the modularization of specifications as well as interactions between specifications, there is a need to think globally. As an illustration, consider this situation: If you submit a web page address to the W3C MarkUp Validator, it could happen that the result page tells you that your document is "Valid XHTML 1.0 Strict" and provides a link to the XHTML 1.0 Recommendation. If you follow this link you will encounter a problem: There is no definition of what it means for a document to be identified as "Valid XHTML 1.0 Strict".
Get colleagues to review the document.
Do a walk-through, where experts do a detailed review of the content. Perhaps, establish a team, where each person focuses on a different aspect of the specification's correctness. For example, each person checks a different part of the specification or do it by expertise - the grammarian checks for consistency, grammar, spelling, readability, the test builder checks requirements for precision and implementability, the conformant checks that conformance criteria exists.
There's a need to split this section and redistribute it in the following good practices.
Why care? Catching problems as you go saves time and effort later on. It helps to mimize problems such as:
Good Practice: Write sample code or tests.
Why care? [TBD]
A test driven specification can provide an understanding of a concept as well as help enhance clarity and usability as well as keep the focus. It provides an illustration of correct usage. It can be used to facilitate issue resolution as well as deployment. An added benefit is that it provides a jump-start on testing efforts, including conformance testing and providing tests to implementers as they build their software.
WebOnt or RDF experience - writing tests to drive the specification. OWL Test Cases.
CSS editors are encouraged (required?) to provide tests with all new content.
Good Practice: Write test assertions
Why care? [TBD]
Test assertions facilitate the development of consistent, complete specifications and promote the early development of conformance tests. This exercise helps to uncover incosistencies, ambiguities, gaps, and non-testable statements in the specification. It can provide early feedback to the editors regarding areas that need attention. An added benefit is that the assertions can be used as input to test development efforts.
When trying to write test assertions for all its conformance requirements, one Working Group found that informative information was needed to complete the normative requirement. Consequently, the conformance requirements were rewritten.
Location of the assertion: SOAP 1.2 Part 1, Section 1.2
Text from the specification: The implementation of an Adjunct MUST implement all the pertinent mandatory requirements expressed in the specification of the Adjunct to claim conformance with the Adjunct.
Comments: This statement applies to all assertions in part 2 and as such will not be tested separately.
Reference: Section 6.14
(must) Script data ( %Script; in the DTD) can be the content of the SCRIPT element and the value of intrinsic event attributes. User agents must not evaluate script data as HTML markup but instead must pass it on as data to a script engine.
Purpose: A well formed document must have one or more elements one
Good Practice: Use formal languages.
Why care? [TBD]
@@ Leave here or as Advance Topic? Either way, we need prose, examples, experiences, etc. Or it needs to be removed.
It is possible using formal languages (e.g., XML Schema, UML) to check for correctness of the specification and that requirements are specified clearly and unambigiously. The checking can be done via automated tools.
EXamples? For formal language specifications - test the formal language. (wiki FormalLanguageVsProse)
@@ need to figure this out. Much of this document is non-normative - the goal is to get people to think about these aspects when writing a specification, in some cases there isn't anything else to do other than to be aware of the topic. For topics that don't apply are won't be part of the specification, it would be silly to require a statement in their specification that they don't do something - although there are some topics that you do want to do this. Thus, it would be difficult to have a meaningful test of SpecLite Principles - basically it is a checklist asking if they read the Principle and considered it. What is key, is that there is a conformance clause containing at least a minimal set of information (we need to determine this).
Do this by -- completing the conformance template (which needs to be developed) would signify conformance to this document
Having a conformance section in the specification that addresses all the Principles in this document, would signify conformance
The following QA Working Group and Interest Group participants have contributed significantly to the content of this document: