When writing specifications it is critical to understand their primary purpose and scope. A clearly defined scope helps to keep the specification content focused and unambiguous.
The scope describes the areas covered by the specification, thereby indicating the intent or applicability of the specification. It does not specify requirements and is worded as a series of statements of fact.
The specification MUST define the subject matter of the specification, and SHOULD include a statement of objectives and/or goals. This information MUST appear at the beginning of the document.
It helps the writer and the reader know the limits of what is specified in a normative document. It tells the reader how the specification could be used and by whom.
Note that defining the classes of product as required by the checkpoint 2.1 contributes to defining the scope of a document.
The specification MUST provide examples or use cases illustrating what is in scope.
A set of broadexamples and/or use cases helps to clarify the specification's scope. It gives the reader additional information in understanding the purpose, audience, and applicability of the specification.
It is up to the Working Group to determine the best way to illustrate the scope, i.e., to use examples or use cases or both.
Providing is understood here and in the following conformance requirements as either included in the specification or linked from it.
The specification MUST provide one or more examples of the functionality, concepts, and behaviors of the specification.
It is difficult to understand some concepts, behaviors, or functionality without informative interpretations to aid the reader. These examples are specific in nature and are used to make clear a concept, behavior, interaction, etc. that is innately complex or for which the WG has had to explain to its members or the public.
The nature of the specification will influence the type of examples that are relevant. For example, for markup specifications, provide at least one example of each markup construct; for transformation specifications, illustrate each transformation capability with an example showing input and output.
Identifying what needs to conform to the specification is essential for the specification writers as well as for the implementors. This guideline relies on the concepts of classes of products and specification categories.
Classes of product is a dimension of variability (DoV).
A specification:
Doing so helps define the scope of the specification. It also helps the reader know what is being targeted by the specification (i.e., what needs to conform to the specification).
The list of classes of products enumerate the most common classes of products. If your class of product matches one or more terms in the list, then use the terms in the list. If no term is an appropriate match, then define your own.
Example. Within the SMIL 2.0 Language Profile ([SMIL20], chapter 13), there are 2 classes of products: documents and basic user agents. Within Mathematical Markup Language (MathML) 2.0 [MATHML20] there are output-compliant, authoring tools and input-compliant, rendering/reading tools.
The conformance requirements indicate the conditions to be satisfied in order to claim conformance. It is likely that these conformance requirements will reference normative text within the specification or in other related specifications.
The specification MUST define the conformance requirements for each class of product identified in checkpoint 2.1.
The specification:
Understanding the specification category that is being written contributes towards understanding the classes of products, as required by the preceding checkpoint. Moreover, it helps both the author(s) and readers understand the scope of the specification and its focus.
If your specification matches one or more a categories in the list, then use those categories. If no category is an appropriate match, then define your own.
Some specifications define more than one of the enumerated category types. XForms [XFORMS] is an example. It defines: Content, via an XML vocabulary; Content, via named datatypes; Syntax, in the form of a set of functions to supplement the XPath core set; behavior of a processor; behavior of a user agent; a set of events.
The specification MUST address and discuss the relations and interactions between classes of product and the other dimensions of variability. This checkpoint is not applicable if there is only one class of product.
Be aware that excessive variability harms interoperability.
To address various needs (re-usability, evolution, scaling, ...), technologies are often divided in different ways. These divisions have an important impact on the conformance to a specification. This guideline relies on the concepts of profiles, modules and levels that the QA WG has identified as being the three main division types in use.
Profiles, modules and levels are each a dimension of variability (DoV).
A specification:
This checkpoint is not applicable if profiles are not used.
For example, content can be required to conform to one of the profiles, or it may be conformant to the specification independently of conformance to one of the profiles. The questions arises also for a producer (of content): is it conforming if it generates content that is otherwise valid but does not conform to a profile?
An example of additional conditions on profiles would be to require that only one profile can be implemented at a time.
Note: If there is a "full" profile defined — for example, incorporating all of the defined functionality of the specification, including extensibility features — then any valid content, as well as any correct producers and fully capable consumers, might seem to be automatically using that profile. However, profiles (e.g., of content) often include self-identification requirements, and these would have to be observed for conformance of valid content to even a "full" profile.
The specification MUST define for each profile the minimal required features/support for each identified class of product. This checkpoint is not applicable if profiles are not used.
Because a profile places explicit requirements on each class of product that have specific and often limited operating environments, these requirements must be defined for each class of product that is affected.
To illustrate, SMIL 2.0 [SMIL20] has a SMIL 2.0 Language Profile for user agents but also provides a SMIL 2.0 Basic Profile for wireless and embedded devices. The SMIL 2.0 Language Profile requires that a user agent implement the BasicAnimation module but not the SplineAnimation Module. The SMIL 2.0 Basic Profile on the other hand does not require implementation of any of the Animation modules.
If the specification allows the creation of derived profiles, the specification MUST provide requirements for derived profiles and these requirements MUST be testable.
Well-designed rules for profiles will facilitate the creation of derived profiles that are well-specified and testability will enable an independent tester to verify conformance of a derived profile to the rules.
Note that previous experience in W3C (e.g., the creation of WebCGM) and by other organizations who have created derived profiles or defined rules for profiles, shows that requirements for derived profiles should impose at least two rules on derived profiles: (1) derived profiles should be specified in a way that meets all the pertinent checkpoints of this document (QA Framework: Specification Guidelines); and, (2) derived profiles should not contradict predefined profiles, if there are any in the base specification.
The specification MUST document any identified mandatory conditions or constraints on the usage of modules. Such conditions include,
This checkpoint is not applicable if modules are not used.
The conditions or constraints normally will be tailored according to class of product.
The specification MUST address and discuss any identified relationships and interaction between profiles, modules, levels and with other dimensions of variability. This checkpoint is not applicable if none of the three DoV — profile, modules, and levels — are used.
Dependency or interrelationship between profiles and modules is common -- XHTML [XHTML-MOD], SMIL [SMIL20], SVG 1.1 [SVG11].
Often there is dependency or interrelationship among modules, on the one hand, and profiles or discretionary choices on the other. Modules may have levels or deprecated features. Extensions could be defined based on modules.
Levels can be dependent on, or apply to, modules. Less often, deprecated features, levels, discretionary choices, or extensions could depend on profiles.
Be aware that excessive variability harms interoperability.
A deprecated feature is an existing feature that has been outdated by newer constructs or is no longer viable. Deprecated features should be avoided (e.g. for a format language, not be used by the authors and authoring tools) and may be removed in some future version, at which time the feature becomes obsolete.
After the initial publication of a specification, specification developers may consider the deprecation of a feature (e.g., function argument, element or attribute) defined in the specification. Deprecated features may become obsolete and no longer defined in future versions of the specification. Deprecation of a feature may warn implementers that the feature was a bad idea and it may be withdrawn in the future. Specification developers need to consider the effect of deprecation on all the classes of products that implement the specification (e.g., authoring tools, user agents) as well as the conformance consequences on each class of product. For the purpose of backward compatibility, it may be necessary to specify different requirements for the support of deprecated features for each class of product. For example, authoring tools (producers) would not use the feature, but user agents (consumers) would continue to support it.
Deprecation is a dimension of variability (DoV).
The specification MUST document in a normative section each deprecated feature. This checkpoint is not applicable if there are no deprecated features.
It helps the reader find the deprecated features by presenting them as a collection in one place rather than distributed throughout the document. This collection may be derived from distributed deprecated features and may take the form of a list of links to where the features appear in the document.
The specification MUST specify the degree of support required for each deprecated feature for each class of product and the conformance consequences of the deprecation. This checkpoint is not applicable if there areno deprecated features.
For example, a deprecated-features section of MathML 2.0 ([MATHML20], section 7.2.1.2) describes, about deprecated MathML 1.x features, that MathML-output-compliant authoring tools may not generate MathML markup containing deprecated features; whereas MathML-input-compliant rendering/reading tools must support deprecated features.
The specification MUST indicate, for each deprecated feature, either (1) that it is independent of all other dimensions of variability or (2) discuss the relationship between the feature and each of the other DoV.
Depending of the breadth of the feature being deprecated, it may impact other DoV. The effect may be contained (e.g., within a single class of product or module), be wide ranging (e.g., across all classes of products), be a dimension of variability (e.g., deprecate an entire module), or introduce changes to a dimension of variability (e.g., spawn new discretionary choices).
Be aware that excessive variability harms interoperability.
The specification MUST document each deprecated feature with a rationale for the deprecation. This checkpoint is not applicable if there are no deprecated features.
Providing the rationale for deprecating a feature helps implementers and users to understand the motivation for the deprecation, the impact and consequences on current and future implementations, and perhaps insight into its eventual disappearance from the specification.
The specification MUST provide an example for each deprecated feature showing how to avoid using it. This checkpoint is not applicable if there are no deprecated features. This checkpoint is only applicable to specifications that identify "producer of content" as one of its classes of product.
Examples are helpful in providing alternatives or better ways to get the same results. Showing what can be done in place of the deprecated feature will help get implementers to discontinue use of the deprecated feature.
Note that this checkpoint only makes sense for specifications that define the behavior of a producer.
The specification MUST document each obsolete feature. This checkpoint is not applicable if there are no obsolete features.
Obsolete features are listed for historical purposes. There is no guarantee of support for obsolete featrues by implementations of the specification.
Obsolete features can be listed in the Change section of a specification, e.g., HTML 4.01.
Discretionary items are defined as deliberate and explicit grants of discretion by the specification to the implementations, that describe or allow optionality of behavior, functionality, parameter values, error handling, etc.
Discretionary items are often made available in specifications, to give implementers of the technology the opportunity to decide from alternatives when building applications and tools. Discretionary items may be considered necessary because of environmental conditions (e.g., hardware limitations or software configuration, or external systems), locality (e.g., time zone or language), optional choices providing flexibility of implementation, dependence on other specifications, etc.
Discretionary items come in three basic variants:
Discretionary items are a dimension of variability (DoV).
The specification MUST indicate the rationale for the discretionary items and MUST identify by labeling all discretionary items. This checkpoint is not applicable for specifications that do not have discretionary items.
Test creators will want to discover all choices and variability allowed to implementers. Having a name for each discretionary item, and a link target for each one, allows systematic reference to individual items from test cases as well as elsewhere in the specifications.
Note that references to use cases and project requirements are usually a good way to justify discretionary items.
The specification MUST describe any permitted variations or constraints for how the implementation dependent value or feature is realized by implementations.
Examples of permitted variations or constraints to be addressed include:
The specification MUST indicate, for each identified discretionary item, whether zero, exactly one, or several of the choices/options are allowed to be implemented. If the allowable number is dependent on other dimensions of variability, the dependencies MUST be stated. This checkpoint is not applicable for specifications that do not use discretionary items.
A test framework for the specification could provide a tailoring mechanism so that each implementation is tested with a set of tests tailored to the choices made by the implementer. The tailoring mechanism for choose-exactly-one items would not be the same as the tailoring mechanism for choose-one-or-more items. The tailoring mechanism would need to accommodate modules, profiles, and levels, when they are used, which may affect the range of choices on some discretionary items (and may drop the range to zero in some situations).
Examples of constraints include mandating that an implementation implement only one choice, implement every choice (allowing the user to choose), be allowed to implement none of the choices, or be required to implement one specified value and as many additional values as they wish from an open-ended list. An example of a constraint being dependent on another dimension of variability is a rule that there must be exactly one instance of the item for each profile that is supported.
It is possible for a discretionary item to affect the range of available options on another item, or to render it moot. For example, if a discretionary choice addresses serialization of characters as entities, one option of that choice may render moot a grant of discretion regarding entity names for character entities.
The specification MUST provide rules for consistent terminology about discretionary items and MUST provide a rationale for discretionary items that do not follow the rules.
This helps to ensure that implementations can consistently handle discretionary choice. Users have an expectation of what to expect and should be able to count on getting the same results under the same conditions with a given implementation. It also helps to propagate the rules about discretionary items onto the implementations, especially when an implementation could offer choices to the user.
Names of discretionary items, and names of choices if used, should use verbiage consistently in order to satisfy Checkpoint 7.3. Identical words should be used for identical behaviors.
Note that the Working Group believes that given identical conditions, the effect of a discretionary choice should be consistent within a single implementation, and thus, that specifications should enforce it.
Specifications should encourage the implementation and any associated documentation to use the specification's terminology when discretionary choices are visible to the user.
The specification MUST address and discuss the relations and interactions between discretionary items and all the other dimensions of variability.
Where multiple discretionary items can be related to each other, the specification MUST justify making them independent of each other. In other words, when individual details are subject to discretion but can be summarized as a policy-level decision, the specification MUST present one named discretionary item for the policy OR present a justification for dividing the item into smaller items. This checkpoint is not applicable for specifications that do not have discretionary items.
The use cases begin the process of setting expectations of how implementations will be used. When further refined, the use cases set expectations for variation or flexibility of implementations. Any variability that is too detailed for the use cases is probably of low value to the user, but it hurts interoperability.
Be aware that excessive variability harms interoperability.
An extension to a specification is a mechanism to incorporate functionality beyond what is defined in the specification. Extensions can be implemented individually (e.g., one function at a time) or by defining new profiles, modules, or levels that incorporate additional functionality beyond what is defined in the specification.
Allowing extensions affects how conformance is defined as well as what conformance claims can be made. Exercise caution in determining the extent to which extensions are allowed or not allowed. Since extensions can seriously compromise interoperability, specification writers should carefully consider the degree to which extensions are be allowed.
Disallowing extensions for any part of the specification enables strict conformance: strict conformance is defined as conformance of an implementation that employs only the requirements and/or functionality defined in the specification and no more (i.e., no extensions to the specification are implemented). Dimensions of variability (e.g., modules, profiles, levels) are not extensions if the specification defines them or allows them to be defined.
Extensions may be private (often vendor specific) or public (a full description of the extension is public). Private extensions are usually truly private, i.e., valid for a specific implementation or are only known by prior agreement between implementations. Public extensions are extensions in which the syntax, semantics, identifiers, etc. are defined and published allowing anyone to implement the extended functionality.
Specifications allow extensions for various reasons. Extensions allow implementers to include features that they consider to be required by their customers. Also, extensions often define new features that may migrate into future versions of the specifications. However, the use of extensions can have a severe negative impact on interoperability. Some methods for enabling extension have less impact on interoperability than other methods.
Extensions are a dimension of variability (DoV).
The specification MUST state the conditions under which extensions are allowed and disallowed, and if allowed, MUST state:
Readers should be able to understand the motivation for the inclusion of an extension and its intended use. Documenting the scope and rationale for extensions helps assess the impact of extensions on interoperability.
If the specification writer wants to enhance interoperability by constraining implementer extensions, wording in the specification must indicate this.
If extensions are not allowed, then it should be clear to the reader that not only are extensions not allowed, but the circumstances under which they are not allowed. Strict conformance is often imposed on applications or content (e.g., a software program or document instance). This prohibition of extensions could be applied to a specific profile or module, rather than to the entire specification.
The specification MUST state that extensions cannot negate or change support for required functionality. This checkpoint is not applicable if extensions are not allowed.
Requirements in the specification cannot be compromised or contradicted by adding extensions.
The specification MUST provide a uniform way to define that extensibility is being invoked and MUST provide the syntax to be used to indicate the extension. This checkpoint is not applicable if extensions are not allowed.
This helps to ensure predictable handling of extensions, that is, its recognition as such and the appropriate actions to take (i.e., to ignore or to implement).
The specification MUST require that the syntax and semantics of the extension be publicly documented. This checkpoint is not applicable if extensions are not allowed.
Public availability with a full description allows the extension to be implemented by anyone without prior arrangement. This enhances interoperability by allowing the same functionality to be uniformly implemented across different implementations.
The specification MUST define a policy about implementation requirements for mitigation of the interoperability impacts of extensions. This checkpoint is not applicable if extensions are not allowed. This checkpoint is only applicable to specifications that identify "producer of content" as one of its classes of products.
Extensions can have a negative affect on implementation interoperability. This checkpoint can be used to impose conformance requirements on producer implementations to minimize the consequences of the extension. For example, the specification could include a requirement that implementations have a no-extensions mode; or could include a requirement that implementations include equivalent alternative (standard) content with any extensions; or could explicitly state that there are in fact no implementation requirements for mitigation of interoperability impacts of extensions. This checkpoint can be used to ensure that there is a way to produce (generate) only conforming content.
The specification MUST address and discuss the relations and interactions between extensions and all the other dimensions of variability.
For example, a specification could define a strict conformance policy for one class of product, while allowing extensions for another class.
Be aware that excessive variability harms interoperability.
The checkpoints of this guideline aim to ensure that normative content and conformance requirements are easily identifiable in specifications. Some of the desirable characteristics of conformance requirements are that they are mutually independent from other requirements, express a minimal requirement (i.e., are atomic), and are distinguishable and labeled (e.g., tagged). It is important that specifications provide unambiguous statements, that clearly define what is required in order to claim conformance and what is optional. It is important also that the conformance statements are easily identifiable as such, and easily locatable. Clarity is fostered by uniformity of structure and style, and consistency of terminology and phraseology.
There is a lot to be said for consistency and clarity within a document - it facilitates the understanding and comprehension of the document. Authors and editors of specifications should already be familiar with the Publication Rules (Member-only) [PUBRULES] and W3C Manual of Style [STYLE-MAN], which help to achieve clarity and uniformity. This guideline builds on those general document conventions, with a particular focus on the presentation and identification of conformance information.
The specification:
Using the RFC 2119 keywords helps to identify the testable statements in a specification as well as those statements that are desirable or allowed. This specification does not place any requirements on the formatting or rendering of these keywords.Guidance on the proper usage of these keywords is given in RFC 2119.
If for some reason, the RFC 2119 keywords cannot be used in the specification, it is important that the reader can identify the conformance requirements explicitly and unambiguously.
Normative content is the prescriptive part of the specification whereas informative content is for informational purposes and assists in the understanding or use of the specification.
The specification MUST distinguish normative text from informative content.
It is important that the reader be able to distinguish between normative and informative contents in order to know what is required to claim conformance. For example, Synchronized Multimedia Integration Language (SMIL 2.0) [SMIL20] indicates within every subsection whether the section is normative or informative, and even separately labels pieces of subsections that contain both kinds of content.
Content includes not only text, but also examples, illustrations, and use cases.
The specification MUST use identical wording to express identical provisions, and analogous wording to express analogous provisions.
Being consistent and following established conventions and W3C editorial practices will enhance the readability and comprehensibility of the specification.
The specification MUST provide at least one navigation mechanism that allows the reader to locate by direct access, all conformance-related information that is relevant to the specification. The mechanism MUST minimally locate:
A reader must be able to easily identify and locate all the information necessary to understand the conformance policy and related conformance information without having to read the document from cover to cover. A navigation mechanism adds to the usability of the specification.
A table of contents entry is one way to accomplish this. In addition to the minimal required set above, other conformance related information such as the ICS, location of test suites, etc., may be helpful to users and implementers.
Note that having a navigation mechanism available to hardcopy rendered specifications can be very useful for test suite developers and users.
A specification depends on another specification when it relies on or requires functionality (or behavior) from the other specification in order to work (function) correctly. This other specification provides a necessary condition or functionality.
The specification MUST have normative references to any identified specification it depends on and MUST describe the relationship between the specifications and any identified conformance implications.
Dependence on other specifications affects the conformance boundaries of the current specification, and thereby affects the requirements on conformant products.
The linking parts of the Manual of Style ([STYLE-MAN], section 11.5.1) describe the recommended way of citing an external reference from the prose of a specification as well as how to construct an entry in its References section.
For example, SVG 1.0 requires that the class of product called "user agent" be consistent with the XML 1.0 Recommendation [XML10] and (conditionally) support Cascading Style Sheets, level 2 [CSS2].
A look at various W3C Technical Reports shows that the term "conformance" is often qualified, resulting in more than one type of conformance. It is important to convey an understanding of what is meant by conformance and how it applies to each class of product as well as each dimension of variability (e.g., modules) if applicable. For example, if the specification defines behavior for more than one class of product, there may be a separate conformance policy for each class. Similarly, if the specification defines modules, there may be a different conformance policy for each module.
In particular, a reader intending to implement a product in one of the product classes addressed by the specification should know what the WG wants for interoperability among products in the class. The developer should understand what forms, if any, of "product differentiation" are allowed among conformant products. The specification may need to explain how the rules apply and provide examples.
A conformance clause is a part or collection of parts of a specification that defines the requirements, criteria, or conditions to be satisfied by an implementation or application in order to claim conformance. Typically the conformance clause is a high-level description of what is required of implementations and applications.
The specification MUST include a normative section enumerating the minimal requirements that apply across conforming products of a class.
The reader must be able to recognize any minimum functionality or support that applies to conforming products of a specific class. These minimum requirements can be considered a starting point for a checklist to be used by a team developing a conforming product. It helps the reader find these requirements by presenting them as a collection, in one place.
Usually, the individual requirements comprising the minima all occur elsewhere, as conformance requirements distributed throughout the document. In principle, the collection could therefore be derived from the distributed requirements, although such a derivation could be difficult. The collection of minimal requirements may apply to a single class of product or across classes of products.
If levels are used (see Guideline 3), the lowest level may represent the minimum set of requirements. If profiles are used (see Guideline 3), there may be different minima for each profile. If modules are used (see Guideline 3), there may be different minima for each module. Furthermore, a module may itself be a minimum (i.e., required) for a particular class of product.
Any conformance concepts used in the specification MUST be defined, either by reference or by including the definition in the text.
It is necessary to define concepts that govern application of the conformance provisions. Ideally, all concepts are from QA documents and other existing literature and need only be cited. If special concepts are constructed, such as to combine modules and levels or modules and discretionary choices, they need to be defined in the specification.
A specification MUST justify each dimension of variability the specification uses.
Dimensions of variabilityadd complexity to a specification. Explaining the usage of each dimension of variability can help implement and test them better and can help reviewers understand their necessity.
References to usage scenarios and requirements are usually a good way to justify the uses of dimensions of variability.
The specification MUST document its conformance policy in a dedicated section of the document.
Having the conformance clause exist as a separate section within the specification makes it clearly identifiable, allowing a reader to find the overall conformance policy from a single starting point.
The specification MUST identify and define all the conformance designations used.
In current W3C practice, a number of different naming conventions are used to label conformance, in cases where there is not a single, monolithic (strict) conformance definition. The naming convention used to label the conformance can provide useful information. Degrees, for example, implies incremental importance or difficulty. This Specification Guidelines document uses "degrees" for example, to refer to three successively more demanding degrees of conformance (A, AA, AAA).
Commonly used conformance designations include categories, degrees, and levels. Use of "conformance levels" is discouraged in new specifications, because of the potential for confusion with "functional levels".
A specification may differentiate conformance claims by designating different degrees or types of conformance in order to apply and group requirements according to modules, profiles, levels, or priorities. When a conformance claim is linked to functionality, impact and/or incremental degrees of implementation, the term 'conformance level' is often used to indicate the varying degrees of conformance. The use of this term, conformance level, is discouraged and should be replaced by the term, 'degree of conformance'. The WG includes in the specification the way they want people to claim their conformance.
An Implementation Conformance Statement (ICS) provides a mechanism whereby a supplier of an implementation of the specification provides information about the implementation in a standardized manner. It is used to indicate which capabilities and options have been implemented, as well as the limitations of the implementation. An ICS usually takes the form of a questionnaire where product implementors report on the conformance of their product to the named specification.
An ICS is useful in disclosing optional functionality and discretionary behavior and values. The results of the ICS can be used to identify the subset of test cases from a conformance test suite that are applicable to the implementation to be tested. This will allow the implementation to be tested for conformance against only the relevant requirements.
The basic and detailed information that an ICS provides can also be used to assess and deduce the interoperability potential of two or more products.
The specification MUST provide specific wording of the claim and the specific wording MUST include at least:
The specification MUST provide a conformance disclaimer.
Although it is possible to prove with certainty when something does not conform, the reverse is not necessarily true. Especially for functional specifications, where a claim goes beyond syntax testing, a claim of conformance is not a guarantee that the claimant is 100% conforming with the specification. A disclaimer can help clarify the meaning of a conformance claim as well as its limitations. For example, this document contains a conformance disclaimer.
The specification MUST NOT include any restriction about who can make a claim nor where claims can be published.
Claimants (or relevant assuring parties) are solely responsible for the validity of their claims, keeping claims up to date, and proper use of any conformance icons.
The specification MUST either:
The inclusion of an ICS may not be applicable to all specifications. A specification's ICS can be thought of as providing a minimal list of items supported by the implementation, capturing information the WG deems necessary to support conformance claims.
An ICS that is part of a specification does not preclude other organizations such as certification organizations, from having their own ICS.
The specification MUST include the ICS as part of its conformance claim requirements. This checkpoint is not applicable, if an ICS is not applicable to the specification.
The ICS is coupled with the requirements for making a conformance claim (checkpoint 9.1), thus providing specific information about the implementation and substantiating the conformance claim.
A test assertion is a statement of behavior, action, or condition that can be measured or tested. It is derived from the specification's requirements and provides a normative foundation from which test cases can be built.. Each test assertion is an independent, complete, testable statement for requirements in the specification. Each test assertion results in one or more test cases. Multiple test assertions can be combined to form a test case, in this case one tests multiple facets of a particular behavior. It is recommended that test assertions be available by the time a specification enters Proposed Recommendation.
The specification MUST provide a normative list of test assertions.
Providing test assertions facilitates and promotes the development of test materials. Tests can point directly to the test assertion, which in turn are grounded in the specification. Specific benefits include:
The list of test assersions can be included in the specification or in a separate document that is referenced by the specification. Test assertions may be developed by the specification authors, others members of the WG, or people external to the WG.
The specification MUST provide a mechanism linking each test assertion to the part of the specification from which it was derived.
This ensures consistency between the specification and the test assertions list and facilitates the building of a test suite based on the test assertions list. In order to enable traceability from the tests back to the specification, use a mechanism to make the test assertions identifiable (i.e., labeled or tagged) in the specification.