This is the schema document for the July 2010 version of the W3C XML Schema Test Suite collection (XSTS), as approved by the W3C XML Schema Working Group on May 12, 2003 and as modified since. It defines the namespace:
http://www.w3.org/XML/2004/xml-schema-test-suite/
This version is makes a number of changes, vis a vis the version of April 2010. That version in turn differs from the version originally approved by the WG in May 2003. For a list of changes in this version, including one important change planned but not yet executed, see the changelist below.
The XSTS consists of a set of test suites, each of which is identified according to the versions of XSD it is designed to test. Previous versions of test suites are archived and are also available, identifiable by the version of the test suite itself.
This schema defines three separate but related document types for the XSTS:
The testSuite element is the root element of a document which defines a test suite as described above (e.g. the 2003-10-25 version of the test suite for version 1 of the Recommendation, or the 2010-08-31 version of the test suite for XSD 1.0 First Edition, XSD 1.0 Second Edition, and XSD 1.1).
The principal purpose of a testSuite document is to provide a high-level description of the test suite and a set of links to documents defining the tests which constitute the test suite.
Files containing testSuite documents conventionally have the filename suffix ".suite" or ".suite.xml" (or are named "suite.xml".
The testSet element is the root element of a document which describes a set of tests. Each testSuite consists primarily of a set of links to testSet documents. The scope of the testSet is typically determined by the contributor of the tests which make up the testSet: it is the unit in which tests are contributed to the collection.
Files containing testSet documents conventionally have the filename suffix ".testSet" or ".testSet.xml".
The testSuiteResults element is the root element of a document describing the results of testing a processor against a testSuite.
Files containing testSuiteResults documents conventionally have the filename suffix ".results" or ".results.xml".
The root element of a document describing a set of tests for one or more versions of W3C XML Schema.
The element has three attributes, each of which is required:
name - the name of this test suite.
releaseDate - the date on which this test suite was released. This value serves to identify the version of the test suite.
schemaVersion - the versions of XSD for which the tests are designed. This has documentary function only, and is intended for human readers. The machine-processable version information is handled by the version attribute.
version - a list of version tokens indicating versions and features for which at least some tests in the test suite are applicable.
Any processor or processor configuration which supports any of the tokens given should run the tests. Processors which support none of the named features can skip the entire test suite without loss. If no version value is given, or if the value is the empty string, all processors should run the tests.
For example version="1.1" on a test suite
element indicates that XSD 1.1 processors will find
relevant tests, and XSD 1.0 processors will not,
while version="1.0 1.1", or no
version attribute at all, indicates
that the test suite contains tests relevant to both.
Logically, the version attribute on the testSuite element, if given explicitly, should include all the tokens used on any testSet, testGroup, schemaTest, or instanceTest in the test suite, and no others. This is not necessarily enforced, however, by the schema for this vocabulary.
Two child elements may optionally be present:
The root element of a document describing a set of tests, normally from a single contributor. A contributor may supply any number of testSet files.
Note: In order to make it possible to browse the test suite in a browser, it is helpful if large test collections are broken up into several testSet documents of no more than a megabyte or so each. If contributions have larger testSet documents, they may be broken up into smaller ones.
The element has two attributes:
contributor (required) - the name of the contributor of this testSet. May contain any string of characters; intended for human readers.
name (required) - the name of this testSet, which must be a name unique among the names of testSet elements within the enclosing testSuite.
version (optional) - a list of version tokens indicating versions and features for which at least some tests in the test set are applicable.
Any processor or processor configuration which supports any of the tokens given should run the tests. Processors which support none of the named features can skip the entire test set without loss. If no version value is given, or if the value is the empty string, all processors should run the tests.
Logically, the tokens given in the version attribute should all also be included in the version attribute [if any] of any testSuite including this test set. And similarly the version attribute on a testSet element should include all the tokens used on any testGroup, schemaTest, or instanceTest in the test set, and no others. Otherwise processors may skip test sets they ought to run. This logical rule is not necessarily enforced, however, by the schema for this vocabulary.
Two child elements may optionally be present:
No two testGroup elements in the same testSet may have the same name.
This element groups a collection of closely related tests. All instance tests in the group are to be validated against the same schema; if a schemaTest is present, it is the schema produced for that test which should be used for the instance tests; if no schemaTest is present, the instance tests should be validated against a schema consisting only of the built-in components.
The testGroup element has one attribute which is required:
And one attribute which is optional:
version - a list of version tokens, indicating that the tests in the group are applicable to implementations supporting any of the versions or features or behaviors indicated. Any processor or processor configuration which supports any of the features indicated should run the tests. Processors which support none of them can skip the entire test set. See the definition of the version-info type.
Logically, all keywords appearing here should also appear in the version attribute of the enclosing testSet, if it has one.
Four child elements may optionally be present:
annotation - zero or more instances of general documentation.
documentationReference - any number of references to external documentation upon which the test is based, e.g. links to relevant sections of the Recommendation, to the Errata, etc.
schemaTest - at most on schemaTest element, containing any number of schemaDocument elements, each of which holds information on a single schema document.
When more than one schema document is present, a single schema is constructed from the set (or from other schemas via import).
Note: XSD's rules for schema composition mean that the order in which schema documents are encountered may be significant. When more than one schema document is listed in the schemaTest element, the test should be run as if the schema documents given were loaded one by one, in order. For most processors that will correspond to the result of processing an otherwise empty schema document for an otherwise unused namespace, containing one xsd:import element for each schema document listed in the schemaTest, with the location indicated, in a processing mode that involves following the schema-location hints in import statements.
Note: the working group has made no decision on whether the schema should be constructed solely from the schema documents listed in the schemaTest element, or from those schema documents plus the transitive closure of their references to other schema documents. Similarly, the working group has not decided whether schemaLocation hints in the instance tests should be honored or not. It is therefore advisable to draft test cases without dependencies on schemaLocation hints and the like.
Note: work is pending on these issues of schema composition. When it's complete, this part o the test suite schema may be expected to change.
Schema documents may be omitted, for the purpose of testing a processor's validation of an instance containing only the built-in datatypes defined in the Recommendation.
instanceTest - any number of elements, each of which holds information on a single instance document to be validated against the included schema.
No two tests within a test group may have the same name.
This element groups together information about the schema for a particular test group.
It has one attribute which is required:
and one attribute which is optional, for identifying a subset of versions and/or editions for which the test is valid:
version - Tests which only apply to certain versions of XML Schema list those versions in the version attribute. Processors supporting any version or feature indicated by a keyword in the attribute should run the test. (Or, phrased more declaratively: the test is meaningful to any processor which supports any of the features or versions listed.)
If no value is specified, all processors which haven't already skipped the enclosing test group, test set, or test suite should run the test.
The value is a list of version tokens. See the definition of the version-info type.
Note that the omission of a version token on a schema test is in some sense strictly advisory: any schema test is meaningful for any processor in any configuration. For processor configurations not supporting any of the features or versions named, the expected result is that the schema is not a conforming schema. This will not be indicated with an explicit expected element.
One child element is required:
Four child elements may optionally be present:
The elements "expected" and "current" may be absent when tests are contributed, but will always be present for tests included in the XSTS.
The current and prior elements were originally designed for tracking changes of status in tests; they can and should be used to keep a general change history of the test. Whenever anything changes that may be of importance for users of the test suite, it is appropriate to clone the existing current element into a pair of similar elements, then rename the second one prior. In the new current element, the change made should be described in the annotation children, and the date of the change should be recorded.
Examples: The status of the test changes. The expected result is questions and reaffirmed. The expected result is changed, or multiple expected results are given for different processor configurations.
For status changes involving bug reports, the relevant status entries should have a Bugzilla cross-reference.
This element groups together information about an instance document which should be validated against the schema referenced in the enclosing testGroup.
Note: per section 5.2 "Assessing Schema-Validity" of the Recommendation "XML Schema Part 1: Structures", validation may be started in a variety of ways. For the purposes of the XSTS, only the third method shall be used:
The processor starts from Schema-Validity Assessment (Element) (3.3.4) with no stipulated declaration or definition.
The validation root is the outermost element in the instance document.
The instanceTest element has one required attribute:
and one attribute which is optional, for signaling that the test is applicable only to a particular set of versions of XSD:
version - Tests which only apply to certain versions of XML Schema list those versions in the version attribute.
Processors supporting any version or feature indicated by a keyword in the attribute should run the test. (Or, more declaratively: the test is meaningful to any processor which supports any of the features or versions listed.) If no value is specified, all processors which haven't already skipped the enclosing test group, test set, or test suite should run the test.
The value is a list of version tokens. See the definition of the version-info type.
Note: running instance tests with a processor for an inapplicable version may produce an failure owing to non-conformant constructs in the schema document; if the processor does not detect the problem or continues anyway, the results are certain to be meaningless.
One child element is required:
Four child elements may optionally be present:
The elements "expected" and "current" may be absent when tests are contributed, but will always be present for tests included in the XSTS.
The current and prior elements should be used to keep a change history of the test; see discussion under the schemaTest element.
The current status of a test in the XSTS.
This element has two attributes, both of which are required:
A former status of a test in the XSTS.
This element has two attributes, both of which are required:
The validation outcome prescribed by the spec for a test in the XSTS.
This element has one optional attribute:
version - a list of version tokens. The result specified is applicable to processor configurations supporting all of the indicated versions or features of XSD. See the definition of the version-info type.
It is an error for more than one expected element to be applicable to any given processor configuration; this is most easily avoided by making sure that any two sibling expected elements have version attributes containing mutually exclusive tokens.
Note: The meaning of the version attribute on this element differs from its meaning elsewhere.
On tests and elements for groups of
tests (testGroup etc.), a version
attribute of the form version="x y
z" means "If any of
x, y, or z are supported, tests
in this group are applicable."
On the expected element, the
meaning changes in a crucial way: the tokens are connected
with an implicit and, not an or. So
version="x y z" means
"If all of x, y, or
z are supported, the prescribed outcome is as
described. So on a test group, version="1.0
1.1" means tests for both versions are included.
On an expected element, version="1.0
1.1" would mean the expected result holds only if a
given processor is using both version 1.0 and version 1.1
in the same validation episode. Since the two tokens are
defined as mutually exclusive, this would be a
contradiction.
As a matter of test suite design, it is a good idea to keep version attributes on expected elements to a single token if possible, to minimize opportunities for confusion.
And one required attribute:
validity - indicates the expected outcome of the test, using a value of type ts:expected-outcome.
For an instance test, this typically indicates the expected
value of the [validity] property on the
root element of the instance document, or indicates
that the value may vary among processors.
For a schema test, this indicates whether the
schema created from the schema documents in the test
is expected to be a conforming schema (valid)
or a non-conforming schema (invalid).
The value notKnown has no meaning
for a schema test.
Enumerates the possible outcomes of running a test. Usually, these are values of the [validity] property on the validation root.
The most common values are:
Note: processors built as instance validators are not required by XSD to distinguish between invalid documents and documents with unknown validity; it is thus not an absolute requirement (although it is desirable for clarity) that a test result distinguish invalid from notKnown outcomes.
One further value is needed only in fairly specialized circumstances (but is essential there):
(For instance tests) The instance has a schema with a latent error (see description below in the documentation for type ts:expected-outcome); the processor did not detect the latent error on the corresponding schema test, but the instance document has exposed the error (by including content which is valid against the apparent content model of the governing type, but not valid against the base type) and the processor has detected the schema error in the course of instance validation.
Note: processors are encouraged, though not required, to distinguish this outcome from invalid, since on an instance test invalid normally means that the processor has found an invalid instance, not a non-conforming schema.
(For schema tests) The value runtime-schema-error is meaningless for schema tests and should not be used for them. (It would be a contradiction in terms.)
Enumerates the possible values for the prescribed outcome of a test. Values include both (a) the possible values of type ts:test-outcome and the following additional values:
Note: in most cases of implementation-defined behaviors, as a matter of test suite design it is better to analyse the set of possible implementation behaviors, define version tokens for the possible behaviors, and specify more informative results based on those tokens. The value implementation-defined is provided for situations where this is not feasible for whatever reason.
N.B. the values implementation-dependent and implementation-defined should be used only when the spec is explicit about the implementation-dependence of the result and it is thus clear that the implementation-dependence is a design choice consciously made by the working group. They must not be used in cases where the spec simply appeals to some concept which it turns out not to define: such cases are to be marked indeterminate.
Note: in most cases, as a matter of language design it is better for the language specification to prescribe clearly a particular result for a test, or to identify the result explicitly as implementation-defined or implementation-dependent. The value indeterminate is provided for situations where this has not been done for whatever reason.
The value invalid-latent described in earlier drafts of this schema document is no longer needed; the version keywords for complex-type restriction behaviors can be used to describe the relevant cases more precisely.
This is the root element of a document containing a test result report. The report takes the form of a set of test results returned by a processor/validator when run against the XSTS.
It has three required attributes:
The element also has one optional attribute:
This element has two optional elements:
The result of an individual instance test or a schema test.
This element has four required attributes:
NOTE: The "set", "group" and "test" attributes are used to uniquely identify the test within the XSTS for which this result reports the validation outcome. Each matches the "name" attribute of the respective element in the test suite.
This element has one optional attribute:
The element has one optional element:
A link to documentation relevant to a test, such as a link to the Recommendation, an erratum, an archived email discussion, etc.
Value type for the version attribute.
The value is a list of version tokens indicating the versions and features for which the test is applicable and/or has the expected result indicated.
Note that on testSuite, testSet, testGroup, schemaTest, and instanceTest, the tokens have an implicit or connecting them: if a processor configuration supports any of them, the tests included are applicable. On the expected element, by contrast, the implicit connector is an and: if a processor configuration supports all of them, the result given is prescribed.
It will reduce confusion if wherever feasible expected elements use version attributes with single tokens. The following is legal:
...
...
]]>
But since only 1.0 processors will be processing this test group anyway, and only 1.0 processors will support either the 1.0-1e or the 1.0-2e feature, the following may be clearer:
...
...
]]>
There may, however, be cases in which it is logically necessary to provide multiple tokens.
Tokens to denote single versions, features, or implementation-defined behaviors, of XSD.
It is expected that users of the test suite will use the version tokens associated with a test to determine whether there is any point in running the test with a given processor (or with a particular configuration of the processor); version tokens are also used to determine which expected result applies to a given processor or a given configuration. See the documentation for the testSuite element for more details.
Ideally, only instances of the known-token type should be used. Decimal and NMTOKEN literals may be used if necessary; this allows new features or behaviors to be described and given keywords without waiting to update this schema document. To document the tokens, however, it is desirable to add all new tokens here once they are accepted by the working group.
Tokens to denote well-known (i.e. documented) versions, features, or implementation-defined behaviors, of XSD.
The known-token type is a union of several other types, each with an enumeration of values. Each sub-type defines keywords for a set of mutually exclusive versions, features, or behaviors, such that in any given schema validation episode, at most one keyword in any subtype will apply. For examples, see the various subtypes defined immediately below.
Tokens to denote specific known versions of XSD.
Each token denotes the version of the XSD language identified by the ts:standard-version-id attribute on the xsd:enumeration element. That is, "1.0" denotes XSD 1.0 (without reference to a particular edition), and "1.1" denotes XSD 1.1 (without referece to a particular edition).
An attribute for use in schema documents, on enumerations, to indicate the correspondence between the token being enumerated and the standard identifier for a particular XSD language version.
The values should be taken from the tabulation of standard language identifiers in appendix K of the XSD 1.1 specification.
Tokens to denote specific editions of XSD 1.0.
Each token denotes the version of the XSD language identified by the ts:standard-version-id attribute on the xsd:enumeration element. That is, "1.0-1e" and "1.0-2e" represent 1.0 First Edition and 1.0 Second Edition, respectively.
Outside the context of XSD 1.0, these edition identifiers have no meaning or applicability.
Tokens to denote different versions of XML-dependent datatypes. Conforming XSD 1.1 processors may support XML 1.0-based datatypes, XML 1.1-based datatypes, or both. There is dispute in the working group over whether conforming XSD 1.0 processors are allowed to suport XML 1.1-based datatypes or not.
The value "XML-1.0" denotes processor support for, or test applicability to, XSD datatypes based on XML 1.0, without specifying a particular edition. (This value is retained for backward compatibility of this schema, but it should be avoided unless there is no difference, for a given test or test result, between editions 1-4 and edition 5 of XML 1.0. Where there is a difference, the values "XML-1.0-1e-4e" and "XML-1.0-5e" should be used in preference. (XSD 1.1 describes XML 1.0 Fifth Edition as the base version in its normative reference, so in theory the distinction between "XML-1.0-1e-4e" and "XML-1.0-5e" is only relevant to XSD 1.0 processors. In practice, it may also be relevant for some XSD 1.1 processors.
The value "XML-1.0-1e-4e" denotes processor support for, or test applicability to, XSD datatypes based on XML 1.0 First Edition through Fourth Edition.
The value "XML-1.0-5e" denotes processor support for, or test applicability to, XSD datatypes based on XML 1.0 Fifth Edition.
The value "XML-1.1" denotes processor support for, or test applicability to, XSD datatypes based on XML 1.1 (for which at the moment there is only one edition).
In most cases, of course, "XML-1.0-5e" and "XML-1.1" will describe the same behaviors.
Tokens to denote specific known versions of Unicode.
Each token denotes the version of the Unicode specification. The list is not complete; in the only cases where results are known to vary between Unicode versions, results are published for version 4.0.0 and 6.0.0. Implementors wishing to provide reference results for other versions of Unicode are welcome to submit such results.
Tokens to denote different implementation-defined behavior in the presence of faulty restriction in a complex-type definition.
A full explanation of this token and its meaning is needed, but not yet available. For the moment let it suffice to say that if an all-group in a restriction allows content not allowed by the base type, the processor is not required to detect the problem by inspection of the schema in isolation. Three behaviors are allowed; the choice among them is implementation-defined. The values denoting the three behaviors are these.
Note, 20 January 2012. Is this distinction still required, or has it been overtaken by the change made to resolve bug 12185 Conditional Type Assignment and substitutability (or other late changes)?
Tokens to distinguish tests which use only the "required subset" of XPath in conditional type assignment from tests which use full XPath (or: any XPath outside the subset) in conditional type assignment. See "3.12.6 Constraints on Type Alternative Schema Components" of the Structures spec, which reads in part
A conforming processor must accept and process any XPath expression conforming to the "required subset" of [XPath 2.0] defined by the following grammar.
Note: Any XPath expression containing no static errors as defined in [XPath 2.0] may appear in a conforming schema. Conforming processors may but are not required to support XPath expressions not belonging to the required subset of XPath.
The value "restricted-xpath-in-CTA" denotes processor support for, or test applicability to, the minimal subset of XPath required of all conforming 1.1 processors. All 1.1 processors should support this feature and run tests marked with it.
The value "full-xpath-in-CTA" denotes processor support for, or test applicability to, full XPath in conditional type assignment expressions.
(These token values were added 29 July 2011 to address bug 13455 XPath subset causes problem.)
Clause 1.2 of validation rule Assertion satisifed (in Structures sec. 3.13.4.1) says
By default, comments and processing instructions are excluded from the partial post-schema-validation infoset, but at user option processors may retain comments and processing instructions instead of excluding them.
The value "comments-and-PIs-excluded" denotes the default situation: comments and processing instructions are suppressed before creating the XDM instance and thus cannot be examined by assertions.
The value "comments-and-PIs-included" denotes the opposite: comments and processing instructions are included in the XDM instance and thus can be examined by assertions. (Since this is required to be "at user option", any processor that supports this token must also be available in a configuration that supports the other token.)
(The user option was added in November 2012 to address bug 13935 xsd 1.1 assertions testing comment nodes. These token values were added 20 January 2012 to allow both configurations to be tested.)
This is an exact copy of the annotation element defined in the Schema Recommendation. It is duplicated here in order to replicate the functionality of the xsd:annotation element and because the Schema for Schemas cannot be imported.
This version of this schema embodies (or will embody, when finished) several changes to the original test suite schema.
Changes projected, planned, contemplated:
The WG has discussed adding a mechanism for implementation-defined behaviors which have more than a small number of possibilities (e.g. "How many decimal digits are supported for xsd:decimal values?"). Such a mechanism could be modeled loosely on the discretionary-option markup in the XSLT test suite.
In the meantime, we have a keyword-based mechanism that involves enumerating all the possible alternatives.
Changes completed:
implementation-defined,
implementation-dependent, and
indeterminate to enumerated values for
expected/@validity.In progress: