XML Schema Test Suite

This version:
28 July 2011
Current Editor:
C. M. Sperberg-McQueen, Black Mesa Technologies LLC for W3C <cmsmcq@w3.org>
Earlier Editors:
Henry S. Thompson, W3C <ht@w3.org>
John Tebbutt, NIST <tebbutt@nist.gov>
Tony Cincotta, NIST

Table of contents

  1. Overview
  2. Namespace
  3. This Document
  4. Getting the Test Suite
  5. Releases
  6. Latest News
  7. Documents
  8. Discussion
  9. Participation
  10. Source files and CVS
  11. Current Activities
  12. Acknowledgments

Overview

The XML Schema Test Suite (XML Schema TS, TS) is a collection of tests for the W3C XML Schema Recommendations. The TS is updated from time to time. All releases of the TS can be accessed from this page, which is also the namespace document for the TS namespace.

The most recent full release of the TS (version 2006-11-06) contains nearly 40,000 test files and associated metadata.

The TS consists of a set of test collections, each of which provides a set of metrics for determining conformance to a particular version of the W3C XML Schema Recommendation. In each case, the test collection is intended to complement the version of the Recommendation with which it is associated.

All interpretations of the Recommendation are subject to confirmation by the W3C XML Schema Working Group (WG).

Developers of W3C Schema aware processors are encouraged to use the TS, to discuss issues which arise, and to report their results to the WG. The WG also welcomes contributions of tests to the TS.

Namespace

The XML Schema Test Suite namespace is http://www.w3.org/XML/2004/xml-schema-test-suite/. The XML Schema Working Group may add definitions for names in this namespace from time to time, but will endeavour not to change the usage of existing names.

A browsable W3C XML Schema document is available on the Web, which defines both the syntax and semantics of names in this namespace.

This Document

This document serves as a first point of information for all interested parties. It contains information on accessing all releases of the TS, and gives a general account of current activities, finalized work, and pointers to further information.

Getting the Test Suite

As shown in the table below (under Releases), the entire test suite is available in tarred and gzipped form, in several versions, reflecting the state of the test suite at various dates.

Recent changes are recorded in the W3C's CVS repository for development work, dev.w3.org. For technical and historical reasons, tests which have not changed since the most recent bundle are not available from the CVS repository.

To obtain the current version of the test suite, therefore, it is necesary both to download the tarred and gzipped form of the most recent release, and also to check out the most recent changes from CVS. (With some justice, this arrangement has been described by a friendly observer as “a rather unorthodox and unfriendly mechanism”, but for the moment it's the only way to get the current test suite.)

The expected usage pattern is this:

  1. Download the latest release in ...tar.gz form.

    At the time this list was last revised (October 2010) the latest full release was that of 20 June 2007, but you should check the table below for later versions.

  2. Unpack the tarred and gzipped file into location B.

    From the command line, for example, one might type tar -xvzf xsts-2007-06-20.tar.gz. This will unpack the archive into a directory. At the moment, this directory is named xmlschema2006-11-06.

  3. Check out the CVS tree from dev.w3.org into location A.

    Specifically, from repository dev.w3.org:/sources/public, check out module XML/xml-schema-test-suite/2004-01-14/xmlschema2006-11-06. The details of how to do this depend on whether you have a CVS account and SSH credentials on dev.w3.org. If you do, then from the command line, you might type cvs -d dev.w3.org:/sources/public checkout -d xsts1.1 XML/xml-schema-test-suite/2004-01-14/xmlschema2006-11-06 to check the current test suite out into a directory named xsts-current.

    If you do not have a CVS account, you can check out a read-only copy of the repository anonymously, using :pserver:anonymous@dev.w3.org:/sources/public as the CVS respository and anonymous as the password. From the command line, you might type cvs -d :pserer:anonymous@dev.w3.org:/sources/public checkout -d xsts1.1 XML/xml-schema-test-suite/2004-01-14/xmlschema2006-11-06 to check the material out into directory xsts-current. Further information about the W3C's public CVS repository is available.

  4. Move everything from the tree at B which is not already under A into the parallel place in A to where it is in B.

    From a bash command line, you might type cp -R -p -n xmlschema2006-11-06 xsts-current to do this.

  5. Whenever the WG updates anything in the test suite, whether metadata or test files, the changes are checked in to the CVS repsitory; users can thus update from the CVS repository, or browse the CVS Web interface to get all current changes.

Releases

All available releases of the TS can be accessed through the links below:

XML Schema Version 1.0 XML Schema Version 1.0, 2nd Edition
Contributor
XML Schema Part 0: Primer
XML Schema Part 1: Structures
XML Schema Part 2: Datatypes
XML Schema Part 0: Primer, 2nd Edition
XML Schema Part 1: Structures, 2nd Edition
XML Schema Part 2: Datatypes, 2nd Edition
Microsoft Corporation
Previous Release:
MS2002-01-16
Latest Release:
MS2006-11-06
Sun Microsystems
Previous Release:
Sun2002-01-16
Latest Release:
Sun2006-11-06
National Institute of
Standards and Technology
Previous Release:
NIST2002-01-16
Latest Release:
NIST2004-01-14
(
Formatted test descriptions)
The Boeing Company
Latest Release:
Boeing2007-06-20
[all]
Download:
Download tar.gz (6.73 Mb)
Download:
Download tar.gz (4.2 Mb)
Updated 2007-06-20
Reports XMLSchema2002-01-16 references the test report for tests for the original XML Schema Test Collection compiled by Henry Thompson and Oriol Carbo of the University of Edinburgh.
XSV 2.11-1
CVS directory containing reports, one per test set from the Microsoft and Sun contributions.
XSV 3.0-1
CVS directory containing a report for the Boeing contribution.
Saxon 8.9
CVS entry for a report on the entire collection as of 2006-11-06
Other reports wanted!
Please add new subdirectories under the 'reports' subdirectory of the current release in CVS.


Latest News

2 July 2010 The schema for the test suite has been revised to reflect the need to specify tests or test results applicable only to certain versions of the XSD specification. A stylesheet processing instruction has been added to the schema document to make it convenient to browse the schema in a Web browser. The schema document now does a more complete job of documenting the meaning and expected usage of catalog constructs.

20 June 2007 The most recent full release of the TS was made on 20 June 2007, following a contribution from Boeing.

6 November 2006 The previous release of the TS was made on 6 November 2006, following contributions from Microsoft and Sun updating and considerably extending their 2002 contributions.

Documents

The TS is developed in accordance with the XML Schema Test Suite Process Document.

There is also a FAQ for the TS, which gives convenient access to basic information.

For information on the XML Schema TS Framework, consult this document.

For background information on conformance testing for XML Schema as well as test descriptions for each of the test files included in this release see the TS test releases.

Discussion

The main forum for discussion of the TS is the XML Schema TS mailing list, public-xml-schema-testsuite@w3.org. This list is intended to be used for general discussion about the test suite, and is archived at http://lists.w3.org/Archives/Public/public-xml-schema-testsuite/. For raising questions about specific tests, please use the W3C Bugzilla service, specifying the product as "XML Schema Test Suite". You can use the Bugzilla search interface to view the status of any tests about which issues have been raised.

Participation

Using the Tests

The TS is openly available for use by all interested parties through the links given above.

Reporting Test Results

The Working Group strongly encourages all members of the community to report the results obtained when exercising processors against the TS or any part thereof. Such feedback gives the WG valuable insight into such issues as the relevance to the community of various parts of the Recommendation, which parts are open to conflicting interpretation, and so on.

The results reporting procedure is outlined in the TS Framework document.

Contributing to the Test Suite

The XML Schema TS is produced in a public framework, and relies on contributions from vendors and other developers in the community. We therefore encourage all members of the community to consider contributing any tests they may have developed to the TS.

The tests submission procedure is outlined in the TS Framework document.

Source files and CVS

TS source files are stored and tracked in the W3C public CVS repository. The project is called XML-Schema-Test-Suite. As of the 2006-11-06 release, only the metadata is currently checked in there for most of the contributions. As and when actual data files are changed from their state on release day (and as available in the distribution above), they will be checked in there as well.

Current Activities

The XML Schema Working Group is currently working to make a version of the test suite for progressing XSD 1.1 to Proposed Recommendation.

Acknowledgments

The original W3C XML Schema Test Collection was compiled and adminstered by Henry Thompson and Oriol Carbo of the University of Edinburgh.

The present Test Suite consists of collections of tests donated by W3C member organizations, including resubmissions of the collections present in the original Test Collection, using the formats and procedures developed for the Working Group by NIST.


Henry S. Thompson, W3C, editor
David Ezell, National Association of Convenience Stores, chair
C.M. Sperberg-McQueen, W3C, staff contact
Valid XHTML 1.0!