TestCaseMetadata
Introduction
A test case illustrates a feature/conformance requirement of a specification, and is used as a unit to build up a test suite. It often comes with a set of metadata allowing to track its status, context, link to the specification.
A W3C WG Note “Test Metadata” has been published by the W3C QA WG and summarizing the discussion and the work done in this wiki. You are welcome to add comments and continue the work either here or on the mailing list www-qa@w3.org which has been done in this document.
Usually Test cases fit in a TestSuiteArchitecture and involve a TestDevelopmentMethodologies
Schemas for Test
- Implementations of Test Metadata as an RDF Schema by Dom
- Schema for the RDF test cases
- MUTAT
- TTCN (pdf file), esp the description of the dynamic part; to investigate further.
Technologies-specific metadata
Examples of metadata/description schema specifics to a given technolgoy; can be useful to assess the various needs for a Test Cases Management system for example (QaTools), or to identify patterns that could lead to more generic types.
- VoiceXML 2.0 Test API
- an XML Schema describes the different input possibles and grammar aspects for the output
- RDF Schema for test cases
- an RDF Schema with RDF-specific classes (e. g. Negative Entailment Test, Positive Entailment Test), describing the type of the test cases.
- OWL Ontology for test cases
- an ontology describes types of test cases adapted to OWL (eg. Consistency Test )
- CSS Test cases filenames
- the test cases in the CSS Test suite carry their metadata in their filenames; they include additional requirements (for the implementations, the tester), the format in which is given the test case (XHTML, XML), and the kind of test [@@@ kind, type... looks like a pattern]
- XML 1.1 and Namespaces in XML 1.1 Test suites
- [1] each test case is categoried into on of 4 categories (e.g. validating, nonvalidating) and includes a description, an entry describing the section or fules from the XML 1.0 (2nd ed) Rec., a unique test id within a given collection, the collection from which the test originated, and if applicable, the kind of external entities a nonvalidating processor must include.
- XSLT/XPath 1.0 test suite
- from an OASIS Technical Committee (see the Documents link, particularly the "How to Submit Tests" document) established several bits of test case metadata, concentrating on the ability to filter a single collection based on the version of XSLT under test, errata having been implemented or not, discretionary choices granted to implementers, and other dimensions of variability. In addition, it has a set of scenarios that allow adapting the inputs and outputs expected on a case-by-case basis. The design of the test case catalog is intended to work for a range of test suites, with technology-specific lists (document names, discretionary choices, scenarios, etc.) kept separately from the main catalog specs, yet full validation of a catalog is possible through clever allocation of IDs. -- David Marston
- DOM test suite
- an XML schema and a DTD describes a well-formed test case for the DOM Test suite
- HTML 4.01 Test Suite
- the HTML 4.01 Test suites uses a well-defined HTML markup to define the test purposes and the tested assertion
- UUAG/HTML Test Suite
- uses a DTD to describe its test cases, and then an XSLT style sheet to turn them into HTML
- The ValidatorTestSuite
- is currently in the process of developing test cases, with an appropriate description language in RDF.
- XML Query Test Suite
- the Test Task Force of the XML Query Working Group (assisted by the XSL Working Group) issued a test suite with several thousand cases and an XML-based catalog. There is a document about submitting cases that gives a few examples of the catalog format, as well as detailed documentation of the metadata values. For deeper study, the Guidelines for Running the suite show how the catalog data guides the tester in customizing test artifacts.