TestMaterialsQualities

From W3C Wiki

When reviewing or producing test materials, esp. test cases, one need to have in mind what are the expected qualities of these materials to get as much value as possible from them. (this is expected to be of use in the Test Guidelines when they come back to life)

Starting from a list of such qualities in the QA Handbook (when needed, we can create a different wiki page for a given quality to go into further details) :

correctness
whether the test case reflects what the specification says
traceability
whether one can know from what part of a specification a test case is derived
atomicity
whether a test case only covers a singe feature (also in SVG Test Suite manual) [@@@ not all test cases needs to be atomic, though...]
completeness
how much of the declared scope of the test materials is covered (see TestCoverageInformation)
reconfigurability
how much the test suite can re-used in different context/profiles (@@@ to be confirmed)
versioning/errata support
whether the test suite can be adapted to support different versions of a given specification, and/or to support errata to the said specification

The following were also in the list quoted above, but are not qualities per se; they look more like constituents of test materials, for which we may want to define specific qualities: user documentation, maintainer documentation, declaration of scope, harnesses or interfaces for application of the Test Materials, results assessment, results recording & reporting, automation features, declaration of publication licenses, integrated submission procedure.

(@@@ integrate more qualities from the SVG and CSS Test Suite, and HTML 4.01 Test Suite manuals)

QA qa.png