Checklist of Checkpoints for "QA Framework: Test Guidelines"

This version:: http://www.w3.org/TR/2003/WD-qaframe-test-20030516/qaframe-test-checklist
This document is an appendix to:: http://www.w3.org/TR/2003/WD-qaframe-test-20030516/
Latest version of "QA Framework: Test Guidelines":: http://www.w3.org/TR/qaframe-test/
Editors:: Dominique Hazaël-Massieux (dom@w3.org); Lofton Henderson (lofton@rockynet.com)
Contributors:: See Acknowledgments in QA Framework: Test Guidelines.

Abstract

This document is an appendix QA Framework: Test Guidelines [QAF-TEST] . It provides a tabular checklist of all checkpoints from the specification guidelines, complete with their priorities. Please refer to QA Framework: Test Guidelines [QAF-TEST] for the full statement and description of the test guidelines and checkpoints, as well as references to related documents and full credits and acknowledgements of contributors to the test guidelines work.

Status of this document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. The latest status of this document series is maintained at the W3C.

This document is derived from and is an appendix to QA Framework: Test Guidelines [QAF-TEST] , which document is a W3C Working Draft (WD), made available by the W3C Quality Assurance (QA) Activity for discussion by W3C members and other interested parties. For more information about the QA Activity, please see the QA Activity statement. Please see the "Status of this document" section of the corresponding specification guidelines [QAF-TEST] , for complete details about the status of the specification guidelines version from which this is extracted and which it accompanies.

Please send comments to www-qa@w3.org, the publicly archived list of the QA Interest Group [QAIG]. Please note that any mail sent to this list will be publicly archived and available. Do not send information you wouldn't want to see distributed, such as private data.

Publication of this document does not imply endorsement by the W3C, its membership or its staff. This is a draft document and may be updated, replaced, or made obsolete by other documents at any time. It is inappropriate to use W3C Working Drafts as reference material or to cite them as other than "work in progress".

A list of current W3C Recommendations and other technical documents can be found at http://www.w3.org/TR.

Introduction

This checklist includes all checkpoints from the QA Framework: Test Guidelines [QAF-TEST] presented in a tabular format. The checkpoint priorities are included, and the order of the test guidelines document is maintained.

The presentation is intended to be convenient for organizers and evaluators of QA projects in W3C Working Groups, to facilitate assessing test materials against the checkpoints. The table includes spaces for scoring each checkpoint, "yes" (satisfied), "no" (not satisfied), "n/a" (not applicable).

Priorities

For a description of the meaning of the priorities in the following table, please consult the conformance clause and the priorities definition section of the test guidelines ([QAF-TEST], chapter 3 and section 1.6) .

Checklist table

Nbr	Checkpoint	Priority	Yes	No	N/A
Guideline 1. Perform a functional analysis of the specification and determine the testing strategy to be used.
1.1	Define the test suite scope	[Priority 1]
1.2	Identify the specification(s) to be tested.	[Priority 1]
1.3	Analyze the structure of the specification, partition it as appropriate, and determine and document the testing approach to be used for the test suite as a whole and for each partition.	[Priority 1]
Guideline 2. Identify and tag testable assertions within the specification.
Nbr	Checkpoint	Priority	Yes	No	N/A
2.1	Identify and list assertions within the specification.	[Priority 1]
2.2	Metadata must be associated with test assertions, enabling test developers, the test-management system, the test-execution framework, and the results-reporting process to make useful distinctions between groups of tests.	[Priority 2]
Guideline 3. Provide a test management system
Nbr	Checkpoint	Priority	Yes	No	N/A
3.1	The Working Group must provide a test management system	[Priority 1]
3.2	The test management system must associate tests with metadata.	[Priority 1]
3.3	Test management system must allow filtering based on metadata.	[Priority 2]
3.4	Test management system must support results.	[Priority 2]
Guideline 4. Provide a test framework
Nbr	Checkpoint	Priority	Yes	No	N/A
4.1	Provide a test framework	[Priority 1]
4.2	Prototype the test framework	[Priority 2]
4.3	Automation of testing is encouraged.	[Priority 2]
Guideline 5. Provide results reporting
Nbr	Checkpoint	Priority	Yes	No	N/A
5.1	Test Materials must support results reporting	[Priority 1]
5.2	Results reporting framework should generate a unified report.	[Priority 2]
5.3	Results reporting framework must indicate result status of each test.	[Priority 1]
5.4	Results reporting framework should allow filtering based on metadata	[Priority 2]
Guideline 6. Plan for conformance testing
Nbr	Checkpoint	Priority	Yes	No	N/A
6.1	Organize conformance testing activities.	[Priority 1]
6.2	Encourage Vendors to publish test results.	[Priority 3]

References

QAF-TEST: QA Framework: Test Guidelines, P. Curran, D. Dimitriadis, P. Fawcett, K. Gavrylyuk, L. Henderson, M. Skall, Eds., W3C Working Draft, May 2003, available at http://www.w3.org/TR/qaframe-test/.