Testing/Terminology

From W3C Wiki

The Testing wiki is no longer maintained.

The canonical resource for W3C's Open Web Platform Testing Effort is now on testthewebforward.org.

idlharness.js
Test runner for WebIDL tests. Runs on top of testharness.js.
Assertion
smallest building block of testing.
Feature
a distinguishing characteristic of a specification.
JavaScript test
test which uses JavaScript to programmatically verify one or more assertions.
Manual test
test which must be run manually.
Manual test runner
test runner which helps running manual tests and collect their results.
Reftest
test which compares the visual output of one file with the output of one or more other files (the references). Reftests require WebDriver (or similar proprietary technology) to be automated.
Script test
see JavaScript test.
Test
verify one or more assertions to test a feature.
Test framework
the combination of the test runner and the repository of test suites and produce test reports.
Test page
single HTML document that contain one or more tests.
Test runner
program which is able to run multiple, potentially nested test suites and collect the results.
Test suite
collection of test pages.
testharness.js
W3C test page runner for Javascript tests.
testharness.js test
see JavaScript test.
WebDriver
W3C spec which allows programs or scripts to introspect into and control the behaviour of a web browser including taking screenshots. Required to automate running Reftests across compatible user agents.
WebIDL test
test which is automatically generated by parsing WebIDL snippets. Run using idlharness.js.