TestSuite

In general: Test Suite should NOT define performance or conformance tests.

### Example of Test Suites:

http://www.w3.org/2008/webapps/ElementTraversal/index.html

### Collected requirements:

- The tests suite should address all features, at least with one test. Tests addressing the mixture of features would be also good (e.g., combining a getParam with a filtering on it). [1]

- Test suite should embed positive and negative tests (e.g. getCreator -> what happens if there are duplicate entries, what happens if there is none?). [1]

- Test suite should define the expected behaviour of the API on specific test data (important to reproduce the tests) [1] (e.g., What happends with broken metadata?)

- Test suite should take into scope, how the metadata will be extracted from different sources [1] (e.g. You Tupe clip: has metadata on server, but might also have metadata references on other servers.)

- Test suite should run automatic and enclose all things in a specification tagged with MUST [1]

- Test suite should be able to execute on the server and client side [1]

- A single test in the test suite should be atomic [3]

- Each test must be passed by two implementations in order to pass to CR [3]

### IDEAS:

- Error codes / Error handling [1] (yet exceptions for the methods have been defined [2]) Florian thinks, we should define "error-gorups" (e.g., connection errors or metadata based errors)...after that we should come up with specific errors

- Defining the test suite with RefTest (WebApps Example) [1]

- Test Suite should only take 2 or 3 metadata formats into account [1]

- Setting up components to extract metadata from several sources [1]

### Links:

[1] http://www.w3.org/2009/11/06-mediaann-minutes.html#item11 (TPAC2009 - Day 2)

[2] http://www.w3.org/2009/11/24-mediaann-minutes.html

[3] http://www.w3.org/2009/12/08-mediaann-minutes.html

### flip chart drawings from Vienna F2F