Re: Common pool of examples and validation

On May 31, 2007, at 09:43, Karl Dubost wrote:

>> For all kinds of HTML5 test cases, it would be very useful to have  
>> a machine-readable annotation of whether the document is supposed  
>> to be conforming even if the test case has been written for  
>> another purpose such as browser testing. (This could be considered  
>> to be part of "expected results" but for test written for other  
>> purposes it may not automatically be.)
>
> What format do you propose? Please, give an example of code.

Echoing Anne's suggestion on IRC, I propose each directory of a test  
suite to have a file named conforming.txt that lists on LF-terminated  
lines the names (one name per line) of conforming test cases in that  
directory. All other test cases in the directory would be expected to  
be non-conforming. If there only non-conforming test cases in a  
directory, a file called conforming.txt would still need to be there  
but empty (to flag the directory as one that has been reviewed for  
test case conformance).

For test suites designed for conformance checker testing, I propose  
putting conforming test cases in a directory named valid/ and putting  
non-conforming cases in a directory called invalid/ and omitting  
conforming.txt.

-- 
Henri Sivonen
hsivonen@iki.fi
http://hsivonen.iki.fi/

Received on Thursday, 31 May 2007 08:54:06 UTC