Idea for More Robust Test Suite Tool

by Nadia Heninger

2002-04-11

Current Situation

The script has a light touch: I wanted to avoid all problems with security and space management, so it's completely ephemeral and leaves nothing on the server. This means that it must carry on all state information as cgi variables, and re-load and re-parse the RDF file with the information for the test suite each time it is called. This works fine for small demo applications, but is not scalable, and I don't trust my n3 parser (or even the fact of even relying completely on the non-standardized n3 format) to be a permanent solution.

My kludgy solution for temporarily storing and being able to ask questions of my RDF is to put it in a rather complicated tree-like hash structure with the top level being sorted by rdf:type declarations. There is a level of abstraction above that (for the most part) with a wrapping object that I have methods to get a test case and all its properties, give me the names of all the user-defined properties possible, etc.

The Idea for Improvement

Looking at Eric's annotea server code, it occurred to me that it would be neat to have a permanent test suite server. With a decent schema for defining test cases, it would be possible to write test cases and submit them to the server, and then retrieve them at will for automated test suite presentation later on. This would create a lot more overhead in the beginning, but for larger applications saves having to re-parse the file, and allows for possible mixing of test suites, and customization just by making a small file.

So I could write up summaries of 50 WCAG check points, submit them to the server, and then later ask for all test cases belonging to the "WCAG" test suite of a certain priority. Later, Charles decides he wants to make an evil browser test suite that combines tests from SVG, MathML, and CSS test suites. So he finds the RDF ids for all the test cases he wants, and submits a file to the server with just a bunch of declarations that each testcase belongs to his test suite as well. Then later he can query for all test cases belonging to "Charles's Evil Test Suite" and have the tests he wants presented to him.

With the algae querying language, it should be fairly easy to create a html form that would gather information from all the test suites and provide parameters to create custom algae queries on the fly to retrieve exactly the tests desired.

Example algae query to get all testcases belonging to a hypothetical SVG test suite:

(ask 
    '((http://www.w3.org/1999/02/22-rdf-syntax-ns#type ?a http://www.w3.org/2001/03/earl/0.95#TestCase)
      (http://www.w3.org/2001/03/earl/0.95#TestSuite ?a http://www.w3.org/xxxx/xx/thingy#SVG) 
      (?p ?a ?o) ) 
 collect '(?a)) 

The database can spit back RDF, and after that it's not difficult to format the information nicely for human presentation.

Problems

It will take a long time to get familiar with the Annotea code, and there seem to be random elements of annotea that are mixed in with other RDF and database utilities, so it's a lot of work to separate them out. Once familiar with the system, the problems have all already been solved.