TestsInWiki

From OWL
Jump to: navigation, search

Status of this page: Still under construction. We are leaning toward using one template versus several, but haven't settled on fields or functionality. Some test cases are entered and will be used to develop scripts, after which this page will be updated

Introduction

This is a first draft attempt to specify how tests can be embedded in the wiki page. The goal is to have the expression of the test case be as simple as possible, and then use templates (possibly including custom htmlets and scripts) and other features of the wiki to present and make available the test cases easy to run and as easy on the eyes as possible.

Things that will would make it easier to express would be:

  • Being able to express the premises and conclusions in any of a number of syntaxes
  • Being able to retrieve or view the premises and conclusions in any of a number of syntaxes
  • Not having to retype repetitive elements such as RDF headers and common namespaces

To this end, here is one design. Note that I haven't yet worked on the presentation. The goal would be to make it easy on the eyes, for instance by adopting the approach suggested in the UFDTF discussions - switchable view of the test in different syntaxes, rather than having them all in line.

Note that some details will be dependent on what we decide the process and workflow is for test cases. I don't assume that this document specifies that, and would adjust this page if necessary after we've come to agreement on that.

An example

The example is at TestCase:Experiment2. To create a new test one would create a page in the TestCase namespace, indicated by prefixing the page name with "TestCase:".

First, the whole page is wrapped in a template. The current version just passes the content through. This might be replaced by a custom parser directive, or custom tag, depending on which is easiest to do the best presentation with.

{{testcase2|

Then one uses the usual wiki markup for the test description

The test case description goes here. Describe what the test is trying to show, 
linking to relevant pages, issues, or email discussions. 

The author of the test case is indicated, and a URL to the irc log with approval for the test case - only use this should it be approved. Other attributes of tests that would be usefully structured would be added using a similar pattern. The author name is surrounded by "[[" and "]]" by the template, turning it in to a link to the author page. Use of testapproved adds [[Category:ApprovedTest]] to the page. Note: It would be nice to automatically have the category UnapprovedTest for pages that don't have an approvedtest marking - will investigate. If we decide that there are more states in the test workflow, we'll need to modify the below to indicate state.

* {{testauthor|Sandro Hawke}}
* {{testapproved|http://www.w3.org/2008/01/09-owl-irc#T17-54-14-1}}

Then one shows the premise and conclusion. This premise is in rdf. There would be similar templates to indicate functional syntax, and perhaps Manchester syntax. If only a single syntax is given for the premise, then the other forms of the syntax should be generated either dynamically, if we can set that up, or by batch script which updates the pages (preserving the information about which syntax the test writer initially wrote it in).

If more than one syntax is entered, then the syntax mapping software is also tested and reported and deviations from what has been typed in is reported on the page. Note that the in the RDF, the RDF header is not given, nor are namespace conventions. The intention would be that xml:base and xmlns would be set by the software, and that the standard set of namespaces are to be included in Template:TestRDFAround. Scripting will extract the standard namespaces from there and reformat them for the other syntaxes.

As of this writing, only the premise-rdf and conclusion-rdf templates are present. There would be analogous ones for -fs (functional syntax) -manchester (manchester syntax) and possibly -dl (for description logic syntax). I anticipate that we will use the OWLAPI to do the conversions between syntaxes. The OWL 1.0 tests specify a number of other possible document names, such as inconsistent. Should we adopt the same approach, there would be templates named inconsistent-rdf, etc.

{{premise-rdf|<pre>
<owl:Thing rdf:about="#a">
   <prop>foo</prop>
</owl:Thing>
</pre>}}

That the <pre></pre> tags are required seems to be a wiki bug - need to look in to this.

{{conclusion-rdf|<pre>
<owl:Thing rdf:about="premises301#a">
   <prop>bar<prop>
</owl:Thing>
</pre>}}

Then close the surrounding template.

}}

The wiki category system will be used for keywords that describe the level and the type of test, as well as any other keywords that might be useful in organizing the tests. Run scripts should be able to indicate a boolean expression of categories of tests to run, or in the absense of such, run all tests. Position of the Category directives doesn't matter. They are displayed at the bottom of the page by the wiki software. TBD: Fully define the set of test type and level tags.

[[Category:NegativeEntailmentTest]]
[[Category:OWLFullTest]]

Template

For convenience, here is a blank one to copy/paste into a new test case. ALLCAPS are what you have to fill in.

{{testcase2|
DESCRIPTION OF TEST 

* {{testauthor|YOUR NAME}}
<!-- uncomment the next line when the test is approved -->
<!-- * {{testapproved|LINK TO IRC LOG OF APPROVAL}} --> 

{{premise-rdf|<pre>
TEST IN RDF HERE
</pre>}}

{{conclusion-rdf|<pre>
CONCLUSION IN RDF HERE
</pre>}}

}}

<!-- Test type categories: NegativeEntailmentTest PositiveEntailmentTest others TBD -->
[[Category:TEST TYPE CATEGORY]]

<!-- Test level categories: OWLFullTest OWLDLTest others TBD-->
[[Category:TEST LEVEL CATEGORY]]

Alternatives

Bijan Parsia would prefer to use a single template, with named parameters instead of the individual templates used here. He suggest using a variety of different templates for the different test types. A possible disadvantage is that it might make it harder to get rid of display of optional elements, but this wouldn't be the case if a custom parser or tag was used. We differ about which is easier and less prone to error to enter. The current version gives some discretion to the writer about the order of elements. That could be considered either a pro or a con.

Comments

Please leave suggestions or comments here.

Author: Alan Ruttenberg. Thanks to Bijan Parsia for comments on earlier drafts.