Copyright © 2008 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark and document use rules apply.
This document describes the test cases developed by the RIF Working Group in accordance with the Working Group's Charter. These test cases are intended to aid in the conformance evaluation of RIF implementations and thus promote interoperability.
This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.
This document is being published as one of a set of 8 documents:
The Rule Interchange Format (RIF) Working Group seeks public feedback on these Working Drafts. Please send your comments to public-rif-comments@w3.org (public archive). If possible, please offer specific changes to the text that would address your concern. You may also wish to check the Wiki Version of this document for internal-review comments and changes being drafted which may address your concerns.
Publication as a Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.
This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.
Contents |
This document describes the test cases developed by the RIF (Rule Interchange Format) Working Group in accordance with the Working Group's Charter. The sections below delineate the scope of the test cases, explain their limits in determining conformance, and present the different types of tests and the format in which they are represented.
The test cases themselves are not included in this document; they are maintained externally to the document in the RIF Test Repository.
The development of a set of RIF test cases aims to enable an empirical investigation of RIF implementations. The tests can help identify problems both with the software developed to implement RIF specifications and with the specifications themselves. A widely used test suite providing good coverage of the target dialect makes it more likely that different implementations will interoperate correctly.
These test cases are designed to be suitable for use by RIF implementers in test harnesses. Developers will need to write their own test harnesses, and are encouraged to report their results as well as any problems they encounter with the tests.
If additional test cases are contributed to the W3C, the Consortium may add them to the set of RIF test cases.
This first Working Draft documents test cases for the Basic Logic Dialect (BLD) of RIF, and for RDF-BLD and OWL-BLD combinations as specified in RIF RDF and OWL Compatibility. Future drafts of this document will cover test cases for other RIF dialects as those dialects are finalized. The current set of tests focusses on testing entailment and syntax, as described in the Test Types section. We expect to add additional types of tests in the future.
The BLD test suite contains tests for RIF-BLD documents and consumers as defined in the RIF-BLD Conformance Clauses, but not for RIF-BLD-producers. The tests are designed to:
The test cases do not constitute a complete conformance test
suite.
Conformance is defined as the fulfillment of specified requirements, which are detailed in the conformance clause of a specification. (See QA Framework specification Guide.) Conformant RIF-BLD documents, consumers and producers are defined in the conformance clauses section of the RIF BLD specification. A complete conformance test suite would be a set of test cases that cover this set of requirements, such that passing all the tests in the suite indicates conformance. However, the development of this type of comprehensive test suite is beyond the scope of the Working Group, and the RIF test cases are not a complete conformance test suite since they do not completely cover the language. Failure to pass all the tests in the suite indicates that the implementation does not meet the relevant specification. But passing all the tests in the suite indicates only that the implementation is conformant to that particular version of the test suite.
The test suites include the following deliverables:
Editor's Note: The web page referenced above will soon contain a directory of all test files. (It currently does not.) In addition, the web page will provide a categorization of all test cases, similar to what is currently contained at http://www.w3.org/2005/rules/wiki/Category:Test_Case.
An Entailment Test is a test T in which a RIF condition is either entailed (for Positive Entailment Tests) or not entailed (for Negative Entailment Tests) by the imports closure of a given RIF document. Each test T has an associated dialect D and is of the form
Where R is a RIF document, which possibly imports other documents, and C is a RIF document of the dialect D such that either there is an indication that T is a positive entailment test and R entails C in the dialect D or there is an indication that T is a negative entailment test and R does not entail C in the dialect D.
Editor's Note: We currently have a test case (RDF_Combination_Constant_Equivalence_Graph_Entailment) where the conclusion is RDF.
A Positive Entailment test is passed if the conclusion document is entailed by the imports closure of the premises document. Each test includes a premises document, indicated by the premisesDocument element in the manifest for the test, a conclusion document, indicated by the conclusionDocument element, and optionally one or more documents in the imports closure of the premises document, indicated by importedDocument elements.
The intended entailment regime is indicated by the dialect element in the manifest file. If the value of this property is "BLD" then the entailment holds according to the RIF-BLD definition of entailment.
In addition, optional importSupport elements in the manifest indicate the types of documents in the imports closure of the premises document. If an importSupport property is present with the value "RDF" then the test constitutes a RIF-RDF combination as defined in RIF RDF and OWL compatibility and there are additional rules of entailment in effect. If an importSupport property is present with the value "OWL-DL" or "OWL-Full" then the test constitutes a RIF-OWL combination and the associated entailment rules are defined in OWL compatibility.
In general the conclusion will not include everything that is entailed by the premises.
Editor's Note: Do we want to address BLDΤ,Ε consumers, and talk about the set DTS, about datatype support?
A Negative Entailment test is passed if the conclusion document is not entailed by the imports closure of the premises document. Each test includes a premises document, indicated by the premisesDocument element in the manifest for the test, a conclusion document, indicated by the conclusionDocument element, and optionally one or more documents in the imports closure of the premises document, indicated by importedDocument elements.
Editor's Note: The RDF test document has this statement - do we want something similar? "The test is failed if the conclusion can be drawn from the premises ...The test is considered to be passed when it can be conclusively demonstrated that the conclusion cannot be drawn. In practice, the test may be considered to be passed when a thorough attempt to fail the test is unable to achieve failure."
The intended entailment regime is indicated by the dialect element in the manifest file. If the value of this property is "BLD" then the entailment does not hold according to the RIF-BLD definition of entailment.
In addition, optional importSupport elements in the manifest indicate the types of documents in the imports closure of the premises document. If an importSupport property is present with the value "RDF" then the test constitutes a RIF-RDF combination as defined in RIF RDF and OWL compatibility; consequently, there are additional rules of entailment in effect. If an importSupport property is present with the value "OWL" then the test constitutes a RIF-OWL combination; the associated entailment rules are defined in OWL compatibility.
These tests check for syntactic validity that cannot be checked by XML Schema validation.
These tests involve a single RIF document, indicated by the testDocument element in the manifest for the test. The document is a conformant RIF document in the dialect indicated by the dialect element.
If the value of the dialect property is "BLD" then the test document is a conformant RIF-BLD document
These tests involve a single RIF document, indicated by the testDocument element in the manifest for the test. The document is not a conformant RIF document in the dialect indicated by the dialect element.
If the value of the dialect property is "BLD" then the test document is not a conformant RIF-BLD document
Each test case has an associated manifest file that contains information needed to properly execute the tests. This file is in a machine-readable format, RDF/XML, in order to enable the development of automated testing frameworks. The manifest file also contains information, such as contributor and status, used for test case management. Some examples of how the data in the manifest file can be used are:
The format of the RIF test case manifest files is based on the W3C QA Working Group's [Test Metadata Note] and the schema that was developed to accompany it. A complete description of the manifest file is given in an appendix. A summary of the metadata property elements is included below, organized by the type of test to which they apply.
In the following, dc is bound to http://purl.org/dc/elements/1.1/, and rifTest is bound to http://www.w3.org/2005/rules/test/rifTestSchema#.
| Description | A brief explanation of what is being tested, suitable for display in tabular form |
|---|---|
| Cardinality | one |
| Format | Free-form (hypertext) or constrained plain text |
| Description | A more detailed explanation, where appropriate, about the nature or characteristics of the test. |
|---|---|
| Cardinality | zero or one |
| Format | Free-form (hypertext) |
| Description | indicates the RIF dialect that this test applies to. |
|---|---|
| Cardinality | one -or- one or more |
| Format | one of "rifTest:BLD" |
Editor's Note: BLD could mean BLD and every dialect that it extends, or we could include an explicit entry for each dialect that the test applies to
| Description | Identification of a portion of a specification relevant to this test. |
|---|---|
| Cardinality | zero or more |
| Format | Free-form (hypertext) or an HTML link to the relevant portion of a specification. |
| Description | One of an enumerated list of values that indicates the state of the test. |
|---|---|
| Cardinality | one |
| Format | one of "rifTest:Proposed", "rifTest:Accepted", "rifTest:Rejected" , "rifTest:Postponed" "rifTest:Obsolete" |
| Description | The individual that contributed this test. |
|---|---|
| Cardinality | one or more |
| Format |
| Description | indicates a language feature that this test is related to |
|---|---|
| Cardinality | zero or more |
| Format | URI |
| Description | An umambiguous reference to the test |
|---|---|
| Cardinality | one |
| Format | URI |
| Description | The file containing a document that participates in the test |
|---|---|
| Cardinality | one |
| Format | URI |
Subproperties of rifTest:testDocument are used to provide testDocuments in alternative syntaxes.
| Description | The file containing the premises document for the test |
|---|---|
| Cardinality | one |
| Format | URI |
| Description | The file containing the conclusion document for the test |
|---|---|
| Cardinality | one |
| Format | URI |
| Description | A file imported by one of the other files participating in the test |
|---|---|
| Cardinality | zero or more |
| Format | URI |
Subproperties of rifTest:importedDocument are used to provide importedDocuments in alternative syntaxes.
| Description | Indicates what type of imported documents this test includes. This is intended for use in filtering for applicable tests. The idea is that if there are multiple imported documents of the same type, there would only be one of these for those files. If all the imported files are BLD and the dialect is BLD, this element would not be present. |
|---|---|
| Cardinality | zero or more |
| Format | one of rifTest:RDF, rifTest:OWL-DL, rifTest:OWL-Full |
The Test Case Repository [REPOSITORY] has subdirectories each containing a single test case. Each test case consists of a manifest file describing the test, the normative files participating in the test, and additional non-normative formats of the test files which are included in order to aid in understanding the test.
Tests can be viewed classified by test type, issue, dialect, and feature.
Each of these tests is intended to be run on different rule systems, each with its own API. The tasks of providing input to the rule engine in a system-dependent manner and of checking that the results are correct are therefore left to the tester. The manifest contains a machine-readable description of the test cases in RDF/XML, and is intended to assist in processing the test cases.
Implementers are expected to write their own test harnesses. Each test harness should carry out the following tasks:
The dialect and importSupport elements in the manifest file can be used to filter for appropriate tests. In the current set of test cases, all tests are in the BLD dialect. Implementations that do not support BLD-RDF combinations should skip tests that have an importSupport property with a value of RDF. Implementations that do not support BLD-OWL combinations should skip tests that have an importSupport property with a value of OWL-DL or OWL-Full.
Testers have the option of making a number of well-defined changes to the test cases in order to facilitate use of the test suite. All changes made to the original test suite should be documented in free-text form as part of the results submission.
Editor's Note: The possibility here is that in the tests provided, import directives point to files on the W3C website, but people may want to customize to access these files in their own environment.
If you believe that a test case is incorrect --- e.g., the test input is underspecified, or ambiguous, or the conclusion should not hold --- we encourage you to enter a bug report in the W3C public version of Bugzilla, http://www.w3.org/Bugs/Public/. To enter a bug report you will need a Bugzilla signon (email address and password). You can create this signon by following the Open a new Bugzilla account link on the Bugzilla Main Page.
General information on Bugzilla, an open-source bug tracking system, is available at http://www.bugzilla.org/. The standard documentation for the version currently being run on the W3C server is available at http://www.bugzilla.org/docs/2.20/html/.
To enter a bug report in Bugzilla, you will need to log in, choose the action "New", and then choose the "RIF Test Cases" product. Please be specific about the problem that you are reporting. Clearly identify the test case or test cases that are problematic. If you believe that the expected result is wrong, then tell us what you believe the expected result should be. If Please cite portions of one or more of the RIF specifications to support your position whenever possible.
The schema for the RIF Test Case manifest files is shown below and is also available as rifTestSchema in the RIF Test Cases Repository.
Editor's Note: This is not complete, but is included here to invite discussion about the approach.
<?xml version="1.0"?>
<!DOCTYPE rdf:RDF [
<!ENTITY owl "http://www.w3.org/2002/07/owl#" >
<!ENTITY rdfs "http://www.w3.org/2000/01/rdf-schema#" >
<!ENTITY rdf "http://www.w3.org/1999/02/22-rdf-syntax-ns#" >
<!ENTITY rifTestSchema "http://www.w3.org/2005/rules/test/rifTestSchema#" >
]>
<rdf:RDF xmlns="http://www.w3.org/2005/rules/test/rifTestSchema#"
xml:base="http://www.w3.org/2005/rules/test/rifTestSchema#"
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
xmlns:owl="http://www.w3.org/2002/07/owl#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:rifTest="http://www.w3.org/2005/rules/test/rifTestSchema#">
<owl:Ontology rdf:about="">
<rdfs:comment>In development - schema for RIF Test Case manifest files</rdfs:comment>
<rdfs:seeAlso>http://www.w3.org/TR/test-metadata/</rdfs:seeAlso>
</owl:Ontology>
<!-- Classes -->
<owl:Class rdf:about="#RIFTest">
<rdfs:subClassOf rdf:parseType="Resource">
<rdf:type rdf:resource="http://www.w3.org/2002/07/owl#Restriction"/>
<owl:minCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1</owl:minCardinality>
<owl:onProperty rdf:resource="http://purl.org/dc/elements/1.1/title"/>
</rdfs:subClassOf>
<rdfs:subClassOf rdf:parseType="Resource">
<rdf:type rdf:resource="http://www.w3.org/2002/07/owl#Restriction"/>
<owl:maxCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1</owl:maxCardinality>
<owl:onProperty rdf:resource="http://purl.org/dc/elements/1.1/title"/>
</rdfs:subClassOf>
<rdfs:subClassOf rdf:parseType="Resource">
<rdf:type rdf:resource="http://www.w3.org/2002/07/owl#Restriction"/>
<owl:minCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1</owl:minCardinality>
<owl:onProperty rdf:resource="#purpose"/>
</rdfs:subClassOf>
<rdfs:subClassOf rdf:parseType="Resource">
<rdf:type rdf:resource="http://www.w3.org/2002/07/owl#Restriction"/>
<owl:maxCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1</owl:maxCardinality>
<owl:onProperty rdf:resource="#purpose"/>
</rdfs:subClassOf>
<rdfs:subClassOf rdf:parseType="Resource">
<rdf:type rdf:resource="http://www.w3.org/2002/07/owl#Restriction"/>
<owl:maxCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1</owl:maxCardinality>
<owl:onProperty rdf:resource="http://purl.org/dc/elements/1.1/description"/>
</rdfs:subClassOf>
<rdfs:subClassOf rdf:parseType="Resource">
<rdf:type rdf:resource="http://www.w3.org/2002/07/owl#Restriction"/>
<owl:minCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1</owl:minCardinality>
<owl:onProperty rdf:resource="http://purl.org/dc/elements/1.1/contributor"/>
</rdfs:subClassOf>
<rdfs:subClassOf rdf:parseType="Resource">
<rdf:type rdf:resource="http://www.w3.org/2002/07/owl#Restriction"/>
<owl:maxCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1</owl:maxCardinality>
<owl:onProperty rdf:resource="http://purl.org/dc/elements/1.1/contributor"/>
</rdfs:subClassOf>
<rdfs:subClassOf rdf:parseType="Resource">
<rdf:type rdf:resource="http://www.w3.org/2002/07/owl#Restriction"/>
<owl:minCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1</owl:minCardinality>
<owl:onProperty rdf:resource="#status"/>
</rdfs:subClassOf>
<rdfs:subClassOf rdf:parseType="Resource">
<rdf:type rdf:resource="http://www.w3.org/2002/07/owl#Restriction"/>
<owl:maxCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1</owl:maxCardinality>
<owl:onProperty rdf:resource="#status"/>
</rdfs:subClassOf>
</owl:Class>
<owl:Class rdf:about="#SyntaxTest">
<rdfs:subClassOf rdf:resource="#RIFTest"/>
</owl:Class>
<owl:Class rdf:about="#PositiveEntailmentTest">
<rdfs:subClassOf rdf:resource="#SyntaxTest"/>
</owl:Class>
<owl:Class rdf:about="#NegativeEntailmentTest">
<rdfs:subClassOf rdf:resource="#SyntaxTest"/>
</owl:Class>
<owl:Class rdf:about="#ImportNegativeEntailmentTest">
<rdfs:subClassOf rdf:resource="#NegativeEntailmentTest"/>
</owl:Class>
<owl:Class rdf:about="#ImportPositiveEntailmentTest">
<rdfs:subClassOf rdf:resource="#PositiveEntailmentTest"/>
</owl:Class>
<owl:Class rdf:about="#ReviewStatus">
<label xml:lang="en">A status for a simple review process containing 6 possible stages</label>
<subClassOf rdf:resource="http://www.w3.org/2006/03/test-description#ReviewStatus"/>
<owl:oneOf rdf:parseType="Resource">
<rdf:first rdf:resource="#InDevelopment"/>
<rdf:rest rdf:parseType="Resource">
<rdf:first rdf:resource="#Proposed"/>
<rdf:rest rdf:parseType="Resource">
<rdf:first rdf:resource="#Accepted"/>
<rdf:rest rdf:parseType="Resource">
<rdf:first rdf:resource="#Rejected"/>
<rdf:rest rdf:parseType="Resource">
<rdf:first rdf:resource="http://www.w3.org/2006/03/test-description#Postponed"/>
<rdf:rest rdf:parseType="Resource">
<rdf:first rdf:resource="http://www.w3.org/2006/03/test-description#Obsolete"/>
<rdf:rest rdf:resource="http://www.w3.org/1999/02/22-rdf-syntax-ns#nil"/>
</rdf:rest>
</rdf:rest>
</rdf:rest>
</rdf:rest>
</rdf:rest>
</owl:oneOf>
</owl:Class>
<!-- Properties -->
<rdf:Property rdf:about="#feature">
<rdfs:comment>This property relates a test to a language feature. The language feature is usually indicated by a class or property. </rdfs:comment>
</rdf:Property>
<owl:ObjectProperty rdf:about="#status">
<rdf:type rdf:resource="&owl;FunctionalProperty"/>
</owl:ObjectProperty>
<owl:DatatypeProperty rdf:about="#purpose">
<domain rdf:resource="#RIFTest"/>
<label xml:lang="en">purpose</label>
<range rdf:resource="http://www.w3.org/2000/01/rdf-schema#Literal"/>
</owl:DatatypeProperty>
<owl:DatatypeProperty rdf:about="#specRef">
<comment xml:lang="en">a description or a link indicating a portion of a specification that is relevant to this test case</comment>
<domain rdf:resource="#RIFTest"/>
<label xml:lang="en">reference in specification</label>
<range rdf:resource="http://www.w3.org/2000/01/rdf-schema#Literal"/>
</owl:DatatypeProperty>
<owl:DatatypeProperty rdf:about="#conclusionDocument">
<rdfs:subPropertyOf rdf:resource="#testDocument"/>
</owl:DatatypeProperty>
<owl:DatatypeProperty rdf:about="#importedDocument"/>
<owl:DatatypeProperty rdf:about="#nonConclusionDocument">
<rdfs:subPropertyOf rdf:resource="#testDocument"/>
</owl:DatatypeProperty>
<owl:DatatypeProperty rdf:about="#premiseDocument">
<rdfs:subPropertyOf rdf:resource="#testDocument"/>
</owl:DatatypeProperty>
<owl:DatatypeProperty rdf:about="#testDocument"/>
<!-- General axioms -->
<rdf:Description>
<rdf:type rdf:resource="&owl;AllDifferent"/>
<owl:distinctMembers rdf:parseType="Collection">
<rdf:Description rdf:about="#InDevelopment"/>
<rdf:Description rdf:about="#Proposed"/>
<rdf:Description rdf:about="#Accepted"/>
<rdf:Description rdf:about="#Rejected"/>
<rdf:Description rdf:about="#Postponed"/>
<rdf:Description rdf:about="#Obsolete"/>
</owl:distinctMembers>
</rdf:Description>
</rdf:RDF>