Copyright © 2008 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark and document use rules apply.
This document describes the test cases developed by the RIF Working Group in accordance with the Working Group's Charter. These test cases are intended to aid in the conformance evaluation of RIF implementations and thus promote interoperability.
This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.
This document is being published as one of a set of 85 documents:
The Rule Interchange Format (RIF) Working Group seeks public feedback on these Working Drafts. Please send your comments to public-rif-comments@w3.org (public archive). If possible, please offer specific changes to the text that would address your concern. You may also wish to check the Wiki Version of this document for internal-review comments and changes being drafted which may address your concerns.
Publication as a Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.
This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.
This document describes the test cases developed by the RIF (Rule Interchange Format) Working Group in accordance with the Working Group's Charter. The sections below delineate the scope of the test cases, explain their limits in determining conformance, and present the different types of tests and the format in which they are represented.
The test cases themselves are not included in this document; theyare maintained externally to the documentin the RIF Test Repository ., and the
developmentnormative test suites can be downloaded from there. Formatted
copies of aall test cases in the repository, plus additional tests
that have not yet been fully processed by the Working Group, can be
viewed at the RIF Test
Collection Site.
This set of RIFtest cases aimsis intended to enable an empirical
investigation of RIF implementations. The testsThey can help identify
problems both with the software developed to implement RIF
specifications and with the specifications themselves. A widely
used test suite providing good coverage of the target RIF dialect
makes it more likely that different implementations will
interoperate correctly.
These test cases areThe format of the tests is designed to be suitable for use by
RIF implementers in test harnesses. Developers will need to write
their own test harnesses, and are encouraged to report their
results as well as any problems they encounter with the tests.
If additional test cases are contributed to the W3C, the Consortium may add them to the set of RIF test cases.
Editor's Note: We may set up a publicly accessible collection site.
This first Working Draft documents test cases for the Basic Logic Dialect (BLD) of RIF, and
for RDF-BLDBLD-RDF and OWL-BLDBLD-OWL combinations as specified in RIF RDF and OWL Compatibility. Future
drafts of this document will cover test cases for other RIF
dialects as those dialects are finalized. The current set of tests
focussesfocuses on testing entailment and syntax, as described in the
Test Types section.
We expect to addAdditional types of tests may be added in the future.
The BLD test suite contains tests for RIF-BLD documents andRIF consumers as defined
in the RIF-BLD
Conformance Clauses, but not for RIF-BLD-producers. TheProducer
tests are designed to: aid in conformance evaluation by providing evidence that the specification has been implemented provide generally broad coverage of thenot included since producers translate from a particular
rule language features focus on non-obvious features and behavior,into RIF, and hard to implement features, since these types ofspecifying tests are more likelyto uncover problems in implementations. pinpoint omissions that canverify this
translation without referring to specific rule languages would be
corrected. reflectdifficult. However, partial syntactic validation of documents
generated by producers can be performed using the resolutionRIF-BLD XML
Schema. For implementations that are both producers and
consumers, some validation of technical issues raisedthe producer can be achieved by
translating entailment tests from RIF to the working grouptarget rule language,
then back to RIF, then again to the target rule language, and then
performing the entailment test.
The tests are designed to:
The test cases dosuite is not constitute a complete conformance test suite.exhaustive: it does not completely cover the
RIF-BLD specification.
Conformance is defined as the fulfillment of specified
requirements, which are detailed in the conformance clause of a
specification. (See QA Framework
specification Guide.) Conformant RIF-BLD documents, consumers
and producers are defined in the conformance
clauses section of the RIF BLD specification. A complete
conformance test suite would be a set of test cases that cover this
set of requirements, such that passing all the tests in the suite
indicates conformance. However, the development of this type of
comprehensive test suite is beyond the scope of the Working Group,
and the RIF test cases aredo not constitute a complete conformance
test suite since they do not completely cover the language.suite. Failure to pass all the tests in the suite indicates
that the implementation does not meet the relevant specification.
But passing all the tests in the suite indicates only that the
implementation is conformant to that particular version of the test
suite.
The RIF test suites include the following deliverables:package consists of:
This section introduces a categorization of allthe tests included
in the test cases, similar to what is currently contained at http://www.w3.org/2005/rules/wiki/Category:Test_Case . 3suite. Each test Types 3.1 Entailment Tests An Entailmentcase describes inputs that can be
provided to a RIF processor, and specifies the behavior required to
satisfy the conformance conditions in that situation. There are
several different types of test iscases detailed in the following
sub-sections. The type of test determines the task, associated
inputs, and expected outcome of the test.
Syntactic tests validate a RIF processor's recognition of a
conformant BLD document in XML syntax [RIF-BLD]. A RIF processor can use XML
Schema validation against the RIF-BLD Schemas
to verify that input documents are valid BLD documents, and so most of the syntax tests
included in the test Tsuite address syntax requirements that are not
enforced by the XML schema.
These tests involve a single RIF document, indicated by the
inputDocument element in whichthe
manifest for the test. The document is a conformant RIF conditiondocument in
the dialect indicated by the dialect element.
The test is either entailed (forpassed if the processor indicates that the document
is syntactically correct, and failed otherwise.
If the value of the dialect property is rifTest:BLD then the test document is a conformant BLD document XML syntax.
Note that the premises of all positive Entailment Tests) or not entailed (forand negative entailment
Tests)tests (defined below) are conformant RIF documents and so can be
used as positive syntax tests.
These tests involve a single RIF document, indicated by the
imports closureinputDocument element in the
manifest for the test. The document is not a conformant RIF
document in the dialect indicated by the dialect element.
The test is passed if the processor indicates that the document is syntactically incorrect, and failed otherwise.
If the value of the dialect property is rifTest:BLD then the test document is not a conformant RIF-BLD document
These tests validate a givenRIF document.processor's computation of the
entailment relation. Each entailment test Thas anone or more
associated dialect Ddialects, and is of the general form
Where R is a RIF document, which possibly imports other documents, andC is a
RIF condition, an RDF document of the dialect D such that either there isor an indication that T is a positive entailment testOWL document, and the imports
closure of R entails C in the dialect D or there is an indication that T is a negative(for positive entailment
test and Rtests) or does not entail (for negative entailment tests)
C in the dialect D . Editor's Note: We currently have a test case (RDF_Combination_Constant_Equivalence_Graph_Entailment) wheregiven dialects. The conclusion is RDF. 3.1.1 Positive Entailmentcomplete specification
of these tests A Positive Entailment testis passed ifgiven in the conclusion document is entailed bysub-sections below.
Editor's Note: In
the imports closure offuture, the premises document.test suite may include tests for
BLDΤ,Ε consumers, which include datatypes and
external terms not included in RIF-DTB
Each test of this type includes a premises document, indicated by the premisesDocument element in the manifest for the test, a conclusion document, indicated by the conclusionDocument element, and optionally one or more documents in the imports closure of the premises document, indicated by importedDocument elements.
The intendedapplicable entailment regime isregimes are indicated by thedialect elementelements in the manifest
file. For example, if thethere is a dialect property with a value of
this property is "BLD""rifTest:BLD" then the entailment holds according to the RIF-BLD
definition of entailment.
In addition, optional importSupport elements in the manifest indicate the types of
documents in the imports closure of the premises document. If an
importSupport property is present with the value "RDF"rifTest:RDF
then the test constitutes a RIF-RDF combination as defined in
RIF RDF and OWL compatibility
and there are additional rules of entailment in effect. If an importSupport
property is present with the value "OWL-DL"rifTest:OWL-DL or
"OWL-Full"rifTest:OWL-Full then the test constitutes a RIF-OWL
combination and the associated entailment rules are defined in
OWL
compatibility.
Note that, in generalgeneral, the conclusion will not include
everything that is entailed by the premises.
Editor's Note: Do we want to address BLD Τ,Ε consumers, and talk about the set DTS, about datatype support? 3.1.2 Negative Entailment TestsA Negative Entailment testconformant RIF consumer should report that the conclusion is
passed ifentailed by the premises, should not report that the answer is
unknown, and must not report that the conclusion documentis not entailed by
the imports closure of the premises document.premises.
Each test of this type includes a premises document, indicated by the premisesDocument element in the manifest for the test, a conclusion document, indicated by the conclusionDocument element, and optionally one or more documents in the imports closure of the premises document, indicated by importedDocument elements.
Editor's Note: The RDF test document has this statement - do we want something similar? "The test is failed if the conclusion can be drawn from the premises ...The test is considered to be passed when it can be conclusively demonstrated that the conclusion cannot be drawn. In practice, the test may be considered to be passed when a thorough attempt to fail the test is unable to achieve failure."The intendedapplicable entailment regime isregimes are indicated by thedialect elementelements in the manifest
file. For example, if thethere is a dialect property with a value of
this property is "BLD""rifTest:BLD" then the entailment does not hold according to the
RIF-BLD
definition of entailment.
In addition, optional importSupport elements in the manifest indicate the types of
documents in the imports closure of the premises document. If an
importSupport property is present with the value "RDF""rifTest:RDF" then
the test constitutes a RIF-RDF combination as defined in RIF RDF and OWL compatibility ; consequently,and there
are additional rules of entailment in effect. If an importSupport
property is present with the value "OWL" then the test constitutes a RIF-OWL combination; the associated entailment rules are defined in OWL compatibility . 3.2 Syntactic Tests These tests check for syntactic validity that cannot be checked by XML Schema validation. 3.2.1 Positive Syntax Tests These tests involve a single RIF document, indicated by the testDocument element in the manifest for the test. The document is a conformant RIF document in the dialect indicated by the dialect element. If the value of the dialect property is "BLD" then the test document is a conformant RIF-BLD document 3.2.2 Negative Syntax Tests These tests involve"rifTest:OWL-DL" or
"rifTest:OWL-Full" then the test constitutes a single RIF document, indicated byRIF-OWL combination
and the testDocument elementassociated entailment rules are defined in OWL
compatibility.
A conformant RIF consumer should report that the manifest forconclusion is
not entailed by the test.premises, should not report that the documentanswer is
unknown, and must not a conformant RIF document inreport that the dialect indicatedconclusion is entailed by the
dialect element. Ifpremises.
Note that while ideally the value ofRIF consumer would be able to
conclusively demonstrate that the dialect property is "BLD" thenconclusion cannot be drawn from
the test document is notpremises, in practice a conformant RIF-BLD documentfailure to draw the conclusion after a
thorough attempt to do so can be considered a successful
outcome.
Editor's Note: The test case format is still under discussion and is likely to undergo some change
Each test case has an associated manifest file that contains
information needed to properly execute the tests.test. This file is in a
machine-readable format, RDF/XML, in order to enable the
development of automated testing frameworks. The manifest file also
contains information, such as contributor and status, used for test
case management. Some examples of how the data in the manifest filecan
be used are:
The format of the RIF test case manifest files is based onmanifests follows the general
guidelines of the W3C QA Working Group's [Test Metadata Note ] and the schema that was developed to accompany it.]. A complete
description ofschema for the manifest filetest case format is given in anAppendix 8. This section
presents a summary of the metadataproperty elementselements, organized by the
type of test to which they
apply.
In the following, dc is bound to http://purl.org/dc/elements/1.1/, and rifTest is bound to http://www.w3.org/2005/rules/test/rifTestSchema#.
The value of this property is a unique identifier for the test case. It conforms to irelative-ref as defined in [RFC-3987] so that it can be appended to a base to generate a URI for the test. Values of this property are of type xs:string.
This is a required element, and there is exactly one for each test.
The value of this property is included below, organized bythe typestatus of the test case
according to which they apply. Inthe following, dctest case approval process. Values of this
property are one of an enumerated list: rifTest:Proposed,
rifTest:Accepted, or rifTest:Rejected.
This is bound to http://purl.org/dc/elements/1.1/ ,a required element, and rifTestthere is boundexactly one for each
test.
This property is used to http://www.w3.org/2005/rules/test/rifTestSchema# . 4.1 Properties of all tests 4.1.1 dc:title Descriptionindicate the name ofRIF dialects that the test
Cardinalityapplies to. There is one Format plain text, no blanks 4.1.2:dialect element for each dialect
that the test is valid for. The possible dialect values are
rifTest:BLD.
This is a required element, and there are one or more for each
test.
DescriptionThe value of this property is a brief explanation of what is being tested,the purpose
of the test, suitable for display in tabular form Cardinalityformat. Values of this
property are of type xs:string.
This is a required element, and there is exactly one Format Free-form (hypertext) or constrained plain text 4.1.3for each
test.
DescriptionA more detailed explanation, where appropriate, about the nature
or characteristics of the test.
Cardinality zero or one Format Free-form (hypertext) 4.1.4 rifTest:dialect Description indicates the RIF dialect thatThis test applies to. Cardinality one -or- one or more Format one of "rifTest:BLD" Editor's Note: BLD could mean BLDis an optional element, and every dialect that it extends,there is zero or we could include an explicit entryone for each
dialect that the test applies to 4.1.5test.
Description Identification ofa portionURI reference to section(s) of a specificationRIF specification(s) relevant
to this test. CardinalityValues of this property are of type rdfs:Literal.
This is an optional element, and there are zero or more Format Free-form (hypertext) or an HTML link to the relevant portion offor each
test.
a specification. 4.1.6 rifTest:status Description One of an enumerated list of values that indicatesURL reference to material related to this test. This property
is used to indicate other RIF test cases, issues from the state ofRIF
issues list, or in general other documents that provide useful
background information for the test. Cardinality one Format oneValues of "rifTest:Proposed", "rifTest:Accepted", "rifTest:Rejected" , "rifTest:Postponed" "rifTest:Obsolete" 4.1.7 dc:contributor Description The individual that contributedthis test. Cardinality oneproperty are of
type rdfs:Literal.
This is an optional element, and there are zero or more Formatfor each
test.
This property indicates a language featurethe individual that contributed the
test. This testis related to Cardinality zero or more Format URI 4.1.9 rifTest:relatedIssue Descriptiona reference to the associated issue on the [RIF issues list] Cardinality zerorequired element, and there are one or more Format URI 4.1.10 rifTest:identifier Description An umambiguous reference to the test Cardinality one Format URIfor
each test.
This property provides a reference to the input document that participates infor the
test Cardinality one Format URItest. Subproperties of rifTest:testDocumentof :inputDocument are used to provide testDocumentsreference
the input document in alternativedifferent syntaxes.
There is one inputDocument for each test.
This property provides a reference to the premises document for
the test Cardinalitytest. Subproperties of :premisesDocument are used to
reference the premises document in different syntaxes.
There is one Format URIpremises document for each test.
Description The file containingThis property provides a reference to the conclusion document
for the test Cardinalitytest. Subproperties of :conclusionDocument are used to
reference the conclusion document in different syntaxes.
There is one Format URIconclusion document for each test.
DescriptionThis property provides a filereference to a document imported by one
of the other files participating indocuments of the test Cardinality zero or more Format URItest. Subproperties
of rifTest:importedDocumentof :importedDocument are used to provide importedDocuments in
alternativedifferent syntaxes.
There can be zero or more imported documents for each test.
Description IndicatesThis property is used to indicate what type of imported
documents this test includes. This is intended for use in filtering for applicable tests. The idea is that if there are multiple imported documents of the same type, there would onlycan be one of theseused in filtering for
those files.applicable tests. If all the imported files are BLD andof the same type as
the dialect is BLD,of the test, this element wouldis not bepresent. Cardinality zero or more FormatThe value of
this property is one of an enumerated list: rifTest:RDF,
rifTest:OWL-DL, rifTest:OWL-Full
This is an optional element, and there are zero or more for each test.
Test suites can be downloaded from the [Test Case Repository [ REPOSITORY] has subdirectoriesas zip
files that contain all the approved tests for a RIF dialect. Each
zip file expands into subdirectories, one for each test, containing
a singlethe test case.input files and individual manifest files. In addition,
each test case consists ofsuite is available as a manifestsingle RDF file describingcontaining all
the test,tests for the normative files participatingsuite, each in the test, and additional non-normative formats of theformat described in Test files whichCase Format. Test cases are
included in order to aidalso available individually in understandingthe test. Tests can be viewed classifiedrepository, organized into
directories by test type, issue, dialect, and feature.dialect.
Each of these testsThe test suite is intended to be runfor use on differenta variety of rule systems,
each with its own API. Therefore, the tasks of providing input to
the rule engine in a system-dependent manner and of checking that
the results are correct are thereforeleft to the tester. The manifest contains a
machine-readable description of the test cases in RDF/XML,cases, and is intended to
assist inenable automated processing of the test cases.
ImplementersUsers are expected to write their own test harnesses. Each test harness shouldharnesses to carry
out the following tasks:
The dialectrifTest:dialect and importSupportrifTest:importSupport
elements in the manifest file can be used to filter for appropriate
tests. In the current set of test cases, all tests are in the
BLDrifTest:BLD dialect. Implementations that do not support BLD-RDF
combinations should skip tests that have an importSupport:importSupport
property with a value of RDF.of :RDF. Implementations that do not
support BLD-OWL combinations should skip tests that have an
importSupport:importSupport property with a value of OWL-DL or OWL-Full.of :OWL-DL
or :OWL-Full.
Testers have the option of making a number of well-definedcertain changes to the test
cases in order to facilitate use of the test suite. All changes
made to the original test suite should be documented in free-text
form as part of the results submission. Acceptable changes are
described below.
Editor's Note: The possibility here isFor tests with input documents that contain import directives,
the location of the imported document specified in the tests provided, import directives point to files ondirective
will be at the W3C website, but people may wantand the imported document will be made
available at that location. Test cases will also include all
imported documents as part of their data (in the zip file), pointed
to customizeby elements in the manifest. Users who wish to access these
files in their own environment.their own environment can change the import directives in
the input documents to reflect the updated location.
Implementers are encouraged to send their test results to public-rif-comments@w3.org.
If you believe that a test case is incorrect --- e.g., the test input is underspecified, or ambiguous, or the conclusion should not hold ---we encourage you to
enter a bug report in the W3C public version of Bugzilla, http://www.w3.org/Bugs/Public/.
To enter a bug reportYou will need a Bugzilla signon (email address and password).password), which
you can create this signonby following the Open a new Bugzilla account
link on the Bugzilla Main Page.
General information on Bugzilla, an open-source bug tracking system, is available at http://www.bugzilla.org/. The standard documentation for the version currently being run on the W3C server is available at http://www.bugzilla.org/docs/2.20/html/.
To enter a bug report in Bugzilla, you will need to log in,
choose the action "New","New", and then choose the "RIF"RIF Test Cases"Cases"
product. Please be specific about the problem that you are
reporting. Clearly identify the test case or test cases that are
problematic. If you believe that the expected result is wrong, then
tell us what you believe the expected result should be. IfPlease cite
portions of one or more of the RIF specifications to support your
position whenever possible.
The schema for the RIF Test Case manifest files is shown below
and is also available as rifTestSchemarifTestSchema.rdf
in the RIF Test Cases Repository.