Difference between revisions of "Conformance"

From OWL
Jump to: navigation, search
(Conformance (Normative))
(tests now maintained externally in two repositories, updated/extended most texts, extended explanation of test types)
Line 5: Line 5:
 
: [http://www.clarkparsia.com/about/profiles/msmith Michael Smith], Clark & Parsia
 
: [http://www.clarkparsia.com/about/profiles/msmith Michael Smith], Clark & Parsia
 
: [http://web.comlab.ox.ac.uk/people/ian.horrocks/ Ian Horrocks], Oxford University
 
: [http://web.comlab.ox.ac.uk/people/ian.horrocks/ Ian Horrocks], Oxford University
 +
: [http://korrekt.org/ Markus Krötzsch], FZI Karlsruhe
 
; Contributors
 
; Contributors
 
: {{Contributors_note}}
 
: {{Contributors_note}}
 
</div>
 
</div>
 
; Abstract:
 
; Abstract:
: <div id='abstract'>{{:Shared_Abstract}}<br />This document describes the conditions that OWL 2 tools must satisfy in order to be conformant with the language specification. It also presents a set of tests that both illustrate the features of the language and can be used for testing conformance.</div>
+
: <div id='abstract'>{{:Shared_Abstract}}<br />This document describes the conditions that OWL 2 tools must satisfy in order to be conformant with the language specification. It also presents a common format for OWL 2 test cases that both illustrate the features of the language and can be used for testing conformance.</div>
 
; Status of this Document
 
; Status of this Document
  
Line 40: Line 41:
 
== Introduction ==
 
== Introduction ==
  
This document describes conformance conditions for OWL 2, and presents a set of test cases.
+
This document describes conformance conditions for OWL 2, and introduces the format of OWL 2 test cases that are provided as part of the OWL 2 Test Case Repository [<cite>[[#ref-tcrepo|OWL 2 Test Cases]]</cite>].
  
 
Conformance conditions are described for both OWL 2 documents and for tools that process such documents. In particular, the conformance conditions for an OWL 2 entailment checker are described in some detail.
 
Conformance conditions are described for both OWL 2 documents and for tools that process such documents. In particular, the conformance conditions for an OWL 2 entailment checker are described in some detail.
  
The presented test cases are intended to illustrate the various features of the language, and can also be used for testing conformance. They are not exhaustive, however, nor do they constitute a conformance test suite for OWL: passing all the tests is no guarantee that a given system conforms to the OWL 2 specification.
+
Moreover, a categorization and common format of test cases is presented. The purpose of test cases is to illustrate various features and to help in testing conformance. The provided set of test cases is not exhaustive, however, nor does it constitute a conformance test suite for OWL: passing all the tests is no guarantee that a given system conforms to the OWL 2 specification. The presented format is intended to facilitate the use of tests by OWL system developers, e.g., in a test harness, as well as the extension of the test suite with new tests.
  
In addition to the tests themselves, a categorization of test types and a format for test cases is also presented. This is intended to facilitate their use by OWL system developers, e.g., in a test harness, and the extension of the test suite with new tests.
+
This document does not contain test cases. Test cases that have been approved by the Working Group are found in the OWL 2 Test Case Repository [<cite>[[#ref-tcrepo|OWL 2 Test Cases]]</cite>], and a public test case repository [<cite>[[#ref-tcpubrepo|Contributed Test Cases]]</cite>] is provided as a platform for collecting further test cases even after the termination of the Working Group.
  
 
The italicized keywords <em title="MUST in RFC 2119 context" class="RFC2119">MUST</em>, <em title="MUST NOT in RFC 2119 context" class="RFC2119">MUST NOT</em>, <em title="SHOULD in RFC 2119 context" class="RFC2119">SHOULD</em>, <em title="SHOULD NOT in RFC 2119 context" class="RFC2119">SHOULD NOT</em>, and <em title="MAY in RFC 2119 context" class="RFC2119">MAY</em> are used to specify normative features of OWL 2 documents and tools, and are interpreted as specified in <nowiki>RFC 2119</nowiki> [<cite>[[#ref-rfc-2119|RFC 2119]]</cite>].
 
The italicized keywords <em title="MUST in RFC 2119 context" class="RFC2119">MUST</em>, <em title="MUST NOT in RFC 2119 context" class="RFC2119">MUST NOT</em>, <em title="SHOULD in RFC 2119 context" class="RFC2119">SHOULD</em>, <em title="SHOULD NOT in RFC 2119 context" class="RFC2119">SHOULD NOT</em>, and <em title="MAY in RFC 2119 context" class="RFC2119">MAY</em> are used to specify normative features of OWL 2 documents and tools, and are interpreted as specified in <nowiki>RFC 2119</nowiki> [<cite>[[#ref-rfc-2119|RFC 2119]]</cite>].
Line 138: Line 139:
  
 
{{EdNote|[[User:MikeSmith|MikeSmith]] 00:55, 22 September 2008 (UTC)|Presently, the content in this document describing test cases is less mature than the content describing conformance.  Among other considerations to be addressed, the test types and formats described below may change, and the manner in which the final test cases will be published remains undecided.}}
 
{{EdNote|[[User:MikeSmith|MikeSmith]] 00:55, 22 September 2008 (UTC)|Presently, the content in this document describing test cases is less mature than the content describing conformance.  Among other considerations to be addressed, the test types and formats described below may change, and the manner in which the final test cases will be published remains undecided.}}
 +
 +
This section introduces various types of test cases. Each test case describes certain inputs that can be provided to OWL 2 tools, and it specifies the behavior required to satisfy the conformance conditions in this situation. Test cases adhere to a common format that simplifies automatic processing, e.g. in a test harness, which is detailed below.
 +
 +
Concrete sets of test cases can be found in various repositories as described below. They are divided into a fixed set of test cases that have been approved based on a process defined later in this section, and an open set of user-contributed test cases that can be collected via a dedicated web site.
  
 
=== Test Types ===
 
=== Test Types ===
  
There are several types of tests detailed in the following sub-sections.
+
There are several distinguished types of test cases detailed in the following sub-sections. The type of a test determines the task and expected outcome of the test. The type thus also affects the data associated to a test case, e.g. since only certain kinds tests require the specification of an entailed ontology.
  
The relationship between a test case and test types is not functional, a specific test may meet the requirements of multiple test types.  In such cases, the test case may be used as described for each of the types. For example, an entailment test is also a consistency test and may be used as either or both, depending on the tool being validated and the goal of validation.
+
While all test cases have some primary purpose specified by their type, it is often possible to use the provided data for other tests as well. For example, the inputs of any negative entailment test can also be used in a consistency test. Such re-interpretations of test cases can generally be useful, depending on the tool being validated and the goal of validation. For this reason, a concrete test case may have more than one type and thus allow multiple uses.
  
 
==== Syntactic Tests ====
 
==== Syntactic Tests ====
 +
 +
Syntactic tests can be applied to OWL tools that process OWL documents, or that transform syntactically between various syntactic forms of OWL.
  
 
===== Profile Identification Tests =====
 
===== Profile Identification Tests =====
Line 156: Line 163:
  
 
==== Semantic Tests ====
 
==== Semantic Tests ====
 +
 +
Semantic tests specifically address the functionality of [[#Entailment Checker|OWL entailment checkers]]. Each test case of this type specifies necessary requirements that must be satisfied by any entailment checker that meets the according conformance conditions.
 +
 +
Each semantic test cases also specifies to whether it is applicable to the [<cite>[[#ref-owl-2-semantics|OWL 2 Direct Semantics]]</cite>], to the [<cite>[[#ref-owl-2-rdf-semantics|OWL 2 RDF-Based Semantics]]</cite>], or to both. A test is only relevant for testing conformance of tools that use a semantics to which the test applies.
  
 
===== Entailment Tests =====
 
===== Entailment Tests =====
  
Entailment tests validate the conformance of an [[#Entailment Checker|Entailment Checker]].  These tests require at least two input ontologies, a premise ontology and a conclusion ontology. A conformant entailment checker will return True if provided the import closure of the premise ontology and the import closure of the conclusion ontology as inputs ''O<sub>1</sub>'' and ''O<sub>2</sub>'', respectively.
+
Entailment tests (or positive entailment tests) specify two ontologies: a premise ontology ''O<sub>1</sub>'' and a conclusion ontology ''O<sub>2</sub>'' where ''O<sub>1</sub>'' entails ''O<sub>2</sub>'' with respect to the specified semantics. If provided with inputs ''O<sub>1</sub>'' and ''O<sub>2</sub>'' (and, if applicable, with access to any imported ontologies), a conforming entailment checker should return True, it should not return Unknown, and it must not return False.
 +
 
 +
In all entailment tests, the ontologies ''O<sub>1</sub>'' and ''O<sub>2</sub>'' are consistent. Therefore, all entailment tests are also consistency tests.
 +
 
 +
{{EdNote|[[User:Markus Krötzsch|Markus Krötzsch]] 17:43, 7 November 2008 (UTC)|The current format disallows tests validating that inconsistent ontologies must entail anything. This could be changed.}}
  
 
===== Non-Entailment Tests =====
 
===== Non-Entailment Tests =====
  
Entailment tests validate the conformance of an [[#Entailment Checker|Entailment Checker]].  These tests require at least two input ontologies, a premise ontology and a non-conclusion ontology. A conformant entailment checker will return False if provided the import closure of the premise ontology and the import closure of the non-conclusion ontology as inputs ''O<sub>1</sub>'' and ''O<sub>2</sub>'', respectively.
+
Non-Entailment tests (or negative entailment tests) specify two ontologies: a premise ontology ''O<sub>1</sub>'' and a non-conclusion ontology ''O<sub>2</sub>'' where ''O<sub>1</sub>'' does not entail ''O<sub>2</sub>'' with respect to the specified semantics. If provided with inputs ''O<sub>1</sub>'' and ''O<sub>2</sub>'' (and, if applicable, with access to any imported ontologies), a conforming entailment checker should return False, it should not return Unknown, and it must not return True.
 +
 
 +
In all non-entailment tests, the ontologies ''O<sub>1</sub>'' and ''O<sub>2</sub>'' are consistent. Therefore, all entailment tests are also consistency tests.
  
 
===== Consistency Tests =====
 
===== Consistency Tests =====
  
Consistency tests validate a tool's recognition of consistency, as defined in the [<cite>[[#ref-owl-2-semantics|OWL 2 Direct Semantics]]</cite>] and the [<cite>[[#ref-owl-2-rdf-semantics|OWL 2 RDF-Based Semantics]]</cite>]. These tests require at least one input ontology, a premise ontology. The import closure of the premise ontology is consistent.
+
Consistency tests validate a tool's recognition of consistency, as defined in the [<cite>[[#ref-owl-2-semantics|OWL 2 Direct Semantics]]</cite>] and the [<cite>[[#ref-owl-2-rdf-semantics|OWL 2 RDF-Based Semantics]]</cite>]. These tests specify an input ontology, a premise ontology ''O'', which is consistent with respect to the specified semantics.  
 +
 
 +
Entailment checkers that directly support consistency checking should determine ''O'' to be consistent, and must not determine ''O'' to be inconsistent. Entailment checkers that do not support this operation may execute consistency tests like non-entailment test: if the ontology ''O'' is consistent, then ''O'' does not entail the inconsistent ontology ''O<sub>in</sub>'' (see [[#Appendix: An Inconsistent Ontology|Appendix]]). Given inputs ''O'' and ''O<sub>in</sub>'', a conforming entailment checker should thus return False, it should not return Unknown, and it must not return True.
  
 
===== Inconsistency Tests =====
 
===== Inconsistency Tests =====
  
Inconsistency tests validate a tool's recognition of consistency, as defined in the [<cite>[[#ref-owl-2-semantics|OWL 2 Direct Semantics]]</cite>] and the [<cite>[[#ref-owl-2-rdf-semantics|OWL 2 RDF-Based Semantics]]</cite>]. These tests require at least one input ontology, a premise ontology. The import closure of the premise ontology is not consistent.
+
Inconsistency tests validate a tool's recognition of consistency, as defined in the [<cite>[[#ref-owl-2-semantics|OWL 2 Direct Semantics]]</cite>] and the [<cite>[[#ref-owl-2-rdf-semantics|OWL 2 RDF-Based Semantics]]</cite>]. These tests specify an input ontology, a premise ontology ''O'', which is inconsistent with respect to the specified semantics.
 +
 
 +
Entailment checkers that directly support inconsistency checking should determine ''O'' to be inconsistent, and must not determine ''O'' to be consistent. Entailment checkers that do not support this operation may execute inconsistency tests like entailment test: if the ontology ''O'' is inconsistent, then ''O'' entails the inconsistent ontology ''O<sub>in</sub>'' (see [[#Appendix|Appendix]]). Given inputs ''O'' and ''O<sub>in</sub>'', a conforming entailment checker should thus return True, it should not return Unknown, and it must not return False.
  
 
=== Test Case Format ===
 
=== Test Case Format ===
Line 576: Line 597:
 
</pre></code>
 
</pre></code>
  
=== The OWL Tests ===
+
=== Test Case Repositories ===
  
==== Index By Type ====
+
A set of approved test cases is provided in the OWL 2 Test Case Repository [<cite>[[#ref-tcrepo|OWL 2 Test Cases]]</cite>]. Those test cases have been collected based on the approval process described below, and they are expected to remain static after the Working Group has finished.
  
===== Profile Identification Tests =====
+
Like any test set, the approved OWL tests are necessarily incomplete in that they cannot cover all relevant situations or possible implementation challenges. For this reason, an additional public test repository [<cite>[[#ref-tcpubrepo|Contributed Test Cases]]</cite>] is provided as a platform for collecting further test cases even after the termination of the Working Group. Since there is no formal approval process for those additional test cases, they are not necessarily subjected to extensive review, possibly resulting in erroneous or misleading information. Yet, it is hoped that the additional repository will provide a valuable tool for the development of OWL after the finalization of the recommendation.
  
{{TestCase query | [[Category:ProfileIdentificationTest]] [[test case status::test:Approved]] }}
+
=== Approval Process Overview ===
  
===== Consistency Tests =====
+
This section outlines the process by means of which test cases have been selected for inclusion into the OWL 2 Test Case Repository [<cite>[[#ref-tcrepo|OWL 2 Test Cases]]</cite>].
  
{{TestCase query | [[Category:ConsistencyTest]] [[test case status::test:Approved]] }}
+
* At the chair's discretion, individual tests or groups of tests are put to the Working Group in the weekly telecon or at a face-to-face meeting.
  
===== Inconsistency Tests =====
+
* The Working Group may approve, reject, or defer decision on a test.
 
+
** If the Working Group approves a test, its status is changed to APPROVED.  All approved and only approved tests are included in the test case repository [<cite>[[#ref-tcrepo|OWL 2 Test Cases]]</cite>].
{{TestCase query | [[Category:InconsistencyTest]] [[test case status::test:Approved]] }}
+
** If the Working Group rejects a test, its status is changed to REJECTED.
 
+
** If the Working Group defers decision on a test,  its status remains PROPOSED.
===== Entailment Tests =====
+
* At the chairs' discretion, the Working Group may review any previous decision regarding any test cases.
 
+
{{TestCase query | [[Category:PositiveEntailmentTest]] [[test case status::test:Approved]] }}
+
 
+
===== Non-Entailment Tests =====
+
 
+
{{TestCase query | [[Category:NegativeEntailmentTest]] [[test case status::test:Approved]] }}
+
 
+
===== Syntax Translation Tests =====
+
 
+
{{TestCase query | [[Category:SyntaxTranslationTest]] [[test case status::test:Approved]] }}
+
 
+
==== Index By Issue ====
+
 
+
==== Index By Profile ====
+
 
+
===== OWL EL =====
+
{{TestCase query | [[test case profile::Test:EL]] [[test case status::test:Approved]] }}
+
 
+
===== OWL RL =====
+
{{TestCase query | [[test case profile::Test:RL]] [[test case status::test:Approved]] }}
+
 
+
===== OWL QL =====
+
{{TestCase query | [[test case profile::Test:QL]] [[test case status::test:Approved]] }}
+
 
+
=== Process Overview ===
+
 
+
At the chair's discretion, individual tests or groups of tests are put to the Working Group in the weekly telecon or at a face-to-face meeting.
+
 
+
The Working Group may approve, reject, or defer decision on a test.
+
 
+
If the Working Group approves a test, its status is changed to APPROVED.  All approved and only approved tests are included in the test case document.
+
 
+
If the Working Group rejects a test, its status is change to REJECTED.
+
 
+
If the Working Group defers decision on a test,  its status remains PROPOSED.
+
 
+
At the chairs' discretion, the Working Group may review any previous decision regarding any test cases.
+
  
 
The Working Group has complete discretion to approve or reject tests independent of their conformance with this process or their conformance with the OWL Working Drafts.
 
The Working Group has complete discretion to approve or reject tests independent of their conformance with this process or their conformance with the OWL Working Drafts.
Line 652: Line 636:
  
 
Status of each test no longer includes "EXTRACREDIT" and "OBSOLETED".
 
Status of each test no longer includes "EXTRACREDIT" and "OBSOLETED".
 +
 +
== Appendix: An Inconsistent Ontology ==
 +
 +
[[#Consistency Tests|Consistency tests]] and [[#Inconsistency Tests|inconsistency tests]] can be considered as entailment tests and non-entailment tests, respectively, by checking the entailment of an (arbitrary) inconsistent ontology. This appendix provides one ontology that can be used for this purpose (under both [<cite>[[#ref-owl-2-semantics|OWL 2 Direct Semantics]]</cite>] and [<cite>[[#ref-owl-2-rdf-semantics|OWL 2 RDF-Based Semantics]]</cite>]), and which is compatible with all profiles:
 +
 +
<code><pre>
 +
Namespace( owl =  &lt;http://www.w3.org/2002/07/owl#&gt; )
 +
Ontology(&lt;http://www.w3.org/2007/OWL/inconsistentOntology&gt;
 +
    SubClassOf( owl:Thing owl:Nothing )
 +
)
 +
</pre></code>
  
 
== References ==
 
== References ==
 +
 +
; <span id="ref-tcpubrepo">[Contributed Test Cases]</span>
 +
: <cite>[http://km.aifb.uni-karlsruhe.de/projects/owltests Public Test Case Repository]</cite>. {{EdNote|[[User:Markus Krötzsch|Markus Krötzsch]] 17:43, 7 November 2008 (UTC)|The final location of the repository of user-contributed test cases is yet to be determined.}}
  
 
; <span id="ref-owl-2-specification">[OWL 2 Specification]</span>
 
; <span id="ref-owl-2-specification">[OWL 2 Specification]</span>
 
: <cite>[[Syntax|OWL 2 Web Ontology Language: Structural Specification and Functional-Style Syntax]]</cite>. Peter F. Patel-Schneider, Ian Horrocks, and Boris Motik, eds., 2008.
 
: <cite>[[Syntax|OWL 2 Web Ontology Language: Structural Specification and Functional-Style Syntax]]</cite>. Peter F. Patel-Schneider, Ian Horrocks, and Boris Motik, eds., 2008.
 +
 
; <span id="ref-owl-2-semantics">[OWL 2 Direct Semantics]</span>
 
; <span id="ref-owl-2-semantics">[OWL 2 Direct Semantics]</span>
 
: <cite>[[Direct_Semantics|OWL 2 Web Ontology Language: Direct Semantics]]</cite>. Bernardo Cuenca Grau and Boris Motik, eds., 2008.
 
: <cite>[[Direct_Semantics|OWL 2 Web Ontology Language: Direct Semantics]]</cite>. Bernardo Cuenca Grau and Boris Motik, eds., 2008.
 +
 
; <span id="ref-owl-2-rdf-semantics">[OWL 2 RDF-Based Semantics]</span>
 
; <span id="ref-owl-2-rdf-semantics">[OWL 2 RDF-Based Semantics]</span>
 
: <cite>[[RDF-Based_Semantics|OWL 2 Web Ontology Language: RDF-Based Semantics]]</cite>. Michael Schneider, ed., 2008.
 
: <cite>[[RDF-Based_Semantics|OWL 2 Web Ontology Language: RDF-Based Semantics]]</cite>. Michael Schneider, ed., 2008.
 +
 
; <span id="ref-owl-mapping">[OWL 2 RDF Mapping]</span>
 
; <span id="ref-owl-mapping">[OWL 2 RDF Mapping]</span>
 
: <cite>[[Mapping to RDF Graphs|OWL 2 Web Ontology Language: Mapping to RDF Graphs]]</cite>. Bernardo Cuenca Grau and Boris Motik, eds., 2008.
 
: <cite>[[Mapping to RDF Graphs|OWL 2 Web Ontology Language: Mapping to RDF Graphs]]</cite>. Bernardo Cuenca Grau and Boris Motik, eds., 2008.
 +
 
; <span id="ref-owl-2-xml-syntax">[OWL 2 XML Syntax]</span>
 
; <span id="ref-owl-2-xml-syntax">[OWL 2 XML Syntax]</span>
 
: <cite>[[XML_Serialization|OWL 2 Web Ontology Language: XML Serialization]]</cite>. Boris Motik and Peter. F. Patel-Schneider, eds., 2008.
 
: <cite>[[XML_Serialization|OWL 2 Web Ontology Language: XML Serialization]]</cite>. Boris Motik and Peter. F. Patel-Schneider, eds., 2008.
 +
 
; <span id="ref-owl-2-profiles">[OWL 2 Profiles]</span>
 
; <span id="ref-owl-2-profiles">[OWL 2 Profiles]</span>
 
: <cite>[[Profiles|OWL 2 Web Ontology Language: Profiles]]</cite>. Boris Motik, Bernardo Cuenca Grau, Ian Horrocks, Zhe Wu, Achille Fokoue and Carsten Lutz, eds., 2008.
 
: <cite>[[Profiles|OWL 2 Web Ontology Language: Profiles]]</cite>. Boris Motik, Bernardo Cuenca Grau, Ian Horrocks, Zhe Wu, Achille Fokoue and Carsten Lutz, eds., 2008.
 +
 +
; <span id="ref-tcrepo">[OWL 2 Test Cases]</span>
 +
: <cite>[http://w3.org OWL 2 Test Case Repository]</cite>. {{EdNote|[[User:Markus Krötzsch|Markus Krötzsch]] 17:43, 7 November 2008 (UTC)|The final location of the repository of approved test cases is yet to be determined.}}
 +
 
; <span id="ref-rdf-syntax">[RDF Syntax]</span>
 
; <span id="ref-rdf-syntax">[RDF Syntax]</span>
 
: <cite>[http://www.w3.org/TR/2004/REC-rdf-syntax-grammar-20040210/ RDF/XML Syntax Specification (Revised)]</cite>. Dave Beckett, Editor, W3C Recommendation 10 February 2004, http://www.w3.org/TR/2004/REC-rdf-syntax-grammar-20040210/ . Latest version available at http://www.w3.org/TR/rdf-syntax-grammar/ .
 
: <cite>[http://www.w3.org/TR/2004/REC-rdf-syntax-grammar-20040210/ RDF/XML Syntax Specification (Revised)]</cite>. Dave Beckett, Editor, W3C Recommendation 10 February 2004, http://www.w3.org/TR/2004/REC-rdf-syntax-grammar-20040210/ . Latest version available at http://www.w3.org/TR/rdf-syntax-grammar/ .
 +
 
; <span id="ref-sroiq">[SROIQ]</span>
 
; <span id="ref-sroiq">[SROIQ]</span>
 
: <cite>[http://www.cs.man.ac.uk/~sattler/publications/sroiq-TR.pdf The Even More Irresistible SROIQ]</cite>. Ian Horrocks, Oliver Kutz, and Uli Sattler. In Proc. of the 10th Int. Conf. on Principles of Knowledge Representation and Reasoning (KR 2006). AAAI Press, 2006.
 
: <cite>[http://www.cs.man.ac.uk/~sattler/publications/sroiq-TR.pdf The Even More Irresistible SROIQ]</cite>. Ian Horrocks, Oliver Kutz, and Uli Sattler. In Proc. of the 10th Int. Conf. on Principles of Knowledge Representation and Reasoning (KR 2006). AAAI Press, 2006.
 +
 
; <span id="ref-owl-metamodeling">[OWL Metamodeling]</span>
 
; <span id="ref-owl-metamodeling">[OWL Metamodeling]</span>
 
: <cite>[http://www.comlab.ox.ac.uk/people/boris.motik/pubs/motik07metamodeling-journal.pdf On the Properties of Metamodeling in OWL]</cite>. Boris Motik. ''Journal of Logic and Computation'', 17(4):617-637, 2007.
 
: <cite>[http://www.comlab.ox.ac.uk/people/boris.motik/pubs/motik07metamodeling-journal.pdf On the Properties of Metamodeling in OWL]</cite>. Boris Motik. ''Journal of Logic and Computation'', 17(4):617-637, 2007.
 +
 
; <span id="ref-rfc-2119">[RFC 2119]</span>
 
; <span id="ref-rfc-2119">[RFC 2119]</span>
 
: <cite>[http://www.ietf.org/rfc/rfc2119.txt RFC 2119: Key words for use in RFCs to Indicate Requirement Levels]</cite>. Network Working Group, S. Bradner. Internet Best Current Practice, March 1997.
 
: <cite>[http://www.ietf.org/rfc/rfc2119.txt RFC 2119: Key words for use in RFCs to Indicate Requirement Levels]</cite>. Network Working Group, S. Bradner. Internet Best Current Practice, March 1997.
 +
 
; <span id="ref-rfc-3987">[RFC-3987]</span>
 
; <span id="ref-rfc-3987">[RFC-3987]</span>
 
: <cite>[http://www.ietf.org/rfc/rfc3987.txt RFC 3987 - Internationalized Resource Identifiers (IRIs)]</cite>. M. Duerst, M. Suignard. IETF, January 2005, http://www.ietf.org/rfc/rfc3987.txt.
 
: <cite>[http://www.ietf.org/rfc/rfc3987.txt RFC 3987 - Internationalized Resource Identifiers (IRIs)]</cite>. M. Duerst, M. Suignard. IETF, January 2005, http://www.ietf.org/rfc/rfc3987.txt.
  
 
</div>
 
</div>

Revision as of 17:43, 7 November 2008

__NUMBEREDHEADINGS__

[Hide Review Comments]

Document title:
OWL 2 Web Ontology Language
Conformance and Test Cases (Second Edition)
Editors
Michael Smith, Clark & Parsia
Ian Horrocks, Oxford University
Markus Krötzsch, FZI Karlsruhe
Contributors
Note: The complete list of contributors is being compiled and will be included in the next draft.
Abstract
The OWL 2 Web Ontology Language, informally OWL 2, is an ontology language for the Semantic Web with formally defined meaning. OWL 2 ontologies provide classes, properties, individuals, and data values and are stored as Semantic Web documents. OWL 2 ontologies can be used along with information written in RDF, and OWL 2 ontologies themselves are primarily exchanged as RDF documents. The OWL 2 Document Overview describes the overall state of OWL 2, and should be read before other OWL 2 documents.
This document describes the conditions that OWL 2 tools must satisfy in order to be conformant with the language specification. It also presents a common format for OWL 2 test cases that both illustrate the features of the language and can be used for testing conformance.
Status of this Document

Copyright © 2008-2009 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark and document use rules apply.

[Show Short TOC]

1 Introduction

This document describes conformance conditions for OWL 2, and introduces the format of OWL 2 test cases that are provided as part of the OWL 2 Test Case Repository [OWL 2 Test Cases].

Conformance conditions are described for both OWL 2 documents and for tools that process such documents. In particular, the conformance conditions for an OWL 2 entailment checker are described in some detail.

Moreover, a categorization and common format of test cases is presented. The purpose of test cases is to illustrate various features and to help in testing conformance. The provided set of test cases is not exhaustive, however, nor does it constitute a conformance test suite for OWL: passing all the tests is no guarantee that a given system conforms to the OWL 2 specification. The presented format is intended to facilitate the use of tests by OWL system developers, e.g., in a test harness, as well as the extension of the test suite with new tests.

This document does not contain test cases. Test cases that have been approved by the Working Group are found in the OWL 2 Test Case Repository [OWL 2 Test Cases], and a public test case repository [Contributed Test Cases] is provided as a platform for collecting further test cases even after the termination of the Working Group.

The italicized keywords MUST, MUST NOT, SHOULD, SHOULD NOT, and MAY are used to specify normative features of OWL 2 documents and tools, and are interpreted as specified in RFC 2119 [RFC 2119].

2 Conformance (Normative)

This section describes conformance conditions for OWL 2 documents and tools. In particular, it describes the syntactic conditions that characterize OWL 2 ontology documents, including those that conform to the various OWL 2 profiles [OWL 2 Profiles], and the syntactic and semantic conditions that must be satisfied by conformant OWL 2 tools.

2.1 Document Conformance

Review comment from AlanRuttenberg AlanRuttenberg 03:27, 6 November 2008 (UTC)
See review comments at http://lists.w3.org/Archives/Public/public-owl-wg/2008Nov/0039.html

Several syntaxes have been defined for OWL 2 ontology documents, some or all of which could be used by OWL 2 tools for exchanging documents. However, conformant OWL 2 tools that take ontology documents as input(s) MUST accept ontology documents using the RDF/XML serialization [OWL 2 Mapping to RDF Graphs], and conformant OWL 2 tools that publish ontology documents MUST be able to publish them in the RDF/XML serialization if asked to do so (e.g., via HTTP content negotiation). OWL 2 tools MAY also accept and/or publish ontology documents using other serializations, for example the XML Serialization [OWL 2 XML Syntax].

2.1.1 Syntactic Conformance

Any RDF document [RDF Syntax] is an OWL 2 Full ontology document.

An OWL 2 Full ontology document is an OWL 2 DL ontology document iff it can be successfully parsed using the canonical RDF parsing process [OWL 2 Mapping to RDF Graphs] and the resulting ontology in the functional-style syntax satisfies all the restrictions on OWL 2 DL ontologies [OWL 2 Specification].

An OWL 2 DL ontology document is an OWL 2 EL ontology document iff it can be successfully parsed using the canonical RDF parsing process [OWL 2 Mapping to RDF Graphs] and the resulting ontology in the functional-style syntax satisfies all the restrictions on OWL 2 EL ontologies [OWL 2 Profiles].

An OWL 2 DL ontology document is an OWL 2 QL ontology document iff it can be successfully parsed using the canonical RDF parsing process [OWL 2 Mapping to RDF Graphs] and the resulting ontology in the functional-style syntax satisfies all the restrictions on OWL 2 QL ontologies [OWL 2 Profiles].

An OWL 2 Full ontology document is an OWL 2 RL ontology document iff it can be successfully parsed using the canonical RDF parsing process [OWL 2 Mapping to RDF Graphs] and the resulting ontology in the functional-style syntax satisfies all the restrictions on OWL 2 RL ontologies [OWL 2 Profiles].

Conformance for other serializations is a direct consequence of the relevant serialization specification, the OWL 2 Syntax specification [OWL 2 Specification] and the OWL 2 Profiles specification [OWL 2 Profiles]. For example, an XML document is an OWL 2 DL ontology document iff it validates against the OWL 2 XML Schema [OWL 2 XML Syntax], and the corresponding functional-style syntax satisfies all the restrictions on OWL 2 DL ontologies [OWL 2 Specification]. It is an OWL 2 EL (respectively QL, RL) ontology document iff the corresponding functional-style syntax also satisfies all the restrictions on OWL 2 EL (respectively QL, RL) ontologies [OWL 2 Profiles].

2.1.2 Datatype Map Conformance

In OWL 2, semantic conditions are defined with respect to a datatype map [OWL 2 Specification]. This MUST be either the OWL 2 Datatype map [OWL 2 Specification], or an extension of the OWL 2 Datatype map to include additional datatypes.

Note that OWL 2 Profiles may support only a reduced set of datatypes. This is, however, a syntactic condition that must be met by documents in order to fall within the relevant profile, and the semantic conditions on the supported datatypes are unchanged, i.e., they are defined by a (possibly extended) OWL 2 Datatype map.

2.2 Document Checker Conformance

An OWL 2 document checker is a tool that takes one or more OWL 2 ontologies and checks some (syntactic or semantic) condition. As noted above, any conformant OWL 2 tool MUST accept ontology documents using the RDF/XML serialisation [OWL 2 Mapping to RDF Graphs]; it MAY also accept ontology documents using other serializations, for example the XML Serialization [OWL 2 XML Syntax].

The conformance conditions related to entailment checking and query answering are defined below. Other OWL 2 tools MUST satisfy similar conditions. In particular, they MUST be consistent with the Direct Semantics [OWL 2 Semantics] and/or the RDF-Based Semantics [OWL 2 RDF-Based Semantics].

2.2.1 Entailment Checker

An OWL 2 entailment checker takes as input two OWL 2 Full, DL, EL, QL, or RL ontology documents O1 and O2 and checks if O1 entails O2 with respect to a datatype map and either the Direct Semantics [OWL 2 Semantics] or the RDF-Based Semantics [OWL 2 RDF-Based Semantics]. Additionally:

  • It MUST provide a means to determine the datatypes supported by its datatype map, and any limits it has on datatype literals and datatype values [OWL 2 Specification]; for example, by listing them in its supporting documentation.
  • It MUST provide a means to determine the semantics it uses (either the Direct Semantics [OWL 2 Semantics] or the RDF-Based Semantics [OWL 2 RDF-Based Semantics]); for example, in its supporting documentation.

An OWL 2 entailment checker returns a single result, being True, False, Unknown or Error. True indicates that the relevant entailment holds; False indicates that the relevant entailment does not hold; Unknown indicates that the algorithm used by the checker is not able to determine if the entailment holds; Error indicates that the checker encountered an error condition such as receiving an invalid input or exceeding resource limits. While sometimes needed (for example, for pragmatic reasons), Unknown and Error are not desired responses for valid inputs.

Additionally, an OWL 2 entailment checker:

  • MUST return Error if the parsing process fails (for example, due to network errors);
  • MUST return Error if an input document uses datatypes that are not supported by its datatype map or literals that it does not support (for example, very large integers); and
  • MUST return Error if the computation fails, for example as a result of exceeding resource limits.

Five different conformance classes of OWL 2 entailment checkers are defined:

An OWL 2 Full entailment checker is an OWL 2 entailment checker that takes RDF documents as input. It MUST return True only when O1 entails O2 with respect to the RDF-Based Semantics [OWL 2 RDF-Based Semantics], and it MUST return False only when O1 does not entail O2 with respect to the RDF-Based Semantics. It SHOULD NOT return Unknown.

An OWL 2 DL entailment checker is an OWL 2 entailment checker that takes OWL 2 DL ontology documents as input. It MUST return True only when O1 entails O2 with respect to the Direct Semantics [OWL 2 Semantics], and it MUST return False only when O1 does not entail O2 with respect to the Direct Semantics. It SHOULD NOT return Unknown.

An OWL 2 EL entailment checker is an OWL 2 entailment checker that takes OWL 2 EL ontology documents as input. It MUST return True only when O1 entails O2 with respect to the Direct Semantics [OWL 2 Semantics], and it MUST return False only when O1 does not entail O2 with respect to the Direct Semantics. It SHOULD NOT return Unknown.

An OWL 2 QL entailment checker is an OWL 2 entailment checker that takes OWL 2 QL ontology documents as input. It MUST return True only when O1 entails O2 with respect to the Direct Semantics [OWL 2 Semantics], and it MUST return False only when O1 does not entail O2 with respect to the Direct Semantics. It SHOULD NOT return Unknown.

An OWL 2 RL entailment checker is an OWL 2 entailment checker that takes RDF documents as input. It MUST return True only when O1 entails O2 with respect to the RDF-Based Semantics [OWL 2 RDF-Based Semantics], and it MUST return False only when O1 does not entail O2 with respect to the RDF-Based Semantics; it MAY return Unknown if FO(O1)R does not entail FO(O2) under the standard first-order semantics, where R denotes the OWL 2 RL/RDF rules [OWL 2 Profiles], and FO(O) denotes the FO theory corresponding to O in which triples are represented using the T predicate — that is, T(s, p, o) represents an RDF triple with the subject s, predicate p, and the object o.

Editor's Note: It has been suggested that we should define two conformance classes for OWL 2 RL entailment checkers: one as above (an RDF entailment checker) and one that only operates on the RL syntactic fragment using the Direct Semantics.

An OWL 2 entailment checker is terminating if, given sufficient resources (memory, addressing space, etc.), it will always return True, False, or Unknown in a finite amount of time (i.e., CPU cycles) on syntactically-valid inputs.

An OWL 2 entailment checker is complete if, given sufficient resources, it will always return True or False on syntactically-valid inputs.

Note: Every OWL 2 Full entailment checker is also an OWL 2 RL entailment checker.

Note: Every OWL 2 DL entailment checker is also an OWL 2 EL and OWL 2 QL entailment checker.

Note: From Theorem 1 of Profiles [OWL 2 Profiles], it follows that an OWL 2 RL entailment checker can return False if O1 and O2 satisfy the constraints described in Theorem 1, and FO(O1)R does not entail FO(O2) under the standard first-order semantics, where R denotes the OWL 2 RL/RDF rules [OWL 2 Profiles], and FO(O) denotes the FO theory corresponding to O in which triples are represented using the T predicate — that is, T(s, p, o) represents an RDF triple with the subject s, predicate p, and the object o. Implementations not wishing to check whether O1 and O2 satisfy the relevant constraints can simply return Unknown whenever they are not able to return True.

2.2.2 Query Answering

Query answering is closely related to entailment checking. A query can be thought of as an ontology Q in which some of the terms have been replaced by variables x1, ..., xn. Given an ontology O, a tuple t = <t1, ..., tn> is an answer for Q with respect to O if O entails Q[x/t], where Q[x/t] is derived from Q by substituting the variables x1, ..., xn with t1, ..., tn; the answer to Q with respect to O is the set of all such tuples.

Although highly inefficient in practice, query answering could be performed simply by iterating through all possible n-tuples formed from terms occurring in O and checking the corresponding entailment using an OWL 2 entailment checker. The properties of OWL 2 entailment checkers mean that the resulting answer will always be sound, i.e., every tuple occurring in the answer set is an answer to the query. If any one of the entailment checks might return Unknown, then the answer to the query may be incomplete, i.e., there may exist a tuple t that is an answer to the query but that does not occur in the answer set; implementations SHOULD issue a warning in this case.

The properties of OWL 2 Full, DL, EL and QL entailment checkers mean that query answering SHOULD be both sound and complete. In the case of OWL RL, query answering SHOULD be sound, and will also be complete if both the ontology and the query satisfy the constraints described in Theorem 1.

3 Test Cases

Editor's Note: Presently, the content in this document describing test cases is less mature than the content describing conformance. Among other considerations to be addressed, the test types and formats described below may change, and the manner in which the final test cases will be published remains undecided.

This section introduces various types of test cases. Each test case describes certain inputs that can be provided to OWL 2 tools, and it specifies the behavior required to satisfy the conformance conditions in this situation. Test cases adhere to a common format that simplifies automatic processing, e.g. in a test harness, which is detailed below.

Concrete sets of test cases can be found in various repositories as described below. They are divided into a fixed set of test cases that have been approved based on a process defined later in this section, and an open set of user-contributed test cases that can be collected via a dedicated web site.

3.1 Test Types

There are several distinguished types of test cases detailed in the following sub-sections. The type of a test determines the task and expected outcome of the test. The type thus also affects the data associated to a test case, e.g. since only certain kinds tests require the specification of an entailed ontology.

While all test cases have some primary purpose specified by their type, it is often possible to use the provided data for other tests as well. For example, the inputs of any negative entailment test can also be used in a consistency test. Such re-interpretations of test cases can generally be useful, depending on the tool being validated and the goal of validation. For this reason, a concrete test case may have more than one type and thus allow multiple uses.

3.1.1 Syntactic Tests

Syntactic tests can be applied to OWL tools that process OWL documents, or that transform syntactically between various syntactic forms of OWL.

3.1.1.1 Profile Identification Tests

Profile identification tests validate a tool's recognition of Syntactic Conformance. These tests require at least one input ontology. Each test describes the conformance of all input ontologies relative to structural and syntactic restrictions.

3.1.1.2 Syntax Translation Tests

Syntax translation tests validate the translation of OWL ontologies from one syntax to another, using the definition of structural equivalence defined in [OWL 2 Syntax]. Each input ontology used by a syntax translation test should be provided in multiple syntaxes such that the ontology produced by parsing any one serialization of an input ontology is structurally equivalent with the ontology produced by parsing any other serialization of the same input ontology.

3.1.2 Semantic Tests

Semantic tests specifically address the functionality of OWL entailment checkers. Each test case of this type specifies necessary requirements that must be satisfied by any entailment checker that meets the according conformance conditions.

Each semantic test cases also specifies to whether it is applicable to the [OWL 2 Direct Semantics], to the [OWL 2 RDF-Based Semantics], or to both. A test is only relevant for testing conformance of tools that use a semantics to which the test applies.

3.1.2.1 Entailment Tests

Entailment tests (or positive entailment tests) specify two ontologies: a premise ontology O1 and a conclusion ontology O2 where O1 entails O2 with respect to the specified semantics. If provided with inputs O1 and O2 (and, if applicable, with access to any imported ontologies), a conforming entailment checker should return True, it should not return Unknown, and it must not return False.

In all entailment tests, the ontologies O1 and O2 are consistent. Therefore, all entailment tests are also consistency tests.

Editor's Note: The current format disallows tests validating that inconsistent ontologies must entail anything. This could be changed.
3.1.2.2 Non-Entailment Tests

Non-Entailment tests (or negative entailment tests) specify two ontologies: a premise ontology O1 and a non-conclusion ontology O2 where O1 does not entail O2 with respect to the specified semantics. If provided with inputs O1 and O2 (and, if applicable, with access to any imported ontologies), a conforming entailment checker should return False, it should not return Unknown, and it must not return True.

In all non-entailment tests, the ontologies O1 and O2 are consistent. Therefore, all entailment tests are also consistency tests.

3.1.2.3 Consistency Tests

Consistency tests validate a tool's recognition of consistency, as defined in the [OWL 2 Direct Semantics] and the [OWL 2 RDF-Based Semantics]. These tests specify an input ontology, a premise ontology O, which is consistent with respect to the specified semantics.

Entailment checkers that directly support consistency checking should determine O to be consistent, and must not determine O to be inconsistent. Entailment checkers that do not support this operation may execute consistency tests like non-entailment test: if the ontology O is consistent, then O does not entail the inconsistent ontology Oin (see Appendix). Given inputs O and Oin, a conforming entailment checker should thus return False, it should not return Unknown, and it must not return True.

3.1.2.4 Inconsistency Tests

Inconsistency tests validate a tool's recognition of consistency, as defined in the [OWL 2 Direct Semantics] and the [OWL 2 RDF-Based Semantics]. These tests specify an input ontology, a premise ontology O, which is inconsistent with respect to the specified semantics.

Entailment checkers that directly support inconsistency checking should determine O to be inconsistent, and must not determine O to be consistent. Entailment checkers that do not support this operation may execute inconsistency tests like entailment test: if the ontology O is inconsistent, then O entails the inconsistent ontology Oin (see Appendix). Given inputs O and Oin, a conforming entailment checker should thus return True, it should not return Unknown, and it must not return False.

3.2 Test Case Format

Test cases are described using OWL, based on the ontology documented in this section. The description includes axioms using the Functional Syntax, and using http://www.w3.org/2007/OWL/testOntology as the default prefix.

3.2.1 Input Ontologies

The :inputOntology property associates a test with one or more input ontologies. Subproperties of this property are used to differentiate among input ontologies when multiple are present.

    Declaration( DataProperty( :inputOntology ) )
    Declaration( DataProperty( :premiseOntology ) )
    Declaration( DataProperty( :conclusionOntology ) )
    Declaration( DataProperty( :nonConclusionOntology ) )

    SubPropertyOf( :premiseOntology :inputOntology )
    SubPropertyOf( :conclusionOntology :inputOntology )
    SubPropertyOf( :nonConclusionOntology :inputOntology )

Similarly, subproperties are used to indicate the syntax of the input ontology.

    Declaration( DataProperty( :fsInputOntology ) )
    Declaration( DataProperty( :owlXmlInputOntology ) )
    Declaration( DataProperty( :rdfXmlInputOntology ) )

    SubPropertyOf( :fsInputOntology :inputOntology )
    SubPropertyOf( :owlXmlInputOntology :inputOntology )
    SubPropertyOf( :rdfXmlInputOntology :inputOntology )

3.2.2 Type

All test cases are individuals in the :TestCase class. Subclasses of this class are used to map tests to the test types described above. The axioms below describe the relationships between the test types and the input ontology requirements of each test type.

    Declaration( Class( :TestCase ) )
    Declaration( Class( :ProfileIdentificationTest ) )
    Declaration( Class( :ConsistencyTest ) )
    Declaration( Class( :InconsistencyTest ) )
    Declaration( Class( :PositiveEntailmentTest ) )
    Declaration( Class( :NegativeEntailmentTest ) )

    SubClassOf( :ProfileIdentificationTest :TestCase )
    SubClassOf( :ConsistencyTest :ProfileIdentificationTest )
    SubClassOf( :InconsistencyTest :ProfileIdentificationTest )
    SubClassOf( :PositiveEntailmentTest :ConsistencyTest )
    SubClassOf( :NegativeEntailmentTest :ConsistencyTest )

    SubClassOf( :ConsistencyTest MinCardinality( 1 :premiseOntology ) )
    SubClassOf( :InconsistencyTest MinCardinality( 1 :premiseOntology ) )
    SubClassOf( :PositiveEntailmentTest MinCardinality( 1 :conclusionOntology ) )
    SubClassOf( :NegativeEntailmentTest MinCardinality( 1 :nonConclusionOntology ) )

    DisjointClasses( :ConsistencyTest :InconsistencyTest )

3.2.3 Species

The :species property describes the syntactic conformance of the input ontology with respect to OWL 2 Full and OWL 2 DL.

If the property associates a test case with the individual :FULL, the input ontology is an OWL Full ontology document. This property assertion should exist for all test cases.

If the property associates a test case with the individual :DL, the input ontology is an OWL 2 DL ontology document.

Each test should either have a property assertion indicating the input ontology is an OWL DL ontology, or a negative property assertion indicating it is not.

    Declaration( ObjectProperty( :species ) )
    PropertyRange( :species OneOf( :DL :FULL ) )
    DifferentIndividuals( :DL :FULL )
    SubClassOf( :TestCase HasValue( :species :FULL ) )

3.2.4 Profiles

The :profile property describes the syntactic conformance of the input ontology with respect to the profiles of OWL 2.

If the property associates a test case with the individual :EL, the input ontology is an OWL 2 EL ontology document.

If the property associates a test case with the individual :QL, the input ontology is an OWL 2 QL ontology document.

If the property associates a test case with the individual :RL, the input ontology is an OWL 2 RL ontology document.

    Declaration( ObjectProperty( :profile ) )
    PropertyRange( :profile OneOf( :EL :QL :RL ) )
    DifferentIndividuals( :EL :QL :RL )

The following axiom is included in the test ontology because OWL 2 EL and OWL 2 QL are syntactic profiles of OWL 2 DL.

    SubClassOf(
        SomeValuesFrom( :profile OneOf( :EL  :QL ) )
        HasValue( :species :DL )
    )

For each profile, for each test there should either be a property assertion indicating the input ontology conforms to the restrictions of the profile, or a negative property assertion indicating it does not.

3.2.5 Imported Ontologies

The :importedOntology property associates a test case with an ontology contained in the imports closure of the input ontologies.

    Declaration( DataProperty( :importedOntology ) )

As with the input ontology, subproperties are used to indicate the syntax of the import ontology.

    Declaration( DataProperty( :fsImportedOntology ) )
    Declaration( DataProperty( :owlXmlImportedOntology ) )
    Declaration( DataProperty( :rdfXmlImportedOntology ) )

    SubPropertyOf( :fsImportedOntology :importedOntology )
    SubPropertyOf( :owlXmlImportedOntology :importedOntology )
    SubPropertyOf( :rdfXmlImportedOntology :importedOntology )
    DisjointProperties( :fsImportedOntology :owlXmlImportedOntology :rdfXmlImportedOntology )

3.2.6 Normative Syntax

The :normativeSyntax functional property associates a test case with an individual indicating which syntax.

The individual :FUNCTIONAL indicates the functional syntax input ontologies are normative and other syntaxes are informative.

The individual :OWLXML indicates the OWL XML syntax input ontologies are normative and other syntaxes are informative.

The individual :RDFXML indicates the RDF/XML syntax input ontologies are normative and other syntaxes are informative.

    Declaration( ObjectProperty( :normativeSyntax ) )
    FunctionalProperty( :normativeSyntax )
    PropertyRange( :normativeSyntax OneOf( :RDFXML :FUNCTIONAL :OWLXML ) )
    DifferentIndividuals( :RDFXML :FUNCTIONAL :OWLXML )

    SubClassOf( :TestCase ExactCardinality( 1 :normativeSyntax ) )

If an input ontology is not an OWL 2 DL ontology document, the normative syntax must be RDF/XML.

    SubClassOf(
        IntersectionOf( :TestCase ComplementOf( HasValue( :species :DL ) ) ) 
        HasValue( :normativeSyntax :RDFXML )
    )


3.2.7 Normative Translation

The :normativeTranslation associates a syntax translation test case with an individual indicating a syntax from which a parse of an input ontology will be structurally equivalent to the same input ontology parsed from the normative syntax.

    Declaration( ObjectProperty( :normativeTranslation ) )
    PropertyRange( :normativeTranslation OneOf( :RDFXML :FUNCTIONAL :OWLXML ) )
    DisjointProperties( :normativeSyntax :normativeTranslation )
    EquivalentClasses( :SyntaxTranslationTest IntersectionOf( :TestCase MinCardinality( 1 :normativeTranslation ) ) )

3.2.8 Specific Semantics

Property assertions using the :semantics property indicate that a test is applicable under one type of OWL semantics, but not another.

The value :FULL indicates that the test is applicable if and only if the [OWL 2 RDF-Based Semantics] are used.

The value :DL indicates that the test is applicable if and only if the [OWL 2 Direct Semantics] are used.

    Declaration( ObjectProperty( :semantics ) )
    FunctionalProperty( :semantics )
    PropertyRange( :semantics OneOf( :DL :FULL ) )

For all tests which depend on specific semantics the :alternativeSemanticsTest symmetric property associates the test with a second test which uses the alternative semantics.

    Declaration( ObjectProperty( :alternativeSemanticsTest ) )
    FunctionalProperty( :alternativeSemanticsTest )
    SymmetricProperty( :alternativeSemanticsTest )

    SubClassOf(
      HasValue( :semantics :DL )
      SomeValuesFrom( :alternativeSemanticsTest HasValue( :semantics :FULL ) )
    )

    SubClassOf(
      HasValue( :semantics :FULL )
      SomeValuesFrom( :alternativeSemanticsTest HasValue( :semantics :DL ) )
    )

3.2.9 Status

The :status property indicates if a test case has been PROPOSED, ACCEPTED, or REJECTED by the OWL working group.

    Declaration( ObjectProperty( :status ) )
    FunctionalProperty( :status )
    PropertyRange( :status OneOf( :Proposed :Approved :Rejected ) )
    DifferentIndividuals( :Proposed :Approved :Rejected )

3.2.10 Creator

A test can be related to a literal description of it's author using the :creator annotation property.

    Declaration( AnnotationProperty( :creator ) )

3.2.11 Description

A literal containing a human readable description can associated with a test using :description annotation property.

    Declaration( AnnotationProperty( :description ) )

3.2.12 Issue

The :issue annotation property can be used to associate a test with a specific WG issue (using a link to the issue tracker).

    Declaration( AnnotationProperty( :issue ) )

3.2.13 Identifier

The :identifier annotation should be used to associate a unique identifier with a test case. This identifier should conform to irelative-ref as defined in [RFC-3987] so that it may be appended to http://www.w3.org/2007/OWL/owlt/ to generate a URI for the test.

    Declaration( AnnotationProperty( :identifier ) )

3.2.14 Complete Test Ontology

Namespace( = <http://www.w3.org/2007/OWL/testOntology> )
Ontology(<http://www.w3.org/2007/OWL/testOntology>
    Label("The OWL 2 Test Ontology")

    Declaration( Class( :TestCase ) )

    Declaration( AnnotationProperty( :identifier ) )
    Declaration( AnnotationProperty( :description ) )
    Declaration( AnnotationProperty( :creator ) )
    Declaration( AnnotationProperty( :issue ) )

    Declaration( DataProperty( :inputOntology ) )
    Declaration( DataProperty( :premiseOntology ) )
    Declaration( DataProperty( :conclusionOntology ) )
    Declaration( DataProperty( :nonConclusionOntology ) )

    SubPropertyOf( :premiseOntology :inputOntology )
    SubPropertyOf( :conclusionOntology :inputOntology )
    SubPropertyOf( :nonConclusionOntology :inputOntology )


    Declaration( Class( :ProfileIdentificationTest ) )

    SubClassOf( :ProfileIdentificationTest :TestCase )
    SubClassOf( :ProfileIdentificationTest MinCardinality( 1 :inputOntology ) )


    Declaration( Class( :ConsistencyTest ) )
    Declaration( Class( :InconsistencyTest ) )

    SubClassOf( :ConsistencyTest :ProfileIdentificationTest )
    SubClassOf( :InconsistencyTest :ProfileIdentificationTest )
    SubClassOf( :ConsistencyTest MinCardinality( 1 :premiseOntology ) )
    SubClassOf( :InconsistencyTest MinCardinality( 1 :premiseOntology ) )
    DisjointClasses( :ConsistencyTest :InconsistencyTest )


    Declaration( Class( :PositiveEntailmentTest ) )

    SubClassOf( :PositiveEntailmentTest :ConsistencyTest )
    SubClassOf( :PositiveEntailmentTest MinCardinality( 1 :conclusionOntology ) )


    Declaration( Class( :NegativeEntailmentTest ) )

    SubClassOf( :NegativeEntailmentTest :ConsistencyTest )
    SubClassOf( :NegativeEntailmentTest MinCardinality( 1 :nonConclusionOntology ) )


    Declaration( ObjectProperty( :status ) )
    FunctionalProperty( :status )
    PropertyRange( :status OneOf( :Proposed :Approved :Rejected ) )
    DifferentIndividuals( :Proposed :Approved :Rejected )


    Declaration( ObjectProperty( :species ) )
    PropertyRange( :species OneOf( :DL :FULL ) )
    DifferentIndividuals( :DL :FULL )
    SubClassOf( HasValue( :species :DL ) HasValue( :species :FULL ) )


    Declaration( ObjectProperty( :profile ) )
    PropertyRange( :profile OneOf( :EL :QL :RL ) )
    DifferentIndividuals( :EL :QL :RL )


    SubClassOf(
        SomeValuesFrom( :profile OneOf( :EL  :QL ) )
        HasValue( :species :DL )
    )


    Declaration( ObjectProperty( :normativeSyntax ) )
    FunctionalProperty( :normativeSyntax )
    PropertyRange( :normativeSyntax OneOf( :RDFXML :FUNCTIONAL :OWLXML ) )
    DifferentIndividuals( :RDFXML :FUNCTIONAL :OWLXML )

    SubClassOf( :TestCase ExactCardinality( 1 :normativeSyntax ) )

    Declaration( ObjectProperty( :normativeTranslation ) )
    PropertyRange( :normativeTranslation OneOf( :RDFXML :FUNCTIONAL :OWLXML ) )
    DisjointProperties( :normativeSyntax :normativeTranslation )
    EquivalentClasses( :SyntaxTranslationTest IntersectionOf( :TestCase MinCardinality( 1 :normativeTranslation ) ) )

    SubClassOf(
        IntersectionOf( :TestCase ComplementOf( HasValue( :species :DL ) ) ) 
        HasValue( :normativeSyntax :RDFXML )
    )
    
    Declaration( ObjectProperty( :semantics ) )
    Declaration( ObjectProperty( :alternativeSemanticsTest ) )

    PropertyRange( :semantics OneOf( :DL :FULL ) )
    FunctionalProperty( :semantics )
    FunctionalProperty( :alternativeSemanticsTest )
    SymmetricProperty( :alternativeSemanticsTest )

    SubClassOf(
      HasValue( :semantics :DL )
      SomeValuesFrom( :alternativeSemanticsTest HasValue( :semantics :FULL ) )
    )

    SubClassOf(
      HasValue( :semantics :FULL )
      SomeValuesFrom( :alternativeSemanticsTest HasValue( :semantics :DL ) )
    )


    Declaration( DataProperty( :fsInputOntology ) )
    SubPropertyOf( :fsInputOntology :inputOntology )

    Declaration( DataProperty( :owlXmlInputOntology ) )
    SubPropertyOf( :owlXmlInputOntology :inputOntology )

    Declaration( DataProperty( :rdfXmlInputOntology ) )
    SubPropertyOf( :rdfXmlInputOntology :inputOntology )

    DisjointProperties( :fsInputOntology :owlXmlInputOntology :rdfXmlInputOntology )


    Declaration( DataProperty( :importedOntology ) )
    Declaration( DataProperty( :fsImportedOntology ) )
    Declaration( DataProperty( :owlXmlImportedOntology ) )
    Declaration( DataProperty( :rdfXmlImportedOntology ) )

    SubPropertyOf( :fsImportedOntology :importedOntology )
    SubPropertyOf( :owlXmlImportedOntology :importedOntology )
    SubPropertyOf( :rdfXmlImportedOntology :importedOntology )
    DisjointProperties( :fsImportedOntology :owlXmlImportedOntology :rdfXmlImportedOntology )


    # The following "intersection properties" have not been described in the test and conformance document but are used

    Declaration( DataProperty( :fsPremiseOntology ) )
    Declaration( DataProperty( :fsConclusionOntology ) )
    Declaration( DataProperty( :fsNonConclusionOntology ) )
    SubPropertyOf( :fsPremiseOntology :premiseOntology )
    SubPropertyOf( :fsPremiseOntology :fsInputOntology )
    SubPropertyOf( :fsConclusionOntology :conclusionOntology )
    SubPropertyOf( :fsConclusionOntology :fsInputOntology )
    SubPropertyOf( :fsNonConclusionOntology :nonConclusionOntology )
    SubPropertyOf( :fsNonConclusionOntology :fsInputOntology )

    Declaration( DataProperty( :owlXmlPremiseOntology ) )
    Declaration( DataProperty( :owlXmlConclusionOntology ) )
    Declaration( DataProperty( :owlXmlNonConclusionOntology ) )
    SubPropertyOf( :owlXmlPremiseOntology :premiseOntology )
    SubPropertyOf( :owlXmlPremiseOntology :owlXmlInputOntology )
    SubPropertyOf( :owlXmlConclusionOntology :conclusionOntology )
    SubPropertyOf( :owlXmlConclusionOntology :owlXmlInputOntology )
    SubPropertyOf( :owlXmlNonConclusionOntology :nonConclusionOntology )
    SubPropertyOf( :owlXmlNonConclusionOntology :owlXmlInputOntology )

    Declaration( DataProperty( :rdfXmlPremiseOntology ) )
    Declaration( DataProperty( :rdfXmlConclusionOntology ) )
    Declaration( DataProperty( :rdfXmlNonConclusionOntology ) )
    SubPropertyOf( :rdfXmlPremiseOntology :premiseOntology )
    SubPropertyOf( :rdfXmlPremiseOntology :rdfXmlInputOntology )
    SubPropertyOf( :rdfXmlConclusionOntology :conclusionOntology )
    SubPropertyOf( :rdfXmlConclusionOntology :rdfXmlInputOntology )
    SubPropertyOf( :rdfXmlNonConclusionOntology :nonConclusionOntology )
    SubPropertyOf( :rdfXmlNonConclusionOntology :rdfXmlInputOntology )
)

3.3 Test Case Repositories

A set of approved test cases is provided in the OWL 2 Test Case Repository [OWL 2 Test Cases]. Those test cases have been collected based on the approval process described below, and they are expected to remain static after the Working Group has finished.

Like any test set, the approved OWL tests are necessarily incomplete in that they cannot cover all relevant situations or possible implementation challenges. For this reason, an additional public test repository [Contributed Test Cases] is provided as a platform for collecting further test cases even after the termination of the Working Group. Since there is no formal approval process for those additional test cases, they are not necessarily subjected to extensive review, possibly resulting in erroneous or misleading information. Yet, it is hoped that the additional repository will provide a valuable tool for the development of OWL after the finalization of the recommendation.

3.4 Approval Process Overview

This section outlines the process by means of which test cases have been selected for inclusion into the OWL 2 Test Case Repository [OWL 2 Test Cases].

  • At the chair's discretion, individual tests or groups of tests are put to the Working Group in the weekly telecon or at a face-to-face meeting.
  • The Working Group may approve, reject, or defer decision on a test.
    • If the Working Group approves a test, its status is changed to APPROVED. All approved and only approved tests are included in the test case repository [OWL 2 Test Cases].
    • If the Working Group rejects a test, its status is changed to REJECTED.
    • If the Working Group defers decision on a test, its status remains PROPOSED.
  • At the chairs' discretion, the Working Group may review any previous decision regarding any test cases.

The Working Group has complete discretion to approve or reject tests independent of their conformance with this process or their conformance with the OWL Working Drafts.

3.5 Changes From WebOnt Tests

3.5.1 Formatting Changes

Each WebOnt test case used multiple files instead of storing ontologies in literals.

3.5.2 Changes to Test Types

"Profile Identification Tests" and "Syntax Translation Tests" did not exist in the WebOnt test suite.

"Tests for Incorrect Use of OWL Namespace" has been removed as a type. These tests were intended to highlight differences between the OWL RDF vocabulary and the DAML+OIL vocabulary. Time has reduced the motivation for such tests.

"True Tests", "OWL for OWL Tests", and "Import Entailment Tests" have been removed as types. These types were each specializations of entailment tests. To the extent that they are present in the current test suite, these tests are marked as positive entailment tests.

"Import Level Tests" has been removed as a type. This type is now included in the "Profile Identification Tests".

3.5.3 Changes to Process

Status of each test no longer includes "EXTRACREDIT" and "OBSOLETED".

4 Appendix: An Inconsistent Ontology

Consistency tests and inconsistency tests can be considered as entailment tests and non-entailment tests, respectively, by checking the entailment of an (arbitrary) inconsistent ontology. This appendix provides one ontology that can be used for this purpose (under both [OWL 2 Direct Semantics] and [OWL 2 RDF-Based Semantics]), and which is compatible with all profiles:

Namespace( owl =  <http://www.w3.org/2002/07/owl#> )
Ontology(<http://www.w3.org/2007/OWL/inconsistentOntology>
    SubClassOf( owl:Thing owl:Nothing )
)

5 References

[Contributed Test Cases]
Public Test Case Repository.
Editor's Note: The final location of the repository of user-contributed test cases is yet to be determined.
[OWL 2 Specification]
OWL 2 Web Ontology Language: Structural Specification and Functional-Style Syntax. Peter F. Patel-Schneider, Ian Horrocks, and Boris Motik, eds., 2008.
[OWL 2 Direct Semantics]
OWL 2 Web Ontology Language: Direct Semantics. Bernardo Cuenca Grau and Boris Motik, eds., 2008.
[OWL 2 RDF-Based Semantics]
OWL 2 Web Ontology Language: RDF-Based Semantics. Michael Schneider, ed., 2008.
[OWL 2 RDF Mapping]
OWL 2 Web Ontology Language: Mapping to RDF Graphs. Bernardo Cuenca Grau and Boris Motik, eds., 2008.
[OWL 2 XML Syntax]
OWL 2 Web Ontology Language: XML Serialization. Boris Motik and Peter. F. Patel-Schneider, eds., 2008.
[OWL 2 Profiles]
OWL 2 Web Ontology Language: Profiles. Boris Motik, Bernardo Cuenca Grau, Ian Horrocks, Zhe Wu, Achille Fokoue and Carsten Lutz, eds., 2008.
[OWL 2 Test Cases]
OWL 2 Test Case Repository.
Editor's Note: The final location of the repository of approved test cases is yet to be determined.
[RDF Syntax]
RDF/XML Syntax Specification (Revised). Dave Beckett, Editor, W3C Recommendation 10 February 2004, http://www.w3.org/TR/2004/REC-rdf-syntax-grammar-20040210/ . Latest version available at http://www.w3.org/TR/rdf-syntax-grammar/ .
[SROIQ]
The Even More Irresistible SROIQ. Ian Horrocks, Oliver Kutz, and Uli Sattler. In Proc. of the 10th Int. Conf. on Principles of Knowledge Representation and Reasoning (KR 2006). AAAI Press, 2006.
[OWL Metamodeling]
On the Properties of Metamodeling in OWL. Boris Motik. Journal of Logic and Computation, 17(4):617-637, 2007.
[RFC 2119]
RFC 2119: Key words for use in RFCs to Indicate Requirement Levels. Network Working Group, S. Bradner. Internet Best Current Practice, March 1997.
[RFC-3987]
RFC 3987 - Internationalized Resource Identifiers (IRIs). M. Duerst, M. Suignard. IETF, January 2005, http://www.ietf.org/rfc/rfc3987.txt.