Evaluation and Report Language (EARL) 1.0 Schema

1. Introduction (Non-Normative)

[Editor's note: this section will be synchronized with the EARL 1.0 Guide [Guide] as it is being developed.]

The Evaluation and Report Language (EARL) is a format to express test results. Test results include bug reports, test suite evaluations, and conformance claims. The test subject might be a Web site, an authoring tool, a user agent or some other entity. Thus, EARL is flexible. It enables any person, entity, or organization to state test results for any thing tested against any set of criteria.

Stating test results in EARL creates a variety of opportunities. The data can be:

aggregated from different sources
exchanged between tools
used to create customized reports
combined to compare how different test subjects fared on the same test.

A companion document [Guide] to this specification provides more introductory material and explanation of the use cases for EARL.

1.1. Structure of EARL Results (Non-Normative)

EARL statements contain the following types of information:

The context information: This can include information about who or what ran the test, the date the test was run, and other information about the test was performed.
The test subject: This can include: Web pages, tools (e.g. accessibility checkers, validators), and user agents
The result: Did the test subject pass or fail the test? How certain can we be?
Test criteria: What are we evaluating the test subject against? This could be a specification, a set of guidelines, a test from a test suite, or some other test case.

Prose examples that demonstrate the above structure:

Example 1: a person carries out a manual evaluation of a Web page against a test criteria.

Context: Mary Thompson claims on the 17th December
Test Subject: A Web page at http://www.example.org/mypage
Test Result: Passed
Test performed: Checkpoint 1.1 of the Web Content Accessibility Guidelines 1.0

Example 2: an evaluation tool carries out automated evaluation of a Web page against a test criteria.

Context: The W3C Markup Validator claims as of 2004-04-14T14:00:04+1000
Test Subject: The XHTML returned from a GET request to the URI http://www.example.org/yourpage
Test Result: Failed
Test Performed: The validity of the XHTML

1.2. Document Conventions and Audience

Editorial Comments (Non-Normative)

There are a niumber of known issues in this draft, that need to be resolved in future drafts. Comments are specifically requested on any of these issues, which are marked as follows:

[Editor's note: there are some items that the working group has discussed but not reached consensus, and others that are known issues that have not yet been discussed]

This subsection will be removed from the final version of the document, and editorial comments will have been resolved.

Audience of this document (Non-Normative)

This document is intended as a brief complete specification of the EARL 1.0 vocabulary. It assumes that the reader is familiar with the ideas of RDF and can read its XML serialisation as bare code. The assumed audience is developers who are implementing EARL in software or processes, or are seeking to understand the ideas, models, or properties and classes used in the EARL vocabulary. Readers who would like a more tutorial introduction to the language, with more explanation of its foreseen use cases, are referred to the EARL 1.0 Guide [Guide]. Readers who wish to understand more about RDF are advised to consider reading a general introduction, or reading the RDF Primer [RDF-PRIMER].

Requirement keywords

The key words must, required, should, may, and optional are used in accordance with RFC 2119 [RFC2119].

Namespaces

The namespace for EARL as specified in this draft is http://www.w3.org/WAI/ER/EARL/nmg-strawman#.

[Editor's note: Versioning terms during the process of developing the vocabulary is an issue the group is working on. It is possible that a new namespace will be used for a final version of the vocabulary]

Where RDF terms are used in their abbreviated form (e.g. Assertion or foaf:Person), if no namespace is provided the term is in the EARL namespace. The following prefixes are used in examples throughout this document:

earl: The EARL namespace http://www.w3.org/WAI/ER/EARL/nmg-strawman# which is described in this document
foaf: The FOAF namespace http://xmlns.com/foaf/0.1/ which is also where the terms are described [FOAF]
dc: The Dublin Core elements namespace http://purl.org/dc/elements/1.1/ whose terms are described in [DC]
dct: The Dublin Core terms namespace http://purl.org/dc/terms/ described at [DCT]
rdf: The RDF namespace http://www.w3.org/1999/02/22-rdf-syntax-ns# described in [RDF]
rdfs: The RDF Schema namespace http://www.w3.org/2000/01/rdf-schema# described in [RDFS]
owl: The OWL namespace http://www.w3.org/2002/07/owl# described in [OWL]

1.3. About RDF Vocabularies (Non-Normative)

EARL is defined as an RDF vocabulary. The Resource Description Framework (RDF) [RDF] is a general-purpose language for describing information in a way that is machine-understandable.

EARL is an RDF vocabulary used to make statements about how a resource performed against a test. In common with other RDF it assumes that other vocabularies will be used as appropriate.

For more information on RDF, please refer to the following references:

RDF Primer [RDF-PRIMER]
Resource Description Framework (RDF) Model and Syntax Specification [RDF]
Why RDF model is different from the XML model [RDF-XML-DIFFS]

2. Core Vocabulary

The EARL 1.0 specification defines an RDF Vocabulary that consists of classes and properties. This section describes the core vocabulary and gives brief examples of its usage. Later sections describe extensibility of EARL 1.0; and conformance to EARL 1.0.

2.1. Assertion

An Assertion is a statement about the results of performing a test. The earl:Assertion class relates the required instances of an Assertor, Test Subject, Test Requirement, and Test Result to a specific Assertion. It can also provide information about the Test Mode, Methodology and Evidence used in generating the result. It is the fundamental unit of an EARL statement or set of statements.

An Assertion must have at least the following properties:

earl:assertedBy: The Assertion must be asserted by an assertor. The assertor is a human or software, or groups of these, that determine the result.
earl:subject: The thing that is being tested against some requirement is the subject of the assertion.
earl:requirement: The requirement that is used to test a subject is the test requirement of the assertion.
earl:result: The result of the test - whether the subject passes or fails the test case (or there is some other result).

An Assertion may also include the following optional properties:

earl:evidence: This property may be used to describe the other results which were used to derive a result by inference.; [Editors' note: the group is waiting for a proposal on whether this is best modelled as an RDF Collection, or through some other method to describe a list of Assertions]
earl:methodology: There are sometimes various methods that can be used to test for the same requirement. This property allows the tester to declare which methodology was used to determine the result given, or in the case of a derived result the ruleset that was used to combine other results. There is currently no particular vocabulary specified for this.
earl:mode: The mode in which the test was performed - as an automated computer process, by a human making a subjective judgement, or otherwise.

Example 3: instance of an assertion expressed as an RDF/XML fragment.

<earl:Assertion rdf:ID="#assertion">
  <earl:assertedBy rdf:resource="#assertor"/>
  <earl:subject rdf:resource="#subject"/>
  <earl:requirement rdf:resource="#testcase"/>   
  <earl:result rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#pass"/> 
  <earl:methodology rdf:resource="http://example.com/some/method#used"/>
</earl:Assertion>

2.2. Assertor

An Assertor determines the results of a test (i.e. an assertor asserts and assertion). An earl:Assertor must be either a SingleAssertor (a Person, Agent, or Software), or Compound Assertor.

An Assertor must belong to one of the following types:

earl:SingleAssertor: There is a single Assertor responsible for making the Assertion, of one of the following types:
foaf:Person: The Assertor is a human being. This uses the FOAF vocabulary term foaf:Person to describe a person. There should be identifying information including a name, and a uniquely identifying property such as email address or an encrypted email address. The properties foaf:name, foaf:mbox and foaf:mbox_sha1sum are defined by FOAF [FOAF].; [Editor's note: the use of FOAF is subject to review of the stability of these terms.]; [Editor's note: the current requirements on properties to identify the tester are subject to revision.]
foaf:Agent: The Assertor is an Agent, as defined in [FOAF]. foaf:Person is a subclass of this class, which also has subclasses of foaf:Organisation and foaf:Group.
earl:Software: The Assertor is a piece of Software, such as a black box testing tool of some sort or an evaluation tool.; [Editor's note: the current requirements on properties to identify a tool are subject to revision, please see section software for more information]
earl:CompoundAssertor: The Assertor is a compound group of persons and/or softwares. Each group must have at least one primary Assertor identified by the earl:mainAssertor property, and may, have secondary Assertor identified by the earl:helpAssertor property, as well as an optional description identified by the Dublin Core dc:description property.; Since the range of both these properties are earl:Assertor class, the instances can be a Person, Agent, Software, or recursively another Compound Assertor

Example 4: an Assertor that is a person called Bob B. Bobbington.

<foaf:Person rdf:ID="bob">
  <foaf:name>Bob B. Bobbington</foaf:name>
  <foaf:mbox rdf:resource="mailto:bob@example.org"/>
  <foaf:mbox_sha1sum>1a9daad476f0158b81bc66b7b27b438b4b4c19c0</foaf:mbox_sha1sum>
</foaf:Person>

Example 5: an Assertor that is a piece of software called Cool Tool.

<earl:Software rdf:ID="tool">
  <dc:title xml:lang="en">Cool Tool</dc:title>
  <dc:description xml:lang="en">My favorite tool!</dc:description>
  <dc:location>http://example.org/tools/#cool</dc:location>
  <dct:hasVersion>1.0.3</dct:hasVersion>
</earl:Software>

Example 6: the Assertor from example 3 is the person from example 4 using the software from example 5.

<earl:compoundAssertor rdf:ID="assertor">
  <earl:mainAssertor rdf:resource="#bob"/>
  <earl:helpAssertor rdf:resource="#tool"/>
</earl:compoundAssertor>

2.3. Test Subject

The Test Subject is the class of things that have been tested. The earl:TestSubject class must have exactly one date specified by the Dublin Core dc:date property. It may also have any of the following properties:

dc:title: Human readable title for the subject (e.g. "Home Page")
dc:description: Human readable description of the subject (e.g. "Node page of the Web site"
dct:hasPart: Relationship to other subjects that are part of this subject
dct:isPartOf: Relationship to other subjects of which this subject is a part of

This class is intentionally generic to serve a wide variety of usages. For more specific use cases, the EARL earl:WebContent or earl:Software classes can be used in place to describe Web Content or Software that is being tested.

Example 7: Java applet that is part of a Web application

<earl:TestSubject rdf:ID="#applet">
  <dc:title xml:lang="en">Login Applet</dc:title> 
  <dc:description xml:lang="en">Java applet that is used for system login</dc:description> 
  <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#gDate">2005-06-25</dc:date> 
  <dct:isPartOf rdf:resource="http://example.org/"/> 
</earl:TestSubject>

2.4. Test Requirement

A Test Requirement is a test - usually one that can be passed or failed. This includes things such as validation requirements, code test cases, checkpoints from guidelines such as Web Content Accessibility Guidelines 1.0 [WCAG10], or others. These should be identified with a URI.

A Test Requirement may be a single test, or may be a part of a larger compound test suite. These relations may be described using Dublin Core's dct:hasPart or dct:isPartOf properties. Additionally, the Web location or a description for the test may be included by using the Dublin Core dc:location or dc:description properties.

[Editor's note: The earl:TestRequirement class is included for convenience, since it allows various useful properties to be described in simple standard RDF. The working group will deprecate this class if they find an appropriate replacement from a language designed for describing test cases, something which is beyond the scope of the current specification.]

Example 8: instance of a test requirement that is described with a title and its relationship to a test suite.

<earl:TestRequirement rdf:ID="html">
  <dc:title xml:lang="en">HTML Test Suite</dc:title>
  <dc:description xml:lang="en">Tests specifically for the Hyper Text Markup Language</dc:title>
  <dct:isPartOf rdf:resource="http://example.org/tests/#all"/>
  <dct:hasPart rdf:resource="http://example.org/tests/#282"/>
  <dc:location>http://example.org/tests/html/</dc:location>
</earl:TestRequirement>

Example 9: test requirements that don't need to be described further can be shortened as follows:

<earl:testcase rdf:resource="http://www.w3.org/TR/WCAG10/wai-pageauth.html#tech-text-equivalent"/>

@@Do we need this example? It doesn't seem to make any sense to have it. And if we do, was it meant to refer to the property testCase (now requirement) or the class Testcase (now TestRequirement)??

2.5. Test Mode

The Test Mode must be exactly one of the following pre-defined values, or subclasses of them (see section Extensibility for more explanation):

earl:manual: Where the test was performed based on a person's judgement. This includes the case where that judgement was aided through the use of a tool, although this should be additionally noted through the earl:compoundAssertor class of the Assertor.
earl:automatic: Where a computer program or similar has done an automated test and generated a result without human assistance.
earl:mixed: Where a person or organisation has run an automated test and generated a result, which may include the use of human judgement.; [@@Does this replace the software-aided judgement above, or only be used for compound claims where some results come from tools and others from people? My memory is that the latter is what was meant]
earl:heuristic: This property was designed to cover assertions which are made by inference, for example based on several existing test results. The earl:evidence property may be used to point to a list of Assertions used to produce results in this mode

Example 10: instance of an assertion that has been carried out in manual mode

<earl:Assertion rdf:about="http://example.org/#assertion">
  <earl:mode rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#manual"/> 
</earl:Assertion>

2.6. Test Result

The actual result of the test. This must have exactly one earl:validity properties to describe a Validity Level. It may also include a Confidence Level, using the EARL earl:confidence properties; or a description, using the Dublin Core dc:description property. The description is meant to be human-readable text or markup (for example HTML included as an XML Literal), and should include an identification for the natural language.

[Editor's note: the confidence property is still under discussion, please refer to the Confidence Level for more information]

The following example shows a result with a validity of fail and a description of the problem in English, and encoded in XHTML format:

Example 11: A test result with a validity and a description property

<earl:result rdf:parseType="Resource">
  <earl:validity rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#fail"/>
  <dc:description rdf:parseType="Literal">
    <div xml:lang="en" xmlns="http://www.w3.org/1999/xhtml">
      <p>The <code>table</code> element is not allowed to appear
        inside a <code>p</code> element</p>
    </div>
  </dc:description>
</earl:result>

2.6.1. Validity Level

Outcome of a test. The Validity Level must be one of the following pre-defined values, or subclasses of them (see section Extensibility for more explanation):

earl:pass: An assertor claims a test passed successfully.
earl:fail: The Test Subject did not meet the criteria defined by the Test Case.
earl:cannotTell: An Assertor can not tell for sure what the outcome of the test is. Usually this happens when an automated test requires human judgement to make a definite decision.
earl:notApplicable: The Test Requirement is not applicable to the given Test Subject.
earl:notTested: Test has not been carried out. This is useful for reporting as well as for other uses of progress monitoring.

2.6.2. Confidence Level

Confidence in the result by an Assertor. This may be used where a tool wants to assert that it has different levels of confidence about two possible results (for example low confidence that a test has been passed, but also high confidence that it is not applicable). The values of the confidence level should be defined either as RDF values, or using a datatype, in order to allow others to work interoperably with them.

[Editor's note: We will have an example of how to do this in a future draft, and further discussion in the [Guide].]

2.7. Software

A piece of software such as an authoring tool, browser, or evaluation tool. Software must have a title (using the Dublin Core property dc:title) and may have descriptions, version information, and Web location (using the Dublin Core terms dc:description, dct:hasVersion, and dc:location). Additionally, if it is used as a Test Subject then it inherits additional properties such as dc:date (required) as well as dct:isPartOf and dct:hasPart (optional).

[Editor's note: earl:Tool used to be a sub class of Assertor in previous versions of this Working Draft but it has been deprecated in favour of earl:Software, which may also be reused as a Test Subject.]

Example 12: The software which was an Assertor in example 5 is now being described as a Test Subject.

<earl:Software rdf:about="#tool">
  <dc:title xml:lang="en">Cool Tool</dc:title>
  <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#gDate">2005-06-25</dc:date> 
</earl:Software>

2.8. Web Content

Information on the World Wide Web. This should include enough information to identify the content actually tested, taking account of content type and language negotiation and similar factors that can change the content actually served. This class is intended to be used as a test subject but can be reused for other purposes (outside EARL) as well. The Web content class has the following properties:

dc:format: Format of the Web resource. For example "text/HTML", "image/PNG" or other mime types that are returned from the Web server.
dc:location: The URL or other types of pointers that identify an address for the resource.
earl:httpRequest: HTTP request sent by the client to the server in order to initiate a response
earl:httpResponse: HTTP response (except the actual payload of the date) sent by the server to the client

A member of the Web Content class may include several request/response sequences to document the content and language negotiation that took place before the content was finally sent.

[Editor's note: the request/response properties of this class are still being refined]

Appendix A: EARL 1.0 Schema in RDF/XML

[Editor's note: need to make a schema available at the namespace URI, which includes OWL information on deprecated terms]

<?xml version="1.0" encoding='UTF-8'?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
         xmlns:earl="http://www.w3.org/WAI/ER/EARL/nmg-strawman#"
         xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
         xmlns:dc="http://purl.org/dc/elements/1.1/"
         xmlns:foaf="http://xmlns.com/foaf/0.1/"
         xmlns:owl="http://www.w3.org/2002/07/owl#">

  <rdfs:Class rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#Assertion">
    <rdfs:label xml:lang="en">An assertion</rdfs:label>
    <rdfs:comment xml:lang="en">Parent node that contains all parts of an assertion</rdfs:comment>
    <rdfs:subClassOf rdf:parseType="Collection">
      <owl:Restriction>
        <owl:onProperty rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#assertedBy"/>
        <owl:minCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1
          </owl:minCardinality>
      </owl:Restriction>
      <owl:Restriction>
        <owl:onProperty rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#subject"/>
        <owl:minCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1
          </owl:minCardinality>
      </owl:Restriction>
      <owl:Restriction>
        <owl:onProperty rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#requirement"/>
        <owl:minCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1
          </owl:minCardinality>
      </owl:Restriction>
      <owl:Restriction>
        <owl:onProperty rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#result"/>
        <owl:minCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1
          </owl:minCardinality>
      </owl:Restriction>
    </rdfs:subClassOf>
  </rdfs:Class>

  <rdf:Property rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#assertedBy">
    <rdfs:label xml:lang="en">asserted by</rdfs:label>
    <rdfs:domain rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#Assertion"/>
    <rdfs:range rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#Assertor"/>
  </rdf:Property>
  <rdf:Property rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#subject">
    <rdfs:label xml:lang="en">subject</rdfs:label>
    <rdfs:domain rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#Assertion"/>
    <rdfs:range rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#TestSubject"/>
  </rdf:Property>
  <rdf:Property rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#requirement">
    <rdfs:label xml:lang="en">requirement</rdfs:label>
    <rdfs:domain rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#Assertion"/>
    <rdfs:range rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#TestRequirement"/>
  </rdf:Property>
  <rdf:Property rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#result">
    <rdfs:label xml:lang="en">result</rdfs:label>
    <rdfs:domain rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#Assertion"/>
    <rdfs:range rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#TestResult"/>
  </rdf:Property>

  <rdf:Property rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#evidence">
    <rdfs:label xml:lang="en">evidence</rdfs:label>
    <rdfs:domain rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#Assertion"/>
  </rdf:Property>
  <rdf:Property rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#methodology">
    <rdfs:label xml:lang="en">methodology</rdfs:label>
    <rdfs:domain rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#Assertion"/>
  </rdf:Property>
  <rdf:Property rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#mode">
    <rdfs:label xml:lang="en">mode</rdfs:label>
    <rdfs:domain rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#Assertion"/>
    <rdfs:range rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#TestMode"/>
  </rdf:Property>

  <rdfs:Class rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#Assertor">
    <rdfs:label xml:lang="en">Assertor</rdfs:label>
    <rdfs:comment xml:lang="en">Persons or evaluation tools that claim assertions</rdfs:comment>
    <owl:oneOf rdf:parseType="Collection">
      <owl:Thing rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#SingleAssertor"/>
      <owl:Thing rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#CompoundAssertor"/>
    </owl:oneOf>
  </rdfs:Class>
	
  <rdfs:Class rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#SingleAssertor">
    <rdfs:label xml:lang="en">Single Assertor</rdfs:label>
    <rdfs:comment xml:lang="en">One person or evaluation tool that claims assertions</rdfs:comment>
    <owl:oneOf rdf:parseType="Collection">
      <owl:Thing rdf:type="http://www.w3.org/WAI/ER/EARL/nmg-strawman#Software"/>
      <owl:Thing rdf:type="http://xmlns.com/foaf/0.1/Person">
      <owl:Thing rdf:type="http://xmlns.com/foaf/0.1/Agent">
      </owl:Thing>
    </owl:oneOf>
  </rdfs:Class>

  <rdfs:Class rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#CompoundAssertor">
    <rdfs:label xml:lang="en">Compound Assertor</rdfs:label>
    <rdfs:comment xml:lang="en">Group of persons or evaluation tools that claim assertions</rdfs:comment>
    <rdfs:subClassOf rdf:parseType="Collection">
      <owl:Restriction>
        <owl:onProperty rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#mainAssertor"/>
        <owl:minCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1
          </owl:minCardinality>
      </owl:Restriction>
    </rdfs:subClassOf>
  </rdfs:Class>

  <rdf:Property rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#mainAssertor">
    <rdfs:label xml:lang="en">Main Assertor</rdfs:label>
    <rdfs:comment xml:lang="en">Assertor mainly responsible for determining assertion result</rdfs:comment>
    <rdfs:domain rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#CompoundAssertor"/>
    <rdfs:range rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#Assertor"/>
  </rdf:Property>
  <rdf:Property rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#helpAssertor">
    <rdfs:label xml:lang="en">Help Assertor</rdfs:label>
    <rdfs:comment xml:lang="en">Assertor assisting to determine assertion result</rdfs:comment>
    <rdfs:domain rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#CompoundAssertor"/>
    <rdfs:range rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#Assertor"/>
  </rdf:Property>
	
  <rdfs:Class rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#TestSubject">
    <rdfs:label xml:lang="en">Test Subject</rdfs:label>
    <rdfs:comment xml:lang="en">Subject of the assertion</rdfs:comment>
    <rdfs:subClassOf rdf:parseType="Collection">
      <owl:Restriction>
        <owl:onProperty rdf:resource="http://purl.org/dc/elements/1.1/date"/>
        <owl:minCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1
          </owl:minCardinality>
        <owl:maxCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1
          </owl:maxCardinality>
      </owl:Restriction>
    </rdfs:subClassOf>
  </rdfs:Class>

  <rdfs:Class rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#TestRequirement">
    <rdfs:label xml:lang="en">Test Requirement</rdfs:label>
    <rdfs:comment xml:lang="en">A requirement against which subjects are tested</rdfs:comment>
  </rdfs:Class>

  <rdfs:Class rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#TestMode">
    <rdfs:label xml:lang="en">Test Mode</rdfs:label>
    <rdfs:comment xml:lang="en">Mode in which tests were conducted</rdfs:comment>
    <owl:oneOf rdf:parseType="Collection">
      <owl:Thing rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#manual">
        <rdfs:label xml:lang="en">Manual Testing Mode</rdfs:label>
        <rdfs:comment xml:lang="en">Test was performed by a human</rdfs:comment>
      </owl:Thing>
      <owl:Thing rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#automatic">
        <rdfs:label xml:lang="en">Automatic Testing Mode</rdfs:label>
        <rdfs:comment xml:lang="en">Test was performed by a tool</rdfs:comment>
      </owl:Thing>
       <owl:Thing rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#mixed">
        <rdfs:label xml:lang="en">Mixed Testing Mode</rdfs:label>
        <rdfs:comment xml:lang="en">Test was performed by a person or organisation using a tool</rdfs:comment>
      </owl:Thing>
      <owl:Thing rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#heuristic">
        <rdfs:label xml:lang="en">Heuristic Testing Mode</rdfs:label>
        <rdfs:comment xml:lang="en">Result was derived from other results</rdfs:comment>
      </owl:Thing>
    </owl:oneOf>
  </rdfs:Class>

  <rdfs:Class rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#TestResult">
    <rdfs:label xml:lang="en">Test Result</rdfs:label>
    <rdfs:comment xml:lang="en">Result from conducting test cases on subjects</rdfs:comment>
    <rdfs:subClassOf rdf:parseType="Collection">
      <owl:Restriction>
        <owl:onProperty rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#validity"/>
        <owl:minCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1
          </owl:minCardinality>
        <owl:maxCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1
          </owl:maxCardinality>
      </owl:Restriction>
    </rdfs:subClassOf>
  </rdfs:Class>

  <rdf:Property rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#validity">   
    <rdfs:label xml:lang="en">Validity Level Property</rdfs:label>
    <rdfs:domain rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#TestResult"/> 
    <rdfs:range rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#ValidityLevel"/> 
  </rdf:Property>
  <rdf:Property rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#confidence">
    <rdfs:label xml:lang="en">Confidence Level Property</rdfs:label>
    <rdfs:domain rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#TestResult"/> 
    <rdfs:range rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#ConfidenceLevel"/> 
  </rdf:Property>

  <rdfs:Class rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#ValidityLevel">
    <rdfs:label xml:lang="en">Validity Level</rdfs:label>
    <rdfs:comment xml:lang="en">Nominal value of the result</rdfs:comment>
    <owl:oneOf rdf:parseType="Collection">
      <owl:Thing rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#pass">
        <rdfs:label xml:lang="en">Pass</rdfs:label>
        <rdfs:comment xml:lang="en">Test passed</rdfs:comment>
      </owl:Thing>
      <owl:Thing rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#fail">
        <rdfs:label xml:lang="en">Fail</rdfs:label>
        <rdfs:comment xml:lang="en">Test failed</rdfs:comment>
      </owl:Thing>
      <owl:Thing rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#cannotTell">
        <rdfs:label xml:lang="en">Can Not Tell</rdfs:label>
        <rdfs:comment xml:lang="en">Outcome of the test is uncertain</rdfs:comment>
      </owl:Thing>
      <owl:Thing rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#notApplicable">
        <rdfs:label xml:lang="en">Not Applicable</rdfs:label>
        <rdfs:comment xml:lang="en">Test is not applicable to the subject</rdfs:comment>
      </owl:Thing>
      <owl:Thing rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#notTested">
        <rdfs:label xml:lang="en">Not Tested</rdfs:label>
        <rdfs:comment xml:lang="en">Test has not been carried out</rdfs:comment>
      </owl:Thing>
    </owl:oneOf>
  </rdfs:Class>

  <rdfs:Class rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#ConfidenceLevel">
    <rdfs:label xml:lang="en">Confidence Level</rdfs:label>
    <rdfs:comment xml:lang="en">Level of confidence in the given result</rdfs:comment>
  </rdfs:Class>

  <rdfs:Class rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#Software">
    <rdfs:label xml:lang="en">Software Tool</rdfs:label>
    <rdfs:comment xml:lang="en">A tool that can perform tests or be the subject of testing</rdfs:comment>
    <rdfs:subClassOf rdf:parseType="Collection">
      <owl:Restriction>
        <owl:onProperty rdf:resource="http://purl.org/dc/elements/1.1/title"/>
        <owl:minCardinality rdf:datatype="http://www.w3.org/2001/XMLSchema#nonNegativeInteger">1
          </owl:minCardinality>
      </owl:Restriction>
    </rdfs:subClassOf>
  </rdfs:Class>

  <rdfs:Class rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#WebContent">
    <rdfs:label xml:lang="en">Web Content</rdfs:label>
    <rdfs:comment xml:lang="en">Subjects that are available on the Web</rdfs:comment>
  </rdfs:Class>

  <rdf:Property rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#httpRequest">
    <rdfs:label xml:lang="en">http-request</rdfs:label>
    <rdfs:domain rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#WebContent"/> 
  </rdf:Property>

  <rdf:Property rdf:about="http://www.w3.org/WAI/ER/EARL/nmg-strawman#httpResponse">
    <rdfs:label xml:lang="en">http-response</rdfs:label>
    <rdfs:domain rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#WebContent"/> 
  </rdf:Property>

  <!-- Deprecated and replaced terms -->
    <rdf:Description rdf:about="#testCase">
      <owl:sameAs rdf:resource="#requirement" />
    </rdf:Description>

    <rdf:Description rdf:about="#Testcase">
      <owl:sameAs rdf:resource="#TestRequirement" />
    </rdf:Description>

    <rdf:Description rdf:about="#singleAssertor">
      <owl:sameAs rdf:resource="#SingleAssertor" />
    </rdf:Description>

    <rdf:Description rdf:about="#compoundAssertor">
      <owl:sameAs rdf:resource="#CompoundAssertor" />
    </rdf:Description>

</rdf:RDF>

Classes in the EARL namespace
Name	Label	required properties	Allowable types	optional properties
`earl:Assertion`	Assertion	`earl:assertedBy` `earl:subject` `earl:requirement` `earl:result`		`earl:evidence` `earl:methodology` `earl:mode`
`earl:Assertor`	Assertor		`earl:SingleAssertor` `earl:CompoundAssertor`
`earl:SingleAssertor`	Single Assertor		`earl:Software` `foaf:Person` `foaf:Agent`
`earl:CompoundAssertor`	Compound Assertor	`earl:mainAssertor`		`earl:helpAssertor`
`earl:TestSubject`	Test Subject	`dc:date`	`earl:WebContent` `earl:Software`	`dct:hasPart` `dct:isPartOf` `dc:title` `dc:description`
`earl:TestRequirement`	Test Requirement			`dct:hasPart` `dct:isPartOf` `dc:title` `dc:description` `dc:location`
`earl:TestMode`	Test mode		`earl:manual` `earl:automatic` `earl:mixed` `earl:heuristic`
`earl:TestResult`	Test Result	`earl:validity`		`earl:confidence` `dc:description`
`earl:ValidityLevel`	Validity Level		`earl:pass` `earl:fail` `earl:cannotTell` `earl:notApplicable` `earl:notTested`
`earl:ConfidenceLevel`	Confidence Level
`earl:Software`	Software tool	`dc:title`		`dct:hasVersion` `dc:description` `dc:location`
`earl:WebContent`	Web Content			`earl:httpRequest` `earl:httpResponse` `dc:location` `dc:format`

Properties in the EARL namespace
Name	Label	Domain	Range
`earl:assertedBy`	asserted by	`earl:Assertion`	`earl:Assertor`
`earl:subject`	subject	`earl:Assertion`	`earl:TestSubject`
`earl:requirement`	requirement	`earl:Assertion`	`earl:TestRequirement`
`earl:result`	result	`earl:Assertion`	`earl:TestResult`
`earl:mode`	mode	`earl:Assertion`	`earl:TestMode`
`earl:methodology`	methodology	`earl:Assertion`	`rdf:Resource`
`earl:evidence`	evidence	`earl:Assertion`	[Editors' note: To be determined. Either an rdf Collection, or some other list of, `earl:Assertion`s]
`earl:mainAssertor`	Main Assertor	`earl:CompoundAssertor`	`earl:Assertor`
`earl:helpAssertor`	Help Assertor	`earl:CompoundAssertor`	`earl:Assertor`
`earl:validity`	validity	`earl:TestResult`	`earl:ValidityLevel`
`earl:confidence`	confidence	`earl:TestResult`	`earl:ConfidenceLevel`
`earl:httpResponse`	HTTP response information	`earl:WebContent`	[Editors' note: to be specified]
`earl:httpRequest`	HTTP request information	`earl:WebContent`	[Editors' note: to be specified]

Values in the EARL namespace
Name	Label	Used In
`earl:manual`	Manual	`earl:TestMode`
`earl:automatic`	Automatic	`earl:TestMode`
`earl:mixed`	Mixed	`earl:TestMode`
`earl:heuristic`	Heuristic	`earl:TestMode`
`earl:pass`	Pass	`earl:ValidityLevel`
`earl:fail`	Fail	`earl:ValidityLevel`
`earl:notTested`	Not tested	`earl:ValidityLevel`
`earl:cannotTell`	Cannot tell	`earl:ValidityLevel`
`earl:notApplicable`	Not Applicable	`earl:ValidityLevel`

Appendix C: Document Changes (Non-Normative)

This section records current changes and future plans for changes as the nd theworking draft matures to Recommendation level.

Differences between this draft and the EARL 1.0 Schema draft of 9 September 2005

The former Appendix B: EARL 1.0 Schema in N3 has been removed
Redundant constraints on optional properties have been removed from the schema.
Added a subsection on the expected audience of this document.
Made namespace and use of RFC 2119 terms normative.
earl:requirement and earl:TestRequirement have replaced earl:testCase and earl:Testcase
Added methodology and evidence properties.
Added mixed value for the TestMode class.
Made the mode property of Assertion optional.
Removed high, medium and low values of earl:confidence
Added foaf:Agent to the list of allowable Assertor types.

Differences between the EARL 1.0 Schema draft of 9 September 2005 and the EARL 1.0 draft of 12 December 2002

Document split into a specification (this document) and a companion document. Most content stripped out of specification document, and new issues under discussion noted.
Schema presented without the use of XML entities
Typographical errors in schema fixed, so it validates
Re-ordering of the Schema and document. Instead of being organised as a list of classes, then properties, then instance classes, the document and the schema are both organised around the key classes as the framework for defining their use.
Redundant (rdf:Resource) and incorrect (rdfs:Resource) sub-class declarations removed
Schema labels given xml:lang attributes as appropriate
Labels converted to grammatical english (from camelCaseAlmostWords)
Schema encoded in default UTF-8

New terms

New terms have been introduced to the vocabulary, sometimes as replacements for deprecated terms that existed in earlier drafts, sometimes to introduce new funtionality

Terms added since the EARL 1.0 Schema draft of 9 September 2005.

evidence: This property can be used to list information used to infer a result automatically.
methodology: This property can be used to describe a testing methodology or a set of rules for inferring a result.
mixed: This value extends the range of allowed values of the test mode.
requirement: This property has replaced the deprecated testCase as the base property for a requirement that the test subject has been tested against.
TestRequirement: This class has replaced the deprecated Testcase as the base container class for a requirement that the test subject has been tested against.

Terms added between the EARL 1.0 Schema draft of 9 September 2005 and the EARL 1.0 draft of 12 December 2002.

Software: This class replaces the deprecated Tool. It can be used to decribe a assertor tools or test subjects.
SingleAssertor, CompoundAssertor: These classes are derivatives of an Assertor to help describe complex groups of person and/or tool evaluators.
mainAssertor, helpAssertor: These properties of a compound assertor can be used to describe the specific roles of evaluators within a group.
httpRequest, httpResponse: These are placeholder properties for describing WebContent (or a "Delivery Unit" on the Web).

Deprecated terms

A number of terms that existed in earlier drafts have been deprecated, either in favour of terms from existing widely-used vocabularies, or because their use is somehow problematic. While this deprecation is provisional, no RDF has been published formally noting them as deprecated (for example through OWL). A quick guide to terms that have been provisionally deprecated:

high, medium and low: These values for the confidence property were deprecated in favour of authors defining their usage in a way that encourages interoperability.
reprOf: This term was deprecated in favour of a more comprehensive approach to describing Web content.
testCase: This property was replaced by the requirement property.
Testcase: This class was replaced by the TestRequirement class.
UserAgent, Tool: These terms were deprecated in favour of the more generic Software class.
platform: This term was deprecated as it did not seem to be widely accepted and implemented.
message, format: These terms were deprecated in favour of the Dublin Core terms dc:description and dc:format.
Person, name, email, contactInfo: These terms were deprecated in favour of equivalent terms from the FOAF vocabulary.

Working group change plans

The working group is considering several open issues which would lead to extending the current vocabulary before it reaches recommendation.

As well as issues noted in the draft, the following issues are under active consideration:

Evidence

A mechanism to provide further evidence used to make a claim. The working group has included the evidence property but has not yet determined the most useful model for the list of statements used in evidence of an inferred result.

Location

Within Web content, for the use case of testing accessibility, it is very valuable to point to the location(s) that cause the test to pass or fail. This is also true of software interfaces and various other kinds of subject. The working group has agreed in principle to include a mechanism for recording this information in an EARL report, but has not yet developed or found the necessary terms.

HTTP transaction information

There are currently placeholder properties httpRequest and httpResponse for HTTP transaction information. The group expects to develop these further, either themselves or with an existing vocabulary. An extensive amount of work has been done on such vocabularies, and this material couldd be included as soon as the next draft of this specification.

Confidence

This is under review by the working group. The current draft requires that developers who use this property provide some means of identifying the way its values are determined in order to allow interoperable use of those values.

Evaluation and Report Language (EARL) 1.0 Schema

Appendices