DAWG: Implementation Reporting - EARL results

Document Editor: Lee Feigenbaum – invited expert
Version:: $Revision: 1.3 $

Abstract

This document describes the use of EARL by the RDF Data Access Working Group. The Working Group is using EARL to collect reports of implementations' experience running the group's test suite.

Status of This Document

Working Document.

The RDF Data Access Working Group (DAWG) uses a test-driven process. The test area is a collection of collection of the current test cases of the working group.

As one of the exit criteria to transition the SPARQL Query Language specification to Proposed Recommendation the group is seeking to demonstrate that

Each identified SPARQL feature has at least two implementations.

To do this, the group is asking for implementors of SPARQL to run their implementations against the test suite and report the results to the group. Reports can be submitted to public-rdf-dawg-comments@w3.org, a mailing list with a public archive. This document presents an example of EARL results and highlights a few best practices that will help the Working Group produce the implementation report.

The Structure of an EARL Report

(In this section, the earl: prefix is shorthand for http://www.w3.org/ns/earl#.)

An EARL report consists of one earl:assertion per test run. Each assertion is earl:assertedBy the person or software that is responsible for running the test. Each assertion references the test being run via the earl:test predicate and identifies the implementation being tested via the earl:subject predicate.

The result of the single test run against an implementation is given by the earl:result predicate. The object of this predicate is an earl:TestResult that uses the earl:outcome predicate to specify the test's result (e.g. earl:pass, but see the EARL Schema for other possible outcome values).

Best Practices for Reporting EARL to the DAWG

In order to assemble a consistent implementation report, the DAWG asks that EARL reports submitted to the group observe the following conventions:

An EARL assertor (the object of earl;assertedBy) can be a URI or a blank node, but should itself contain a foaf:homepage triple that uniquely identifies the assertor.
The object of earl:subject should be a URI. We use the DOAP vocabulary to provide further information on the implementation being tested.
The canonical URIs for a test (the objects of earl:test) should be based on the test's URI as given by its manifest file. (For example, http://www.w3.org/2001/sw/DataAccess/tests/data-r2/graph/manifest#dawg-graph-01.)

Example EARL

The following example of EARL reporting is excerpted from an example report by DAWG participant Chimezie Ogbuji.

@prefix _6: <http://>.
@prefix _7: <http://www.w3.org/2001/sw/DataAccess/tests/data-r2/graph/manifest#>.
@prefix _8: <http://purl.org/net/chimezie/foaf#>.
@prefix _9: <http://purl.org/net/chimezie/>.
@prefix doap: <http://usefulinc.com/ns/doap#>.
@prefix earl: <http://www.w3.org/ns/earl#>.
@prefix foaf: <http://xmlns.com/foaf/0.1/>.
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>.
@prefix xml: <http://www.w3.org/XML/1998/namespace>.

 _8:chime a foaf:Person;
     rdfs:seeAlso _9:foaf;
     foaf:homepage <http://metacognition.info>;
     foaf:name "Chimezie Ogbuji". 

 <http://rdflib.net> a doap:Project;
     doap:name "RDFLib";
     doap:release
       [ a doap:Version;
         doap:created "2007-07-06"^^<http://www.w3.org/2001/XMLSchema#date>;
         doap:name "rdflib-2.4.1.dev-r1155"].

 [ a earl:Assertion;
         earl:assertedBy _8:chime;
         earl:result [ a earl:TestResult;
                 earl:outcome earl:pass];
         earl:subject <http://rdflib.net>;
         earl:test _7:dawg-graph-02].

 [ a earl:Assertion;
         earl:assertedBy _8:chime;
         earl:result [ a earl:TestResult;
                 earl:outcome earl:pass];
         earl:subject <http://rdflib.net>;
         earl:test _7:dawg-graph-01].

 [ a earl:Assertion;
         earl:assertedBy _8:chime;
         earl:result [ a earl:TestResult;
                 earl:outcome earl:pass];
         earl:subject <http://rdflib.net>;
         earl:test _7:dawg-graph-05].

 [ a earl:Assertion;
         earl:assertedBy _8:chime;
         earl:result [ a earl:TestResult;
                 earl:outcome earl:pass];
         earl:subject <http://rdflib.net>;
         earl:test _7:dawg-graph-10].

 [ a earl:Assertion;
         earl:assertedBy _8:chime;
         earl:result [ a earl:TestResult;
                 earl:outcome earl:pass];
         earl:subject <http://rdflib.net>;
         earl:test _7:dawg-graph-08].

 [ a earl:Assertion;
         earl:assertedBy _8:chime;
         earl:result [ a earl:TestResult;
                 earl:outcome earl:pass];
         earl:subject <http://rdflib.net>;
         earl:test _7:dawg-graph-03].

 [ a earl:Assertion;
         earl:assertedBy _8:chime;
         earl:result [ a earl:TestResult;
                 earl:outcome earl:pass];
         earl:subject <http://rdflib.net>;
         earl:test _7:dawg-graph-06].

 [ a earl:Assertion;
         earl:assertedBy _8:chime;
         earl:result [ a earl:TestResult;
                 earl:outcome earl:pass];
         earl:subject <http://rdflib.net>;
         earl:test _7:dawg-graph-04].

 [ a earl:Assertion;
         earl:assertedBy _8:chime;
         earl:result [ a earl:TestResult;
                 earl:outcome earl:pass];
         earl:subject <http://rdflib.net>;
         earl:test _7:dawg-graph-11].

 [ a earl:Assertion;
         earl:assertedBy _8:chime;
         earl:result [ a earl:TestResult;
                 earl:outcome earl:pass];
         earl:subject <http://rdflib.net>;
         earl:test _7:dawg-graph-07].

 [ a earl:Assertion;
         earl:assertedBy _8:chime;
         earl:result [ a earl:TestResult;
                 earl:outcome earl:pass];
         earl:subject <http://rdflib.net>;
         earl:test _7:dawg-graph-09].