Coding EARL by example

Charles McCathieNevile (versión castellano or version française or versione italiana)

About the presentation
Part I - the basics
- EARL - the general idea
- Implications of RDF
- EARL structure
- Each point explained
Part II - advanced
- More detail about RDF
- Advanced possibilities
- Current issues

About this presentation

This presentation of EARL is designed to help people who are comfortable with the idea of copying and pasting XML source code to edit new XML, but who are not familiar with EARL nor RDF, and may not have written XML from scratch. It is meant to explain how to create simple correct EARL, and to show how to use some of the more advanced possibilities offered by EARL.

If you are lookig for general inforamtion on what EARL is and what it is for, try "EARL - what and why?"

Please send comments to w3c-wai-er-ig@w3.org - archived publicly

This is a Work in Progress - $Date: 2004/01/25 23:52:03 $ It describes the 6 december 2002 working draft

Acknowledgements

Many thanks to the G1 group of Sidar, Humana, and the WAI ERT working group, for comments and suggestions

Thanks to Maurizio Vittoria for help with the Italian translation., and the Sidar translator's group for help with Spanish.

This presentation was produced as part of the SWAD-Europe project.

The general idea...

EARL lets you record the results of testing something:

Basics - EARL as RDF

EARL is RDF - you should validate it!

For creating EARL, you can pretend it is XML.

(But reading someone else's EARL requires RDF.

So do the really cool things you can do.

That's covered later.)

Structure of EARL

An EARL assertion has at least 5 properties:

A subject - the thing we did the test to
A checkpoint - testCase - what the test was for
A result - the result of the test
An evaluator - assertedBy - who said so
A mode - how the result was decided

And often there is a message attached

What was tested - `subject`

Where something has a URI

<earl:subject rdf:resource="http://example.net/#item"/>

For a web page, it is often better to test a snapshot of it:

<earl:subject>
  <earl:WebContent>
    <earl:reprOf rdf:resource="http://example.org/" />
    <earl:date>31-12-1977</earl:date>
  </earl:WebContent>
</earl:subject>

The `testCase`

If the test is described in RDF elsewhere:

<earl:testCase rdf:resource="http://example.org/tests#number1"/>

EARL assumes each test has a recognised URI

The `result`

This is one of a few possibilities:

Pass
Fail
NotTested
CannotTell
NotApplicable

for example:

<earl:result rdf:type"http://www.w3.org/WAI/ER/EARL/nmg-strawman#CannotTell"/>

The evaluator - `assertedBy`

Who actually makes the claims

Some tools really do have a URI:

<earl:assertedBy rdf:resource="http://validator.w3.org/"/>

(But many tools don't. And people don't - you can't download them.

See the advanced section for how to handle these cases)

Mode

Manual: A person made a decision
Automatic: A tool made the decision
By inference - heuristic: The assertion is the result of inferences from other assertions. (Yes, "heuristic" doesn't mean that in english. But it doesn't matter if you only read arabic...)

<earl:mode rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#automatic"/>

Message

This is just free comment. Very useful on manual tests, can be used for other stuff as well

<earl:message>No Comment. I just decided this</earl:message>

Simple complete example

<?xml version="1.0" encoding="utf-8"?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
         xmlns:earl="http://www.w3.org/WAI/ER/EARL/nmg-strawman#">
  <earl:Assertion>
    <earl:subject rdf:resource="#http://www.w3.org/" />
    <earl:result rdf:type"http://www.w3.org/WAI/ER/EARL/nmg-strawman#Pass"/>
    <earl:testcase rdf:resource="http://example.org/1999/xhtml#transitional"/>
    <earl:assertedBy rdf:resource="http://validator.w3.org" />
    <earl:mode rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#automatic"/>
    <earl:message>This page is valid XHTML</earl:message>
  </earl:Assertion>
</rdf:RDF>

Part II - advanced

more impllications of RDF
Multiple results
More detailed description
- the subject
- the testcase
- new types of result
- the evaluator
Some complex examples
Current issues

RDF - more implications

Order is not significant
Structure of elements/attributes is not fixed (relation is)
You can mix in other namespaces (see foaf:Person example below)
- and extend the current vocabulary

Which means

If you only have an XML parsing setup you may not understand other people's RDF.

RDF - explicit data

RDF elements can have a specific language or datatype

...
<earl:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2004-01-05T13:20:00Z</earl:date>
<earl:message xml:lang="fr">pendant les vacances</earl:message>
<earl:message xml:lang="en">during the holidays</earl:message>
...

Multiple results in a file

Often more than one assertion will be made at a time.

These can be stored in a single file, and some code factored out.

For example, detailed subject or evaluator information

Good practice: Identify each assertion for reference

Multiple results - code example

<earl:WebContent rdf:about="#subject">
  <earl:reprOf rdf:resource="http://example.org/" /> 
  <earl:date>last sunday</earl:date>
</earl:WebContent>
<earl:Assertion rdf:about="#test1result">
  <earl:subject rdf:resource="#subject" />
  <earl:testcase rdf:resource="http://example.org/#test1"/>
  <!-- ... -->
</earl:Assertion>
<earl:Assertion rdf:about="#test2result">
  <earl:subject rdf:resource="#subject" />
  <earl:testcase rdf:resource="http://example.org/#test2"/>
  <!-- ... -->
</earl:Assertion>

More about the subject

Tools identified indirectly (like people)
A page, at a particular time (with:
- accept headers or
- HTTP HEAD information or
- CC/PP settings or
- AccLIP profile...

More about the test

You can describe the test in RDF:

<earl:testcase>
  <earl:TestCase rdf:about="http://www.w3.org/TR/WCAG10/#tech-text-equivalent">
    <wcag:checkpoint>1.1 </wcag:checkpoint>
    <dc:title xml:lang="es">Proporcione un texto equivalente para todo elemento no textual </dc:title>
  </earl:TestCase>
</earl:testcase>

EARL is not a vocabulary for defining tests.

New types of result

It can be useful to have mmore detailed variations on result types.

You need to declare the relationship to the result types they are modifying

For example, to create a new type of Fail, where the checkpoint is partially met:

<rdf:RDF
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
<rdfs:Class rdf:about="http://www.sidar.org/rdf-def/mas-earl#Partial">
   <rdfs:subClassOf rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#Fail"/>
   <rdfs:isDefinedBy rdf:resource="http://www.sidar.org/rdf-def/mas-earl#"/>
   <rdfs:label xml:lang="en">Only partially meets the checkpoint</rdfs:label>
   <rdfs:label xml:lang="es">Cumple solo parte del punto</rdfs:label>
   <rdfs:comment xml:lang="en">An extension used in Hera, to identify partially conforming content</rdfs:comment>
</rdfs:Class>

Using new types of result

Systems that understand full RDF, including the Vocabulary language (sometimes called RDF Schemas) will know that Partial is a type of Fail.

Because EARL doesn't currently limit the number of results, we have included both here, to help even the most simplistic tools understand the result:

<?xml version="1.0" encoding="utf-8"?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
          xmlns:earl="http://www.w3.org/WAI/ER/EARL/nmg-strawman#">
   <earl:Assertion>
     <earl:subject rdf:resource="#http://example.org/" />
     <earl:result rdf:type"http://www.sidar.org/rdf-def/mas-earl#Partial"/>
     <earl:result rdf:type"http://www.w3.org/WAI/ER/EARL/nmg-strawman#Fail"/>
     <earl:testcase rdf:resource="http://www.w3.org/TR/WCAG10/wai-pageauth.html#tech-use- 
metadata"/>
     <earl:assertedBy rdf:resource="http://www.sidar.org/hera/" />
     <earl:mode rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#automatic"/>
     <earl:message xml:lang="en">You still need to specify the author of the page</earl:message>
   </earl:Assertion>
</rdf:RDF>

Describing the evaluator

Using the Friend-of-a-friend vocabulary to identify a person:

<earl:assertedBy>
  <foaf:Person>
    <foaf:mbox rdf:resource="mailto:charles@w3.org"/>
    <foaf:name>Chaals</foaf:name>
  <foaf:Person>
</earl:assertedBy>

Complex examples

The following examples demonstrate some more advanced features:

earlinst.rdf: multiple assertions, describes the assertor in FOAF, bilingual description of the testcase

Locating the error

This is an open issue:

stability when the page or tool is updated
annotea:context - requires Xpointer
source code reference - very unstable
fuzzy pointer - low implementation

Important for repair processes, and where there are multiple possible errors

Using different versions

There are a number of different versions of EARL around. It is therefore important to note which one you use (with the correct namespace). The version of EARL described in this presentation uses the following:

<rdf:RDF xmlns:earl="http://www.w3.org/WAI/ER/EARL/nmg-strawman#">