W3C

Requirements for the Evaluation and Report Language (EARL) 1.0

W3C Working Group Note 2 February 2017

This version:
https://www.w3.org/TR/2017/NOTE-EARL10-Requirements-20170202/
Latest version:
https://www.w3.org/TR/EARL10-Requirements/
Previous version:
https://www.w3.org/TR/2009/WD-EARL10-Requirements-20091029/
Editors:
Michael Squillace, IBM Corporation
Shadi Abou-Zahra, W3C/WAI

Abstract

This document describes the requirements for the scope, design, and features of the Evaluation and Report Language (EARL) 1.0. EARL is a vocabulary, the terms of which are defined across a set of specifications and technical notes, and that is used to describe test results. The primary motivation for developing this vocabulary is to facilitate the exchange of test results between Web accessibility evaluation tools in a vendor-neutral and platform-independent format. It also provides reusable terms for generic quality assurance and validation purposes.

The terms of EARL are defined by Evaluation and Report Language (EARL) 1.0 Schema, HTTP Vocabulary in RDF 1.0, Representing Content in RDF 1.0, and Pointer methods in RDF 1.0. A companion document, Evaluation and Report Language (EARL) 1.0 Guide, describes the motivations for EARL and provides an introduction to its uses. An Evaluation and Report Language (EARL) Overview is also available.

Status of this Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at https://www.w3.org/TR/.

This Requirements for Evaluation and Report Language (EARL) 1.0 is for the Evaluation and Report Language (EARL) 1.0 Schema and its supporting documents. Work on EARL 1.0 is currently stopped because the Evaluation and Repair Tools Working Group (ERT WG) reached the end of its Charter.

If you wish to make comments regarding this Requirements for Evaluation and Report Language (EARL) 1.0 document, please send them to public-earl10-comments@w3.org (publicly visible mailing list archive).

Publication as a Working Group Note does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document has been produced by the Evaluation and Repair Tools Working Group (ERT WG) as part of the Web Accessibility Initiative (WAI) Technical Activity.

This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

This document is governed by the 1 September 2015 W3C Process Document.

Introduction

On January 1, 2005 and again on ??? 2008, the ERT WG was chartered to continue the development of EARL 1.0. The following is a collection of requirements set forth by the Working Group. Each requirement is labeled with one of the following key words:

WILL
The collection of specifications and technical notes that comprise the EARL 1.0 vocabulary produced by the Working Group must satisfy this requirement.
SHOULD
The collection of specifications and technical notes that comprise the EARL 1.0 vocabulary produced by the Working Group must address this requirement.
MAY
The collection of specifications and technical notes that comprise the EARL 1.0 vocabulary produced by the Working Group can exclude this requirement.

Note: These key words, as used in this requirements document, should not be confused with the key words defined by [RFC 2119].

1. Scope

S01
EARL 1.0 will be a vocabulary, the terms of which are given in a set of specifications and explanatory technical notes for describing test results.
S02
EARL 1.0 will be sufficiently complete to describe test results from the domain of Web accessibility evaluation, such as evaluation with regard to the W3C Web Content Accessibility Guidelines [WCAG].
S03
EARL 1.0 will be abstract enough to describe test results in other domains or contexts, such as Web quality assurance or Web Security.
S04
EARL 1.0 may be generic enough to describe test results eminating from testing non-web artifacts.

2. Design

D01
EARL 1.0 will define its vocabulary as a set of Resource Description Framework Schemas [RDF]
D02
The specifications and technical notes that comprise the EARL 1.0 vocabulary will be developed and published according to the [W3C QA Framework].
D03
The EARL 1.0 vocabulary will provide for test results and reports (collections of results) that can be generated or processed efficiently and in a time that is reasonable relative to the other functions implemented by applications that perform such processing or generation.
D04
EARL 1.0 should reuse existing vocabularies whenever possible and appropriate.
D05
EARL 1.0 will support the extension of its vocabulary to specific domains, including the domain of Web accessibility evaluation.
D06
EARL 1.0 will support automated, semi-automated, or manual testing.
D07
EARL 1.0 will provide a vocabulary that supports the internationalization infrastructure included in [RDF].
D08
EARL 1.0 will define a conformance model consisting of a minimal set of required terms and restrictions on how to use them.

3. Features

F01
EARL 1.0 will provide terms in its vocabulary for describing the context in which the results were obtained.
F02
EARL 1.0 will provide mechanisms to relate test results with the specific location within the subject that triggered the result where applicable.
F03
EARL 1.0 should support persistency of the validity of the results with respect to modifications or variations of the subject due to its dynamic nature where applicable.
F04
EARL 1.0 will support the combining of sets of test results into a single report consisting of these sets of results or of portions of a report (e.g. specification of test criteria, test subject, etc.) into a single report.

Appendix A: References

[Overview]
Evaluation and Report Language (EARL) Overview
[Schema]
Evaluation and Report Language (EARL) 1.0 Schema
[Guide]
Evaluation and Report Language (EARL) 1.0 Guide
[HTTP-in-RDF]
HTTP Vocabulary in RDF 1.0
[Content-in-RDF]
Representing Content in RDF 1.0
[Pointers-in-RDF]
Pointer methods in RDF 1.0
[RFC 2119]
Key words for use in RFCs to Indicate Requirement Levels
[WCAG]
Web Content Accessibility Guidelines
[W3C QA Framework]
W3C QA Framework: Specification Guidelines
[RDF]
Resource Description Framework (RDF): Concepts and Abstract Syntax
RDF Vocabulary Description Language 1.0: RDF Schema
RDF Primer
RDF/XML Syntax Specification
Why RDF model is different from the XML model