RDF Test Suites

From RDF Working Group Wiki
(Redirected from Turtle Test Suite)
Jump to: navigation, search

See also the test reports:


RDF Syntax Test Suites (Turtle family)

The RDF-WG produces test suites for the RDF syntaxes related to Turtle.

A test suite consists of manifest file, which describes each test in the suite and also contains a list of the tests that are currently


See also:

Manifest File Format

The manifest file is written in RDF. The form consists of a header section, which lists the test included and then each test. It is possible to process the file as RDF but also exploit the regular to process with text processing tools.

There are syntax tests, negative syntax tests, evaluation tests and negative evaluation tests.

The following prefixes apply:

 @prefix rdf:    <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
 @prefix rdfs:   <http://www.w3.org/2000/01/rdf-schema#> .
 @prefix mf:     <http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#> .
 @prefix qt:     <http://www.w3.org/2001/sw/DataAccess/tests/test-query#> .
 @prefix rdft:   <http://www.w3.org/ns/rdftest#> .

The assumed base URI for the tests is <http://example/base/> if needed.

Each test has a type, indicating what sort of test it is. It then has either an action given by the mf:action property, or an action and a result given by the mf:result property, (evaluation tests).

Triples may be produced during the running of a test; they are ignored except for evaluation tests, which specify the expected triples. Negative evaluation tests do not specify any expected output triples.

Manifest Header

The manifest header looks like:

 <>  rdf:type mf:Manifest ;
     rdfs:comment "Turtle tests" ;
     mf:entries
     (
     ....
     ) .

where .... is a list of links to test descriptions, one per line.

Turtle Tests

The Turtle tests are at <http://www.w3.org/2013/TurtleTests/>. The manifest has four types of tests:

Relative IRI Resolution

The home of the test suite is <http://www.w3.org/2013/TurtleTests/>. Per RFC 3986 section 5.1.3, the base IRI for parsing each file is the retrieval IRI for that file. For example, the tests turtle-subm-01 and turtle-subm-27 require relative IRI resolution against a base of <http://www.w3.org/2013/TurtleTests/turtle-subm-01.ttl> and <http://www.w3.org/2013/TurtleTests/turtle-subm-27.ttl> respectively.


Character Encoding

The Turtle language uses UTF-8 encoding. The following tests include non-ascii characters:

  • localName_with_assigned_nfc_bmp_PN_CHARS_BASE_character_boundaries
  • localName_with_assigned_nfc_PN_CHARS_BASE_character_boundaries *
  • localName_with_nfc_PN_CHARS_BASE_character_boundaries *
  • labeled_blank_node_with_PN_CHARS_BASE_character_boundaries *
  • LITERAL1_with_UTF8_boundaries *
  • LITERAL_LONG1_with_UTF8_boundaries *
  • LITERAL2_with_UTF8_boundaries *
  • LITERAL_LONG2_with_UTF8_boundaries *

Those marked with a * include characters with codepoints greater than U+FFFD and are thus expressed as a pair of surrogate characters when represented in UCS2.


Turtle Syntax Test

This is an example of a syntax test:

 <#turtle-syntax-file-01> rdf:type rdft:TestTurtlePositiveSyntax ;
    mf:name    "turtle-syntax-file-01" ;
    mf:action  <turtle-syntax-file-01.ttl> ;
    .

A test is passed if the system under test can parse the content at the URL identified by the object of the mf:action property as Turtle.

Turtle Negative Syntax Test

 <#turtle-syntax-bad-uri-01> rdf:type rdft:TestTurtleNegativeSyntax ;
   mf:name    "turtle-syntax-bad-uri-01" ;
   mf:action  <turtle-syntax-bad-uri-01.ttl> ;
   .

A test is passed if the system under test raises an error when it parses the content at the URL identified by the object of the mf:action property as Turtle.

Turtle Evaluation Test

A Turtle evaluation test consists of an action and a result. The result is an N-triples file. The system under test is expected to parse the file given by the action and as a result produce the triples given in the result file (and no other triples).

The output of parsing should be compared to the N-triples result using blank node isomorphism. The two graphs, one from the Turtle file, one from the N-triples file are the same except for triple order and for consistently mapping blank nodes in one file to blank nodes in the other.

 <#turtle-eval-struct-01> rdf:type rdft:TestTurtleEval ;
   mf:name    "turtle-eval-struct-01" ;
   mf:action  <turtle-eval-struct-01.ttl> ;
   mf:result  <turtle-eval-struct-01.nt> ;
   .

Turtle Negative Evaluation Test

A negative evaluation test is passed if the system under test produces an error while processing the contents given by the action.

 <#turtle-eval-bad-01> rdf:type rdft:TestTurtleNegativeEval ;
   mf:name    "turtle-eval-bad-01" ;
   mf:action  <turtle-eval-bad-01.ttl> ;
   .

Note: passing a negative evaluation test may be generating a parse error or by some other indication of error in the file.

The test may be covering a feature that can be detected at parse time (e.g. a URI with a valid escape sequence for a bad IRI character) but may be detected in a later process stage, depending on implementation design. They are not violating the strict grammar of Turtle, but some other feature required of a parser (e.g. valid IRIs).

TriG Tests

The TriG tests are at <https://dvcs.w3.org/hg/rdf/file/default/trig/tests>. The manifest has four types of tests:

  • Positive Syntax Test (rdft:TestTriGPositiveSyntax),
  • Negative Syntax Test (rdft:TestTrigNegativeSyntax),
  • Positive Evaluation Test (rdft:TestTrigPositiveEvaluation), and
  • Negative Evaluation Test (rdft:TestTrigNegativeEvaluation).

Syntax tests are handled similar to the Turtle Syntax tests. Evaluation tests are also handled similar to Turtle Evaluation tests, except tests are for Dataset isomorphism rather than graph isomorphism.

Relative IRI Resolution

The home of the test suite is < https://dvcs.w3.org/hg/rdf/file/default/trig/tests>. Per RFC 3986 section 5.1.3, the base IRI for parsing each file is the retrieval IRI for that file.

Character Encoding

As with Turtle, the TriG language uses UTF-8 encoding.

N-Triples Syntax Test

This is an example of a syntax test:

 <#nt-syntax-file-01> rdf:type rdft:TestNTriplesPositiveSyntax ;
    mf:name    "nt-syntax-file-01" ;
    mf:action  <nt-syntax-file-01.nt> ;
    .

A test is passed if the system under test can parse the content at the URL identified by the object of the mf:action property as N-Triples.

N-Triples Negative Syntax Test

 <#nt-syntax-bad-uri-01> rdf:type rdft:TestNTriplesNegativeSyntax ;
   mf:name    "nt-syntax-bad-uri-01" ;
   mf:action  <nt-syntax-bad-uri-01.nt> ;
   .

A test is passed if the system under test raises an error when it parses the content at the URL identified by the object of the mf:action property as N-Triples.

N-Quads Syntax Test

This is an example of a syntax test:

 <#nq-syntax-file-01> rdf:type rdft:TestN Quads PositiveSyntax ;
    mf:name    "nq-syntax-file-01" ;
    mf:action  <nq-syntax-file-01.nq> ;
    .

A test is passed if the system under test can parse the content at the URL identified by the object of the mf:action property as N-Quads.

N-Quads Negative Syntax Test

 <#nq-syntax-bad-uri-01> rdf:type rdft:TestN Quads NegativeSyntax ;
   mf:name    "nq-syntax-bad-uri-01" ;
   mf:action  <nq-syntax-bad-uri-01.nq> ;
   .

A test is passed if the system under test raises an error when it parses the content at the URL identified by the object of the mf:action property as N-Quads.