Please refer to the errata for this document, which may include some normative corrections.
The previous errata for this document, are also available.
See also translations.
This document is also available in these non-normative formats: XML and XHTML with color-coded revision indicators.
Copyright © 2012 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark and document use rules apply.
RDF is a directed, labeled graph data format for representing information
in the Web. SPARQL can be used to express queries
across diverse data sources, whether the data is stored natively as RDF or
viewed as RDF via middleware. This specification defines the syntax and semantics of SPARQL 1.1
Federated Query extension for executing queries distributed over different SPARQL endpoints. The SERVICE
keyword extends SPARQL 1.1 to support queries that merge data distributed across the Web.
This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.
This document is being published as one of a set of 11 documents:
The W3C Director seeks review and feedback from W3C Advisory Committee representatives, via their review form by 6 December 2012. This will allow the Director to assess consensus and determine whether to issue this document as a W3C Recommendation.
Others are encouraged by the SPARQL Working Group to continue to send reports of implementation experience, and other feedback, to public-rdf-dawg-comments@w3.org (public archive). Reports of any success or difficulty with the test cases are encouraged. Open discussion among developers is welcome at public-sparql-dev@w3.org (public archive).
The advancement of this Proposed Recommendation is supported by the disposition of comments on the previous drafts, the Test Suite, and the list of implementations (with test results).
Publication as a Proposed Recommendation does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.
This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.
1 Introduction
1.1 Document Conventions
1.1.1 Namespaces
1.1.2 Result Descriptions
1.1.3 Terminology
2 SPARQL 1.1 Federated Query Extension
2.1 Simple query to a remote SPARQL endpoint
2.2 SPARQL query with OPTIONAL to two remote SPARQL endpoints
2.3 Service Execution Failure
2.4 Interplay of SERVICE and VALUES (Informative)
3 SPARQL 1.1 Simple Federation Extension: semantics
3.1 Translation to the SPARQL Algebra
3.2 SPARQL 1.1 Simple Federation Extension Algebra
3.2.1 SERVICE Examples
4 SERVICE Variables (Informative)
5 Conformance
6 Security Considerations (Informative)
A References
A.1 Normative References
A.2 Other References
B Acknowledgements
C CVS History (Last Call and after)
The growing number of SPARQL query services offer data consumers an opportunity to merge data distributed across the Web. This specification defines the syntax and semantics of the SERVICE
extension to the SPARQL 1.1 Query Language. This extension allows a query author to direct a portion of a query to a particular SPARQL endpoint. Results are returned to the federated query processor and are combined with results from the rest of the query.
This document uses the same namespaces as from the SPARQL 1.1 Query document.
Result sets are illustrated in tabular form as in the SPARQL 1.1 Query document.
A 'binding' is a pair (variable,
RDF term). There are three
variables:
x
, y
and z
(shown as column headers). Each
solution is shown as one row in the body of the table. Here, there is a single
solution, in which variable x
is bound to "Alice"
, variable
y
is bound to http://example/a
, and variable z
is not bound to an RDF term. Variables are not required to be bound in a
solution.
The following terms are defined in SPARQL 1.1 Query Language [SQRY] and reused in this document:
RDF URI reference
)
The SERVICE
keyword instructs a federated query processor to invoke a portion of a SPARQL query against a remote SPARQL endpoint. This section presents examples of how to use the SERVICE
keyword. The following sections define the syntax and semantics of this extension.
This example shows how to query a remote SPARQL endpoint and join the returned data with the data from the local RDF Dataset. Consider a query to find the names of the people we know. Data about the names of various people is available at the http://people.example.org/sparql
endpoint:
@prefix foaf: <http://xmlns.com/foaf/0.1/> . @prefix : <http://example.org/> . :people15 foaf:name "Alice" . :people16 foaf:name "Bob" . :people17 foaf:name "Charles" . :people18 foaf:name "Daisy" .and one wants to combine with a local FOAF file
http://example.org/myfoaf.rdf
that contains the single triple:
<http://example.org/myfoaf/I> <http://xmlns.com/foaf/0.1/knows> <http://example.org/people15> .
Query:
PREFIX foaf: <http://xmlns.com/foaf/0.1/> SELECT ?name FROM <http://example.org/myfoaf.rdf> WHERE { <http://example.org/myfoaf/I> foaf:knows ?person . SERVICE <http://people.example.org/sparql> { ?person foaf:name ?name . } }
This query, on the data above, has one solution:
Query Result:
Imagine we want to query people and optionally obtain their interests and the names of people they know. Imagine for instance, two endpoints containing data about people:
Data in the default graph at remote SPARQL endpoint: http://people.example.org/sparql
@prefix foaf: <http://xmlns.com/foaf/0.1/> . @prefix : <http://example.org/> . :people15 foaf:name "Alice" . :people16 foaf:name "Bob" . :people17 foaf:name "Charles" . :people17 foaf:interest <http://www.w3.org/2001/sw/rdb2rdf/> .
and data in the default graph the remote SPARQL endpoint: http://people2.example.org/sparql
@prefix foaf: <http://xmlns.com/foaf/0.1/> . @prefix : <http://example.org/> . :people15 foaf:knows :people18 . :people18 foaf:name "Mike" . :people17 foaf:knows :people19 . :people19 foaf:name "Daisy" .
Query:
PREFIX foaf: <http://xmlns.com/foaf/0.1/> SELECT ?person ?interest ?known WHERE { SERVICE <http://people.example.org/sparql> { ?person foaf:name ?name . OPTIONAL { ?person foaf:interest ?interest . SERVICE <http://people2.example.org/sparql> { ?person foaf:knows ?known . } } } }
This query, on the data above, has three solutions:
Query Result:
Notice that in the query above there is a nested SERVICE
in the OPTIONAL
clause. This query requires the SPARQL query service at http://people.example.org/sparql
to support basic federated query.
The execution of a SERVICE
pattern may fail due to several reasons: the remote service may be down, the service IRI may not be dereferenceable, or the endpoint may return an error to the query. Normally, under such circumstances the invoked query containing a SERVICE
pattern fails as a whole. Queries may explicitly allow failed SERVICE
requests with the use of the SILENT
keyword. The SILENT
keyword indicates that errors encountered while accessing a remote SPARQL endpoint should be ignored while processing the query. The failed SERVICE
clause is treated as if it had a result of a single solution with no bindings.
In the following query the SILENT
keyword is present. If the remote SPARQL endpoint is not available because the SPARQL endpoint does not exist, it is down or it is not accessible the query will return a solution sequence of one empty solution mapping. If the SILENT
keyword is not present, the query will stop and return the error.
Data in <http://people.example.org/sparql>
endpoint:
<http://example.org/people15> <http://xmlns.com/foaf/0.1/name> "Charles" .
SPARQL 1.1 Query includes the VALUES
clause (VALUES), which can be used to provide an unordered solution sequence that is joined with the results of the query evaluation. Implementers of SPARQL 1.1 Federated Query may use the VALUES
clause to constrain the results received from a remote endpoint based on solution bindings from evaluating other parts of the query.
The following example shows how SERVICE
and VALUES
can work together. Suppose a query that asks for all instances of foaf:Person in the default graph and also their known people in the remote endpoint http://example.org/sparql
:
Data in the default graph:
@prefix foaf: <http://xmlns.com/foaf/0.1/> . @prefix : <http://example.org/> . :a a foaf:Person ; foaf:name "Alan" ; foaf:mbox; "alan@example.org" . :b a foaf:Person ; foaf:name "Bob" ; foaf:mbox "bob@example.org" .
and data in the default graph the remote SPARQL endpoint http://example.org/sparql
:
@prefix foaf: <http://xmlns.com/foaf/0.1/> . @prefix : <http://example.org/> . :a foaf:knows :b . :b foaf:knows :c . :c foaf:knows :a . :a foaf:interest "SPARQL 1.1 Basic Federated Query" . :b foaf:interest "SPARQL 1.1 Query" . :c foaf:interest "RDB2RDF Direct mapping" .
Query:
PREFIX foaf: <http://xmlns.com/foaf/0.1/> SELECT ?s { ?s a foaf:Person . SERVICE <http://example.org/sparql> {?s foaf:knows ?o } }
When the original query is executed naively, with an unconstrained service call the endpoint may return more results than necessary. It may also happen that the SPARQL endpoint will not return all of them. Many existing SPARQL endpoints have restrictions in the number of results they return and may miss the ones matching subjects ?s
from the local default graph. Thus, an implementation of a query planner for federated queries may decide to decompose the query into two queries instead, where first the bindings from the local default graph are evaluated:
Query:
PREFIX : <http://example.org/> PREFIX foaf: <http://xmlns.com/foaf/0.1/> SELECT ?s { ?s a foaf:Person }
This query, on the data above, has two solutions:
Query Result:
Next, dispatch to the remote endpoint <http://example.org/sparql> a constrained query with the solutions for ?s
:
PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX : <http://example.org/> SELECT * {?s foaf:knows ?o } VALUES (?s) { (:a) (:b) }
The query process involving SERVICE
limits the data returned to the data it needs for the overall query:
Query:
PREFIX foaf: <http://xmlns.com/foaf/0.1/> SELECT ?s ?o { ?s a foaf:Person SERVICE <http://example.org/sparql> {?s foaf:knows ?o } }
This query, on the data above using VALUES
, has the expected two solutions to the overall query:
Query Result:
The SERVICE
extension is defined as an additional type of GroupGraphPattern, with an accompanying addition to SPARQL Query 1.1's Transform (syntax form):
If the form is
GroupGraphPattern
From the Translate Graph Patterns section of [SPARQL 1.1 Query Language] we extend the transformation of GroupGraphPattern to define the transformation of
SERVICE
patterns:
Let FS := the empty set Let G := the empty pattern, Z, a basic graph pattern which is the empty set. Let SilentOp := boolean, indicating SERVICE error behavior. For each element E in the GroupGraphPattern If E is of the form FILTER(expr) FS := FS ∪ {expr} End If E is of the form OPTIONAL{P} Let A := Transform(P) If A is of the form Filter(F, A2) G := LeftJoin(G, A2, F) Else G := LeftJoin(G, A, true) End End If E is of the form MINUS{P} G := Minus(G, Transform(P)) End If E is of the form BIND(expr AS var) G := Extend(G, var, expr) End If E is any other form Let A := Transform(E) G := Join(G, A) End If E is of the form SERVICE [SILENT] IRI {P} Let G := Join(G, Service(IRI, Transform(P), SilentOp)) End End If FS is not empty: Let X := Conjunction of expressions in FS G := Filter(X, G) The result is G.
The evaluation of SERVICE
is defined in terms of the SPARQL Results [RESULTS] returned by a SPARQL Protocol [SPROT] execution of the nested graph pattern:
Definition: Evaluation of a Service Pattern
LetΩ0
the solution set with one empty solution, andthen:
SELECT * WHERE Q
against the service endpoint with IRI iri where Q is the serialization of P in SPARQL syntax, in case of a successful service invocation according to the SPARQL protocol, and otherwiseΩ0.
in case SilentOp is true, and otherwiseIn the folowing section we introduce two examples showing the evaluation of SERVICE
patterns in the SPARQL algebra:
Example: a SERVICE
graph pattern in a series of joins:
Example: a SERVICE SILENT
graph pattern in a series of joins:
In the this section we do not present official evaluation semantics for the SPARQL pattern SERVICE VAR
. We only provide indications about how the evaluation of the SPARQL pattern SERVICE VAR
can be evaluated.
A variable used in place of a service IRI indicates that the service call for any solution depends on that variable's binding in that solution. For instance, the default graph may contain data about which services contain data about project endpoints. We assume the following data on various projects that contains information about SPARQL endpoints where data about these projects (using the DOAP vocabulary) can be queried from:
@prefix void: <http://rdfs.org/ns/void#> . @prefix dc: <http://purl.org/dc/elements/1.1/> . @prefix doap: <http://usefulinc.com/ns/doap#> . [] dc:subject "Querying RDF" ; void:sparqlEndpoint <http://projects1.example.org/sparql> . [] dc:subject "Querying RDF remotely" ; void:sparqlEndpoint <http://projects2.example.org/sparql> . [] dc:subject "Updating RDF remotely" ; void:sparqlEndpoint <http://projects3.example.org/sparql> .
Data in the default graph at remote SPARQL endpoint http://projects2.example.org/sparql:
_:project1 doap:name "Query remote RDF Data" . _:project1 doap:created "2011-02-12"^^xsd:date . _:project2 doap:name "Querying multiple SPARQL endpoints" . _:project2 doap:created "2011-02-13"^^xsd:date .
Data in the default graph at remote SPARQL endpoint http://projects3.example.org/sparql:
_:project3 doap:name "Update remote RDF Data" . _:project3 doap:created "2011-02-14"^^xsd:date .
We now want to query the project names of projects on the subject "remote":
PREFIX void: <http://rdfs.org/ns/void#> PREFIX dc: <http://purl.org/dc/elements/1.1/> PREFIX doap: <http://usefulinc.com/ns/doap#> SELECT ?service ?projectName WHERE { # Find the service with subject "remote". ?p dc:subject ?projectSubject ; void:sparqlEndpoint ?service . FILTER regex(?projectSubject, "remote") # Query that service projects. SERVICE ?service { ?project doap:name ?projectName . } }
In the following table we present the intuitive solutions for this query with the data above:
Query Result:
A SERVICE
clause involving a variable can be executed as a series of separate invocations of SPARQL query services. The results of each invocation are combined using union.
The query engine must determine the possible target SPARQL query services. The exact mechanism for doing this is not defined in this document. Execution order may also be used to determine the list of services to to be tried. The example above suggests a specific order of execution: evaluating the basic graph pattern and filter outside the SERVICE
block first will yield bindings for ?service
which may then be used to evaluate the SERVICE
block:
?p dc:subject ?projectSubject ; void:sparqlEndpoint ?service FILTER regex(?projectSubject, "remote")
Once ?service
has been evaluated it is possible to execute SERVICE
for each value of ?service
:
SERVICE ?service { ?project doap:name ?projectName . }
Note that blank nodes are unique to any document which serializes them. Also, SERVICE
calls depend on the SPARQL Protocol [SPROT] which transfers serialized RDF documents making blank nodes unique between service calls.
See section 4 SPARQL 1.1 Federated Query Grammar regarding conformance of
SPARQL Query strings that include the SPARQL 1.1 Federated Query Extensions. See section 3.1 Definition of SERVICE for conformance of query results for the SERVICE
keyword.
This specification is intended for use in conjunction with the SPARQL 1.1 Query Language. See that specification for its conformance criteria.
SPARQL queries using SERVICE
imply that a URI will
be dereferenced, and that the result will be incorporated into a working data set. All of the security issues
of SPARQL Protocol 1.1 [SPROT] Section 3.1
SPARQL 1.1 Query [SQRY] Section 21, and
Uniform Resource Identifier
(URI): Generic Syntax [RFC3986] Section 7 should be considered.
The SPARQL 1.1 Federated Query document is a product of the whole of the W3C SPARQL Working Group, and our thanks for discussions, comments and reviews go to all present and past members.
In addition, we have had comments and discussions with many people through the working group comments list. All comments go to making a better document. Carlos would also like to particularly thank Jorge Pérez, Oscar Corcho and Marcelo Arenas for their discussions on the syntax and semantics of the Federated query extension.