PROV-AQ: Provenance Access and Query

1.1 Concepts

In defining the specification below, we make use of the following concepts.

Provenance information

refers to provenance represented in some fashion.

Provenance-URI

a URI denoting some provenance information.

Entity

an aspect of a resource, about which one wishes to present some provenance information. For example, a weather report for a given date may be an aspect of a resource that is maintained as the current weather report. An entity is itself a resource. See also [PROV-DM], and [WEBARCH] section 2.3.2.

Entity-URI

a URI denoting an entity, which allows that entity to be identified for the purpose of finding and expressing provenance information (see section 1.2 for discussion)

Provenance service

a service that provides a provenance-URI or provenance information given a resource URI or an entity-URI.

Service-URI

the URI of a provenance service.

Resource

also referred to as web resource: a resource as described by the Architecture of the World Wide Web [WEBARCH], section 2.2. A resource may be associated with multiple entities (see section 1.2 for discussion)

1.2 Provenance, entities and resources

Fundamentally, provenance information is about resources. In general, resources may vary over time and context. E.g., a resource describing the weather in London changes from day-to-day, or one listing restaurants near you will vary depending on your location. Provenance information, to be useful, must be persistent and not itself dependent on context. Yet we may still want to make provenance assertions about dynamic or context-dependent web resources (e.g. the weather forecast for London on a particular day may have been derived from a particular set of Meteorological Office data).

Provenance descriptions of dynamic and context-dependent resources are possible through the notion of entities. An entity is simply a web resource that is a contextualized view or instance of an original web resource. For example, a W3C specification typically undergoes several public revisions before it is finalized. A URI that refers to the "current" revision might be thought of as denoting the specification through its lifetime. Separate URIs for each individual revision would then be entity-URIs, denoting the specification at a particular stage in its development. Using these, we can make provenance assertions that a particular revision was published on a particular date, and was last modified by a particular editor. Entity-URIs may use any URI scheme, and are not required to be dereferencable.

Requests for provenance about a resource may return provenance information that uses one or more entity-URIs to refer to versions of that resource. Some given provenance information may use multiple entity-URIs if there are assertions referring to the same underlying resource in different contexts. For example, provenance information describing a W3C document might include information about all revisions of the document using statements that use the different entity-URIs of the various revisions.

In summary, a key notion within the concepts outlined above is that provenance information may be not universally applicable to a resource, but may be expressed with respect to that resource in a restricted context (e.g. at a particular time). This restricted view is called an entity, and an entity-URI is used to refer to it within provenance information.

1.3 Interpreting provenance information

Provenance information describes relationships between entities, activities and agents. As such, any given provenance information may contain information about several entities. Within some provenance information, the entities thus described are identified by their Entity-URIs.

When interpreting provenance information, it is important to be aware that statements about several entities may be present, and to be accordingly selective when using the information provided. (In some exceptional cases, it may be that the provenance information returned does not contain any information relating to a specific associated entity.)

3.1 Resource accessed by HTTP

For a document accessible using HTTP, provenance information may be indicated using an HTTP Link header field, as defined by Web Linking (RFC 5988) [LINK-REL]. The Link header field is included in the HTTP response to a GET or HEAD operation (other HTTP operations are not excluded, but are not considered here).

A provenance link relation type for referencing provenance information is registered according to the template in section 7. IANA considerations, and may be used as shown::

Link: provenance-URI; rel="provenance"; anchor="entity-URI"

If no anchor link is provided then the entity-URI is assumed to be the URI of the resource.

At this time, the meaning of these links returned with other HTTP response codes is not defined: future revisions of this specification may define interpretations for these.

An HTTP response may include multiple provenance link header fields, indicating a number of different provenance resources that are known to the responding server, each providing provenance information about the accessed resource.

The presence of a provenance link in an HTTP response does not preclude the possibility that other publishers may offer provenance information about the same resource. In such cases, discovery of the additional provenance information must use other means (e.g. see section 4. Provenance services).

Provenance resources indicated in this way are not guaranteed to be authoritative. Trust in the linked provenance data must be determined separately from trust in the original resource, just as in the web at large, it is a users' responsibility to determine an appropriate level of trust in any other linked resource; e.g. based on the domain that serves it, or an associated digital signature. (Ssee also section 8. Security considerations.)

Link: provenance-service-URI; anchor="entity-URI"; rel="provenance-service"

3.2 Resource represented as HTML

For a document presented as HTML or XHTML, without regard for how it has been obtained, provenance information may be associated with a resource by adding a <Link> element to the HTML <head> section. Two new link relation types for referencing provenance information are registered according to the template in section 7. IANA considerations, and may be used as shown:

  <html xmlns="http://www.w3.org/1999/xhtml">
     <head>
        <link rel="provenance" href="provenance-URI">
        <link rel="anchor" href="entity-URI">
        <title>Welcome to example.com</title>
     </head>
     <body>
        ...
     </body>
  </html>

The provenance-URI given by the provenance link element identifies the provenance-URI for the document.

The entity-URI given by the anchor link element specifies an identifier for the presented document view, and which may be used within the provenance information when referring to this document.

An HTML document header may include multiple "provenance" link elements, indicating a number of different provenance resources that are known to the creator of the document, each of which may provide provenance information about the document.

Likewise, the header may include multiple "anchor" link elements indicating that, e.g., different revisions of the document can be identified in the provenance information using the different entity-URIs.

If no "anchor" link element is provided then the entity-URI is assumed to be the URI of the document. It is recommended that this convention be used only when the document is static.

  <html xmlns="http://www.w3.org/1999/xhtml">
     <head>
        <link rel="provenance-service" href="service-URI">
        <link rel="anchor" href="entity-URI">
        <title>Welcome to example.com</title>
     </head>
     <body>
        ...
     </body>
  </html>

3.3 Resource represented as RDF

If a resource is represented as RDF (in any of its recognized syntaxes, including RDFa), it may contain references to its own provenance using additional RDF statements.

For this purpose a new RDF property, prov:hasProvenance, is defined as a relation between two resources, where the object of the property is a resource that provides provenance information about the subject resource. Multiple prov:hasProvenance assertions may be made about a subject resource.

Another new RDF property, prov:hasAnchor, is defined to allow the RDF content to specify one or more entity-URIs of the RDF document for the purpose of provenance information (similar to the use of the "anchor" link relation in HTML).

@@TODO: document namespace. Check naming style. Use provenance model namespace? Define as part of model?
@@TODO: example, when vocabulary issues are settled.

3.4 Arbitrary data

We have so far decided not to try and define a common mechanism for arbitrary data, because it's not clear to us what the correct choice would be. Is this a reasonable position, or is there a real need for a generic solution for provenance discovery for arbitrary, non-web-accessible data objects?

If a resource is represented using a data format other than HTML or RDF, and no URI for the resource is known, provenance discovery becomes trickier to achieve. This specification does not define a specific mechanism for such arbitrary resources, but this section discusses some of the options that might be considered.

For formats which have provision for including metadata within the file (e.g. JPEG images, PDF documents, etc.), use the format-specific metadata to include a entity-URI, provenance-URI and/or service-URI. Format-specific metadata provision might also be used to include provenance information directly in the resource.

Use a generic packaging format that can combine an arbitrary data file with a separate metadata file in a known format, such as RDF. At this time, it is not clear what format that should be, but some possible candidates are:

• MIME multipart/related [RFC2387]: both email and HTTP are based on MIME or MIME-derivatives, so this has the advantage of working well with the network transfer mechanisms discussed in the motivating scenarios considered.
• Composite object-packaging work from the digital library community, of which there are several (ORE, MPEG-21, BagIt @@refs) to name a handful. Practical implementations of these seem to commonly be based on the ZIP file format.
• Packaging formats along the lines of those used for shipping Java web applications or (basically, a ZIP file with a manifest and some imposed structure)
• Ongoing work in the research community (e.g. Why Linked Data is Not Enough for Scientists, ePub, etc.) to encapsulate data, code, annotations and metadata into a common exchangeable format.

Fix references in above text.

4.1 Using the provenance service API

This section describes general procedures for using the provenance service API. Later sections describe the resources presented by the API, and their representation using JSON. section B. Provenance service format examplesgives examples of alternative representations. Normal HTTP content negotiation mechanisms may be used to retrieve representations using formats convenient for the client application.

4.2 Resources presented and representations used

{
  "provenance_service_uri":         "http://example.org/provenance_service/",
  "provenance_locations_template":  "http://example.org/provenance_service/locations/?uri={uri}",
  "provenance_content_template":    "http://example.org/provenance_service/provenance/?uri={uri}"
}

{
  "uri": "http://example.org/qdata/",
  "provenance": [
    "http://source1.example.org/provenance/qdata/",
    "http://source2.example.org/prov/qdata/",
    "http://source3.example.com/prov?id=qdata"
  ]
}

4.3 Provenance service discovery

This specification does not define any specific mechanism for discovering provenance services. Applications may use any appropriate mechanism, including but not limited to: prior configuration, search engines, service registries, etc.

5.1 Find provenance-URI given entity-URI of resource

If the requester has an entity-URI, a simple SPARQL query may be used to return the corresponding provenance-URI. E.g., if the original resource has a entity-URI http://example.org/resource,

  @prefix prov: <@@TBD>
  SELECT ?provenance_uri WHERE
  {
    <http://example.org/resource> prov:hasProvenance ?provenance_uri
  }
            

@@TODO: specific provenance namespace and property to be determined by the model or ontology specification?

5.2 Find Provenance-URI given identifying information about a resource

If the requester has identifying information that is not the URI of the original resource, then they will need to construct a more elaborate query to locate an entity description and obtain its provenance-URI(s). The nature of identifying information that can be used in this way will depend upon the third party service used, further definition of which is out of scope for this specification. For example, a query for a document identified by a DOI, say 1234.5678, using the PRISM vocabulary [PRISM] recommended by FaBio [FABIO], might look like this:

@prefix prov: <@@TBD>
@prefix prism: <http://prismstandard.org/namespaces/basic/2.0/>
SELECT ?provenance_uri WHERE
{
  [ prism:doi "1234.5678" ] prov:hasProvenance ?provenance_uri
}

@@TODO: specific provenance namespace and property to be determined by the model specification?

5.3 Obtain provenance information directly given an entity-URI of a resource

This scenario retrieves provenance information directly given the URI of a resource or entity, and may be useful where the provenance information has not been assigned a specific URI, or when the calling application is interested only in specific elements of provenance information.

If the original resource has an entity-URI http://example.org/resource, a SPARQL query for provenance information might look like this:

@prefix prov: <@@TBD>
CONSTRUCT
{
  <http://example.org/resource> ?p ?v
}
WHERE
{
  <http://example.org/resource> ?p ?v
}

@@TODO: specific provenance namespace and property to be determined by the model specification? The above query pattern assumes provenance information is included in direct properties about the entity. When an RDF provenance vocabulary is fully formulated, this may well turn out to not be the case. A better example would be one that retrieves specific provenance information when the vocabulary terms have been defined.

6.1 Via Web Retrieval

Publishers are not required to publish all the provenance information associated with a given entity at a particular provenance-URI. The amount of provenance information exposed is application dependent. However, it is possible to incrementally retrieve (i.e. walk the provenance graph) by progressively looking up provenance information using HTTP. The pattern is as follows:

1. For a given entity (entity-uri-1) retrieve it's associated provenance-uri-1 using the HTTP Link header (section 3.1 Resource accessed by HTTP)
2. Dereference provenance-uri-1
3. Navigate the provenance information
4. When reaching a dead-end during navigation, that is on encountering a reference to an entity (entity-uri-2) with no provided provenance information, find its provenance-URI and continue from Step 1. (Note: an HTTP HEAD operation may be used to obtain the Link headers without retrieving the entity content.)

To reduce the overhead of multiple HTTP requests, a provenance information publisher may link entities to their associated provenance information using the prov:hasProvenance predicate. Thus, the same pattern above applies, except instead of having to retrieve a new Link header field, one can immediately dereference the entity's associated provenance.

The same approach can be adopted when using the provenance service API (section 4. Provenance services). However, instead of performing an HTTP HEAD or GET against a resource one queries the provenance service using the given entity-uri.

6.2 Via Queries

Provenance information may be made available using a SPARQL endpoint (section 5. Querying provenance information) [RDF-SPARQL-PROTOCOL] [RDF-SPARQL-QUERY]. Using SPARQL queries, provenance can be selectively retrieved using combinations of filters and or path queries.

7.1 Registration template for link relation: "provenance"

Relation Name:

provenance

Description:

the resource identified by target IRI of the link provides provenance information about the entity identified by the context link

Reference:

@@this spec, @@provenance-model-spec

Notes:

...

Application Data:

...

7.2 Registration template for link relation: "anchor"

The name "anchor" has been used for the link relation name, despite the corresponding URI being described as an entity-URI. This terminology has been chosen to align with usage in the description of the HTTP Link header field, per RFC 5988.

Relation Name:

anchor

Description:

when used in conjunction with a "provenance" link, the resource identified by target IRI of the link is an entity for which provenance information may be provided. This may be used, for example, to isolate relevant information from a referenced document that contains provenance information for several entities.

Reference:

@@this spec, @@provenance-model-spec

Notes:

...

Application Data:

...

7.3 Registration template for link relation: "provenance-service"

Relation Name:

provenance-service

Description:

the resource identified by target URI of the link is an provenance service per section 4. Provenance services of this specification.

Reference:

@@this spec, @@provenance-model-spec

Notes:

...

Application Data:

...