Publishing LinkedData Update Logs
When RDB data is published on the Web e.g. as LinkedData it is important to keep track of DB (and hence RDF) updates so crawlers know what has changed (after the last crawl) and should be re-retrieved from that endpoint.
To have a centralized registry (such as e.g. implemented by PingTheSemanticWeb service) does not seem to be feasible when Linked Data becomes more popular - think of millions of Linked Data endpoints pinging such a registry.
Possible solution: Standardized LinkedData Update Logs
Let's assume the Example.com company provides a Linked Data endpoint with information about their products, employees etc. The endpoint is reachable via
The LOD endpoint contains a special LOD space below
http://example.com/lod/updates which contains information about updates.
http://example.com/lod/updates could for example return the following RDF:
http://example.com/lod/updates/2007 rdf:type r2rul:UpdateCollection . http://example.com/lod/updates/2008 rdf:type r2rul:UpdateCollection .
http://example.com/lod/updates/2008 could then return the following RDF:
http://example.com/lod/updates/2008/Jan rdf:type r2rul:UpdateCollection . http://example.com/lod/updates/2008/Feb rdf:type r2rul:UpdateCollection .
This nesting could continue until we finally reach a URL, which exposes all updates performed in a certain second in time. For very frequently updated LOD endpoints (e.g. Wikipedia) this interval of one second will be sufficiently small enough, so the related update information can be still easily retieved. For rarely updated LOD endpoints (e.g. a personal Weblog) links should only point to non-empty UpdateCollections in order to prevent crawlers from performing unnecessary HTTP requests.
http://example.com/lod/updates/2008/Jan/01/17/58/06 then would for example contain RDF links (and additional metadata) to the LinkedData documents updated on Jan 1st, 2008 at 17:58:06, e.g. following triples:
http://example.com/lod/updates/2008/Jan/01/17/58/06/123 r2rul:updatedResource http://example.com/lod/users/JohnDoe . http://example.com/lod/updates/2008/Jan/01/17/58/06/123 r2rul:updatedAt "20080101T00:00:01"^<xsd:dateTime> . http://example.com/lod/updates/2008/Jan/01/17/58/06/123 r2rul:updatedBy http://example.com/lod/users/JohnDoe .
Individual updates are identified by a sequential number (i.e. "123" in the example). Arbitrary meta data can be attached to these updates, such as the time of the update (probably redundant since that can be inferred from the URL) or a certain person who performed the update.
This mechanism as well as some base update log vocabulary (i.e. the r2rul:updatedResource, r2rul:updatedAt, r2rul:updatedBy properties) could be standardized by this XG.
- Relevance for this XG
- Timezone of timestamps