SWEO Community Project: Linking Open Data on the Semantic Web

OLTP Benchmarks for Triplestores

This page is the home of a community effort to define OLTP benchmarks for triplestores. See this message for background.

This effort is part of the Linking Open Data Project

A list of existing RDF store benchmarks if found here.

Overview

The goal of this effort is to define a standard set of OLTP benchmarks that can be run against any triplestore in order to understand its performance in write-heavy environments.

Deliverables

Document with tests + dataset description
Reference program for running tests

Roadmap

Define questions to be addressed by tests
Use above to define initial set of tests
Define synthetic, reference dataset
Draft document
Publish document (where?)

Questions To Be Addressed

The following questions should inform the development of the tests.

Adds

How quickly can I load a big batch of triples?
What size transactions work well for this?
How is add throughput affected by the size of the triplestore?
If delayed indexing is possible, is it beneficial to do so for bulk loads?

Updates

Once populated, how quickly can I write batches of updates?
Small/medium/big batches?
How is update throughput affected by the size of the triplestore?

Deletes

How quickly can I delete a big batch of triples?
What size transactions work well for this?
How is delete throughput affected by the size of the triplestore?

Tests

TBD.

Dataset Description

TBD.

Participants

Chris Wilper