CSRBench
CSRBench: Correctness checking Benchmark for Streaming RDF/SPARQL Version 0.1
Introduction
SRBench [1] is a Streaming RDF/SPARQL Benchmark that aims at assessing the abilities of streaming RDF/SPARQL engines.
CSRBench extends this benchmark for checking correctness of query results, w.r.t. their operational semantics.
Benchmark Queries
The queries provided in this page are parametrized, i.e. they receive parameters that need to be filled in order to make them full compilable SPARQL streaming queries. The parameters are identified as text surrounded by "%" characters, e.g. %WSIZE%.
The full 7 instantiated queries with actual parameters can be found in the test code for the CSRBench oracle:
https://github.com/dellaglio/csrbench-oracle-engines
Q1, Q2 and Q5.
Query description
This is a basic but important query, asking for the air temperature observations, their values and the sensor that reported them, in a time window. It tests an engines ability to handle combinations of window sizes (WSIZE) and slides (WSLIDE) in RDF/SPARQL engines. By varying the window size and slide, the query allows testing different cases: different window sizes and window slides (e.g. 10, 100, 1000 ms., etc.). Therefore, the value assigned to these two parameters will allow obtaining sliding windows (slide is smaller than size) or tumbling windows (slide is equal to size). The additional %TEMP% parameter filters the data, which is used to limit the number of matching results in the graph pattern. This will influence the number of data items in the window content.
Note: queries 1,2 and 5 in CSRBench oracle tests are instances of this query
SPARQLStream Query
PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> SELECT ?sensor ?tempvalue ?obs FROM NAMED STREAM <http://cwi.nl/SRBench/observations> [NOW - %WSIZE% MS SLIDE %WSLIDE% MS] WHERE { ?obs om-owl:observedProperty weather:_AirTemperature ; om-owl:procedure ?sensor ; om-owl:result [om-owl:floatValue ?tempvalue] . FILTER(?tempvalue > %TEMP%) }
CQELS Query
PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> SELECT ?sensor ?obs WHERE { STREAM <http://ex.org/streams/test> [RANGE %WSIZE%s SLIDE %WSLIDE%s] { ?obs om-owl:observedProperty weather:_AirTemperature ; om-owl:procedure ?sensor ; om-owl:result ?res . ?res om-owl:floatValue ?value . } FILTER(?value > %TEMP%) }
C-SPARQL Query
PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> REGISTER QUERY q AS SELECT ?sensor ?tempvalue ?obs FROM NAMED STREAM <http://cwi.nl/SRBench/observations> [RANGE %WSIZE% MS STEP %WSLIDE% MS] WHERE { ?obs om-owl:observedProperty weather:_AirTemperature ; om-owl:procedure ?sensor ; om-owl:result [om-owl:floatValue ?tempvalue] . FILTER(?tempvalue > %TEMP%) }
Q3.
Query description
A variation of Q1, but focused on a different observed property (Relative Humidity). This query tests the engine's ability to handle less frequent events (humidity occurs less often than temperature), even if both are in the same input RDF stream. The same window variations as in Q1 are applicable
SPARQLStream Query
PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> SELECT ?sensor ?humidvalue ?obs FROM NAMED STREAM <http://cwi.nl/SRBench/observations> [NOW - %WSIZE% MS SLIDE %WSLIDE% MS] WHERE { ?obs om-owl:observedProperty weather:_RelativeHumidity ; om-owl:procedure ?sensor ; om-owl:result [om-owl:floatValue ?humidvalue] . FILTER(?tempvalue > %HUMID%) }
CQELS Query
PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> SELECT ?sensor ?obs WHERE { STREAM <http://ex.org/streams/test> [RANGE %WSIZE%s SLIDE %WSLIDE%s] { ?obs om-owl:observedProperty weather:_RelativeHumidity ; om-owl:procedure ?sensor ; om-owl:result ?res . ?res om-owl:floatValue ?value . } FILTER(?value > %HUMID%) }
C-SPARQL Query
PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> REGISTER QUERY q AS SELECT ?sensor ?humidvalue ?obs FROM NAMED STREAM <http://cwi.nl/SRBench/observations> [RANGE %WSIZE% MS STEP %WSLIDE% MS] WHERE { ?obs om-owl:observedProperty weather:_RelativeHumidity ; om-owl:procedure ?sensor ; om-owl:result [om-owl:floatValue ?humidvalue] . FILTER(?tempvalue > %HUMID%) }
Q4.
Query description
Aggregate queries are commonly used in stream processing, as the focus is sometimes on data trends and summarization rather than on individual data points. Aggregates pose challenges to the computation of the window content, and depending on the streaming processor report and tick policies, the results of a sum, average or other function may greatly vary. This type of issues are often overlooked when querying single stream triples.
SPARQLStream Query
PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> SELECT (AVG(?value) AS ?avg) FROM NAMED STREAM <http://cwi.nl/SRBench/observations> [NOW - %WSIZE% S SLIDE %WSLIDE% S] WHERE { ?obs om-owl:observedProperty weather:_AirTemperature ; om-owl:procedure ?sensor ; om-owl:result ?res . ?res om-owl:floatValue ?value . FILTER(?value > $TEMP$) }
CQELS Query
PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> SELECT (AVG(?value) AS ?avg) WHERE { STREAM <http://ex.org/streams/test> [RANGE %WSIZE%s SLIDE %WSLIDE%s] { ?obs om-owl:observedProperty weather:_AirTemperature ; om-owl:procedure ?sensor ; om-owl:result ?res . ?res om-owl:floatValue ?value . } FILTER(?value > %TEMP%) }
C-SPARQL Query
PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> REGISTER QUERY q AS SELECT (AVG(?value) AS ?avg) FROM NAMED STREAM <http://cwi.nl/SRBench/observations> [RANGE %WSIZE% S STEP %WSLIDE% S] WHERE { ?obs om-owl:observedProperty weather:_AirTemperature ; om-owl:procedure ?sensor ; om-owl:result ?res . ?res om-owl:floatValue ?value . FILTER(?value > $TEMP$) }
Q6.
Query description
The previous queries include graph pattern matching of triples that are typically received at the same timestamp (or nearly): e.g. an observation and its value, its type, etc. However, there are cases where queries including joins at different timestamps may be relevant. This is more challenging for query engines and correctness checking. This query asks for sensor stations that record a high atmospheric temperature variation, in a time window.
SPARQLStream Query
PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> SELECT ?sensor ?ob1 ?value1 ?ob2 FROM NAMED STREAM <http://cwi.nl/SRBench/observations>[NOW - %WSIZE% S SLIDE %WSLIDE% S] WHERE { { ?ob1 om-owl:procedure ?sensor ; om-owl:observedProperty weather:_AirTemperature ; om-owl:result [om-owl:floatValue ?value1].} { ?ob2 om-owl:procedure ?sensor ; om-owl:observedProperty weather:_AirTemperature ; om-owl:result [om-owl:floatValue ?value2].} FILTER(?value1-?value2 > %VARIATION_THRESHOLD%) }
CQELS Query
PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> SELECT ?sensor ?ob1 ?value1 ?ob2 WHERE { STREAM <http://ex.org/streams/test>[RANGE %WSIZE%s STEP %WSLIDE%s]{ ?ob1 om-owl:procedure ?sensor ; om-owl:observedProperty weather:_AirTemperature ; om-owl:result [om-owl:floatValue ?value1]. ?ob2 om-owl:procedure ?sensor ; om-owl:observedProperty weather:_AirTemperature ; om-owl:result [om-owl:floatValue ?value2].} FILTER(?value1-?value2 > %VARIATION_THRESHOLD%) }
C-SPARQL Query
PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> REGISTER QUERY q AS SELECT ?sensor ?ob1 ?value1 ?ob2 FROM NAMED STREAM <http://cwi.nl/SRBench/observations>[RANGE %WSIZE% S STEP %WSLIDE% S] WHERE { ?ob1 om-owl:procedure ?sensor ; om-owl:observedProperty weather:_AirTemperature ; om-owl:result [om-owl:floatValue ?value1]. ?ob2 om-owl:procedure ?sensor ; om-owl:observedProperty weather:_AirTemperature ; om-owl:result [om-owl:floatValue ?value2]. FILTER(?value1-?value2 > %VARIATION_THRESHOLD%) }
Q7.
Query description
This query also exploits joins between triples in different timestamps, but this time it joins not on the sensor as in the previous query, but it is a cross product of all temperature observations and a particular fized temperature observation, comparing if its value is greater.
SPARQLStream Query
PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> PREFIX sens-obs: <http://knoesis.wright.edu/ssw/> SELECT ?sensor ?ob1 FROM NAMED STREAM <http://cwi.nl/SRBench/observations>[NOW - %WSIZE% S SLIDE %WSLIDE% S] WHERE { {?ob om-owl:procedure sens-obs:System_C1190 ; om-owl:observedProperty weather:_AirTemperature ; om-owl:result [om-owl:floatValue ?value]} {?ob1 om-owl:procedure ?sensor ; om-owl:observedProperty weather:_AirTemperature ; om-owl:result [om-owl:floatValue ?value1]} FILTER(?value1>?value) }
CQELS Query
PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> PREFIX sens-obs: <http://knoesis.wright.edu/ssw/> REGISTER QUERY q AS SELECT ?sensor ?ob1 WHERE { STREAM <http://ex.org/streams/test>[RANGE %WSIZE%s SLIDE %WSLIDE%s]{ ?ob om-owl:procedure sens-obs:System_C1190 ; om-owl:observedProperty weather:_AirTemperature ; om-owl:result [om-owl:floatValue ?value] ?ob1 om-owl:procedure ?sensor ; om-owl:observedProperty weather:_AirTemperature ; om-owl:result [om-owl:floatValue ?value1]} FILTER(?value1>?value) }
C-SPARQL Query
PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> PREFIX sens-obs: <http://knoesis.wright.edu/ssw/> REGISTER QUERY q AS SELECT ?sensor ?ob1 FROM NAMED STREAM <http://cwi.nl/SRBench/observations>[RANGE %WSIZE% S STEP %WSLIDE% S] WHERE { ?ob om-owl:procedure sens-obs:System_C1190 ; om-owl:observedProperty weather:_AirTemperature ; om-owl:result [om-owl:floatValue ?value]. ?ob1 om-owl:procedure ?sensor ; om-owl:observedProperty weather:_AirTemperature ; om-owl:result [om-owl:floatValue ?value1]. FILTER(?value1>?value) }