CSRBench

CSRBench: Correctness checking Benchmark for Streaming RDF/SPARQL Version 0.1

Introduction

SRBench [1] is a Streaming RDF/SPARQL Benchmark that aims at assessing the abilities of streaming RDF/SPARQL engines.

CSRBench extends this benchmark for checking correctness of query results, w.r.t. their operational semantics.

Benchmark Queries

The queries provided in this page are parametrized, i.e. they receive parameters that need to be filled in order to make them full compilable SPARQL streaming queries. The parameters are identified as text surrounded by "%" characters, e.g. %WSIZE%.

The full 7 instantiated queries with actual parameters can be found in the test code for the CSRBench oracle:

https://github.com/dellaglio/csrbench-oracle-engines

Q1, Q2 and Q5.

Query description

This is a basic but important query, asking for the air temperature observations, their values and the sensor that reported them, in a time window. It tests an engines ability to handle combinations of window sizes (WSIZE) and slides (WSLIDE) in RDF/SPARQL engines. By varying the window size and slide, the query allows testing different cases: different window sizes and window slides (e.g. 10, 100, 1000 ms., etc.). Therefore, the value assigned to these two parameters will allow obtaining sliding windows (slide is smaller than size) or tumbling windows (slide is equal to size). The additional %TEMP% parameter filters the data, which is used to limit the number of matching results in the graph pattern. This will influence the number of data items in the window content.

Note: queries 1,2 and 5 in CSRBench oracle tests are instances of this query

SPARQL_Stream Query

PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> 
PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> 

SELECT ?sensor ?tempvalue ?obs 
FROM NAMED STREAM <http://cwi.nl/SRBench/observations> [NOW - %WSIZE% MS SLIDE %WSLIDE% MS]
WHERE { 
  ?obs om-owl:observedProperty weather:_AirTemperature ;  
       om-owl:procedure ?sensor ;  
       om-owl:result [om-owl:floatValue ?tempvalue] .
  FILTER(?tempvalue > %TEMP%)
}

CQELS Query

PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> 
PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> 
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
SELECT ?sensor ?obs 
WHERE { 
  STREAM <http://ex.org/streams/test> [RANGE %WSIZE%s SLIDE %WSLIDE%s] { 
   ?obs om-owl:observedProperty weather:_AirTemperature ; 
	om-owl:procedure ?sensor ; 
	om-owl:result ?res . 
   ?res om-owl:floatValue ?value . 
  } 
  FILTER(?value > %TEMP%) 
}

C-SPARQL Query

PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> 
PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> 

REGISTER QUERY q AS SELECT ?sensor ?tempvalue ?obs 
FROM NAMED STREAM <http://cwi.nl/SRBench/observations> [RANGE %WSIZE% MS STEP %WSLIDE% MS]
WHERE { 
  ?obs om-owl:observedProperty weather:_AirTemperature ;  
       om-owl:procedure ?sensor ;  
       om-owl:result [om-owl:floatValue ?tempvalue] .
  FILTER(?tempvalue > %TEMP%)
}

Q3.

Query description

A variation of Q1, but focused on a different observed property (Relative Humidity). This query tests the engine's ability to handle less frequent events (humidity occurs less often than temperature), even if both are in the same input RDF stream. The same window variations as in Q1 are applicable

SPARQL_Stream Query

PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> 
PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> 

SELECT ?sensor ?humidvalue ?obs 
FROM NAMED STREAM <http://cwi.nl/SRBench/observations> [NOW - %WSIZE% MS SLIDE %WSLIDE% MS]
WHERE { 
  ?obs om-owl:observedProperty weather:_RelativeHumidity ;  
       om-owl:procedure ?sensor ;  
       om-owl:result [om-owl:floatValue ?humidvalue] .
  FILTER(?tempvalue > %HUMID%)
}

CQELS Query

PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> 
PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> 
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
SELECT ?sensor ?obs 
WHERE { 
  STREAM <http://ex.org/streams/test> [RANGE %WSIZE%s SLIDE %WSLIDE%s] { 
   ?obs om-owl:observedProperty weather:_RelativeHumidity ; 
	om-owl:procedure ?sensor ; 
	om-owl:result ?res . 
   ?res om-owl:floatValue ?value . 
  } 
  FILTER(?value > %HUMID%) 
}

C-SPARQL Query

PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> 
PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> 

REGISTER QUERY q AS SELECT ?sensor ?humidvalue ?obs 
FROM NAMED STREAM <http://cwi.nl/SRBench/observations> [RANGE %WSIZE% MS STEP %WSLIDE% MS]
WHERE { 
  ?obs om-owl:observedProperty weather:_RelativeHumidity ;  
       om-owl:procedure ?sensor ;  
       om-owl:result [om-owl:floatValue ?humidvalue] .
  FILTER(?tempvalue > %HUMID%)
}

Q4.

Query description

Aggregate queries are commonly used in stream processing, as the focus is sometimes on data trends and summarization rather than on individual data points. Aggregates pose challenges to the computation of the window content, and depending on the streaming processor report and tick policies, the results of a sum, average or other function may greatly vary. This type of issues are often overlooked when querying single stream triples.

SPARQL_Stream Query

PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> 
PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> 
SELECT (AVG(?value) AS ?avg)
FROM NAMED STREAM <http://cwi.nl/SRBench/observations> [NOW - %WSIZE% S SLIDE %WSLIDE% S]
WHERE { 
  ?obs om-owl:observedProperty weather:_AirTemperature ;  
       om-owl:procedure ?sensor ;  
       om-owl:result ?res .
  ?res om-owl:floatValue ?value .
  FILTER(?value > $TEMP$)
}

CQELS Query

PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> 
PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> 
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
SELECT (AVG(?value) AS ?avg)  
WHERE { 
  STREAM <http://ex.org/streams/test> [RANGE %WSIZE%s SLIDE %WSLIDE%s] { 
   ?obs om-owl:observedProperty weather:_AirTemperature ; 
	om-owl:procedure ?sensor ; 
	om-owl:result ?res . 
   ?res om-owl:floatValue ?value . 
  } 
  FILTER(?value > %TEMP%) 
}

C-SPARQL Query

PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> 
PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> 
REGISTER QUERY q AS SELECT (AVG(?value) AS ?avg)
FROM NAMED STREAM <http://cwi.nl/SRBench/observations> [RANGE %WSIZE% S STEP %WSLIDE% S]
WHERE { 
  ?obs om-owl:observedProperty weather:_AirTemperature ;  
       om-owl:procedure ?sensor ;  
       om-owl:result ?res .
  ?res om-owl:floatValue ?value .
  FILTER(?value > $TEMP$)
}

Q6.

Query description

The previous queries include graph pattern matching of triples that are typically received at the same timestamp (or nearly): e.g. an observation and its value, its type, etc. However, there are cases where queries including joins at different timestamps may be relevant. This is more challenging for query engines and correctness checking. This query asks for sensor stations that record a high atmospheric temperature variation, in a time window.

SPARQL_Stream Query

PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#>
PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#>
SELECT ?sensor ?ob1 ?value1 ?ob2
FROM NAMED STREAM <http://cwi.nl/SRBench/observations>[NOW - %WSIZE% S SLIDE %WSLIDE% S]
WHERE { 
{ ?ob1 om-owl:procedure ?sensor ;
       om-owl:observedProperty weather:_AirTemperature ;
       om-owl:result [om-owl:floatValue ?value1].}
{ ?ob2 om-owl:procedure ?sensor ;
       om-owl:observedProperty weather:_AirTemperature ;
       om-owl:result [om-owl:floatValue ?value2].}
  FILTER(?value1-?value2 > %VARIATION_THRESHOLD%)
}

CQELS Query

PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#>
PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#>
SELECT ?sensor ?ob1 ?value1 ?ob2
WHERE { 
  STREAM <http://ex.org/streams/test>[RANGE %WSIZE%s STEP %WSLIDE%s]{
  ?ob1 om-owl:procedure ?sensor ;
       om-owl:observedProperty weather:_AirTemperature ;
       om-owl:result [om-owl:floatValue ?value1].
  ?ob2 om-owl:procedure ?sensor ;
       om-owl:observedProperty weather:_AirTemperature ;
       om-owl:result [om-owl:floatValue ?value2].}
  FILTER(?value1-?value2 > %VARIATION_THRESHOLD%)
}

C-SPARQL Query

PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#>
PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#>
REGISTER QUERY q AS SELECT ?sensor ?ob1 ?value1 ?ob2
FROM NAMED STREAM <http://cwi.nl/SRBench/observations>[RANGE %WSIZE% S STEP %WSLIDE% S]
WHERE { 
  ?ob1 om-owl:procedure ?sensor ;
       om-owl:observedProperty weather:_AirTemperature ;
       om-owl:result [om-owl:floatValue ?value1].
  ?ob2 om-owl:procedure ?sensor ;
       om-owl:observedProperty weather:_AirTemperature ;
       om-owl:result [om-owl:floatValue ?value2].
  FILTER(?value1-?value2 > %VARIATION_THRESHOLD%)
}

Q7.

Query description

This query also exploits joins between triples in different timestamps, but this time it joins not on the sensor as in the previous query, but it is a cross product of all temperature observations and a particular fized temperature observation, comparing if its value is greater.

SPARQL_Stream Query

PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> 
PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> 
PREFIX sens-obs: <http://knoesis.wright.edu/ssw/>
SELECT ?sensor ?ob1     
FROM NAMED STREAM <http://cwi.nl/SRBench/observations>[NOW - %WSIZE% S SLIDE %WSLIDE% S]
WHERE { 
  {?ob  om-owl:procedure sens-obs:System_C1190 ; 
        om-owl:observedProperty weather:_AirTemperature ;       
        om-owl:result [om-owl:floatValue ?value]}
  {?ob1 om-owl:procedure ?sensor ; 
        om-owl:observedProperty weather:_AirTemperature ;         
        om-owl:result [om-owl:floatValue ?value1]}
  FILTER(?value1>?value)
}

CQELS Query

PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> 
PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> 
PREFIX sens-obs: <http://knoesis.wright.edu/ssw/>
REGISTER QUERY q AS SELECT ?sensor ?ob1     
WHERE { 
  STREAM <http://ex.org/streams/test>[RANGE %WSIZE%s SLIDE %WSLIDE%s]{
  ?ob  om-owl:procedure sens-obs:System_C1190 ; 
        om-owl:observedProperty weather:_AirTemperature ;       
        om-owl:result [om-owl:floatValue ?value]
  ?ob1 om-owl:procedure ?sensor ; 
        om-owl:observedProperty weather:_AirTemperature ;         
        om-owl:result [om-owl:floatValue ?value1]}
  FILTER(?value1>?value)
}

C-SPARQL Query

PREFIX om-owl: <http://knoesis.wright.edu/ssw/ont/sensor-observation.owl#> 
PREFIX weather: <http://knoesis.wright.edu/ssw/ont/weather.owl#> 
PREFIX sens-obs: <http://knoesis.wright.edu/ssw/>
REGISTER QUERY q AS SELECT ?sensor ?ob1     
FROM NAMED STREAM <http://cwi.nl/SRBench/observations>[RANGE %WSIZE% S STEP %WSLIDE% S]
WHERE { 
  ?ob  om-owl:procedure sens-obs:System_C1190 ; 
        om-owl:observedProperty weather:_AirTemperature ;       
        om-owl:result [om-owl:floatValue ?value].
  ?ob1 om-owl:procedure ?sensor ; 
        om-owl:observedProperty weather:_AirTemperature ;         
        om-owl:result [om-owl:floatValue ?value1].
  FILTER(?value1>?value)
}