1st BDE Workshop on Secure, Clean and Efficient energy

First Community Workshop for Societal Challenge Secure, Clean and fficient energy, 16th of June 2015 at Vertretung des Landes Nordrhein-Westfalen bei der Europäischen Union, Brussels

WORKSHOP SUMMARY

The aim of the workshop, the first of a scheduled series on the domain, was the identification of current and future challenges for data management and analysis in the energy domain; challenges to be tackled with the evolving Big Data technology. In the workshop real examples of the potential, challenges and complexities of using big data in the energy domain were discussed.

The outcome of the workshop supported the design and realisation of the necessary ICT infrastructure on which the deployment and use of the BigDataEurope platform will be based. The platform targets the facilitation of big data usage in real world examples and will consist of the architecture, components, guidelines and best practices.

BigDataEurope platform will offer to the interested participating third parties the opportunities of the latest European RTD developments, including real time streaming, multilingual data harvesting, data analytics and data visualisation.

The workshop was divided in three parts, namely:

Part I: General introduction to the BDE background, objectives and targets, as well as an overview of the tools and technologies envisaged within the project.

Specifically on the technical background, the BDE platform is built upon existing Big Data industry best-practices making use of the Lambda-Architecture that constitutes generic, scalable and fault-tolerant data processing architecture. The envisaged implementation integrates mature, existing, open-source components into a comprehensive software stack suitable for serving and consuming interoperable data. The platform will be available as an open source implementation maximizing software re-usability and community involvement, while paving the new comer path to data products and services.

The architecture of the BDE platform is tailored to consume high-volume streams of real-time data (e.g. sensor measurements, social network activity, mobility data) and process them in two parallel pipelines, namely the:

• batch pipeline: that handles data at preset time intervals (e.g. hourly/daily) using Map-Reduce algorithms to provide aggregated views and
• real-time pipeline: that interactively manipulates incoming data and provides data views up to a certain timeframe

Part II: Keynote presentations were given by invited speakers in selected data management related topics.

Topic A: Electricity Industry

The views of the Electricity Industry on Data value were presented by Mr Hans ten Berge (Secretary General of EurElectric).

Topic B: Resource Forecasting

The data management challenges in energy resource forecasting were presented by Mr Martin Qvist (Head of Super-Computing and BigData applications of VESTAS, the leading company in wind energy sector) and Prof. George Kallos (Forecasting Unit of University of Athens)

Topic C: System monitoring

The data management challenges in system monitoring and a candidate use case on the topic were presented by Mr. A.Papoutsakis of TERNA S.A., a RES developer and operator.

Topic D: Smart grids

The data challenges in Smart Grids field were presented by Dr. S. Tselepis (SmartGrid.eu platform) and Mr Tierry Pollet (ETP, Landis+Gyr).

Part III: The third section constituted the interactive part of the workshop during which the participants were split into two breakout sessions according to two types of stakeholder groups to serve the user requirements elicitation process. In the first group the candidate use cases were discussed namely the asset monitoring and resource forecasting, whereas in the second group the technology aspects were discussed.

Topic A: Monitoring and forecasting

The group discussed the general attitudes of people involved in the energy sector towards big data usage and potential. The key points that were raised and discussed were the following:

• In the industrial sector the data are generated internally and as such they are proprietary
• Large companies develop in-house BigData applications or the rely on available commercial tools provided by the major ICT companies
• The majority of the energy industry related companies they do not exploit the full value of their data, as they do not invest in BigData solutions
• The convergence of Information Technology (IT) with Operation Technology (OT) is of primary importance; this is a field for BigData applications
• Available standards in data exchange in energy domain (i.e. IEC 61400-25).

The group considered two options for candidate pilot applications on BigDataEurope platform, namely the development of a platform capable to provide a complete asset fleet operational and condition monitoring and/or the development of a platform capable to provide the data management of localised (point) weather prediction in country level.

Topic B: Technology

During the data acquisition phase, energy domain experts identified data heterogeneity as something that BDE could help with. Experts typically work with streams of data originating at sensors located on distributed asset devices. Other data of interest include more traditional, yet still streaming, multimedia data, such as video. These data are analysed both on the fly as well as in-situ, i.e. after having been stored in data centres. The various information models were discussed. For accessing data on a large scale some federated querying and aggregation solution would be required. This solution also needs to be able to convert the incoming streams into the desirable format, by making use of existing, standard mappings.

Energy experts indicated that they typically make use of in-house analysis tools, with R being the de facto standard. Other, commercial, software packages are also in use. Regarding processing and analysis, it often needs to take as soon as the data arrives, in a streaming fashion, as delays may incur costs, for instance when such analysis is used for the purpose of maintaining remote devices.

Regarding storage and curation, a number of items were raised, such as the need for mapping between standards used. It may be the case that these transformations take place on the fly in order to support streaming analysis, before results and byproducts are optionally transferred onto disk for longer-term storage.

BDE will need to cover the needs above by making use of the HortonWorks solution, which encapsulates technologies overlapping these required by the energy community, such as Hive+ORC. It will further need to provide relevant data-transformations in order to support the chosen pilot use-cases.

LINKS

A.1 Slides & Presentations

1. BigDataEurope Project Introduction (BDE Coordinator)
2. BDE Energy Societal Challenge (CRES)
3. Data the Gate to a Smart Energy System (EurElectric)
4. BDE_technology_Overview (NCSR Demokritos)
5. Big Data Applications For Siting And Forecasting (VESTAS)
6. Data_Management_In_Resource_Forecasting (University of Athens – Forecasting Unit)
7. Data_Management_in_Wind_Energy (TERNA)
8. System_Monitoring_Case (CRES)
9. SmartGrid data management challenges (CRES, ETP Smart Grids)
10. ETP_Smart_Grids_Utility_Survey (ETP Smart Grids)

A.2 Photos

Link to the Photos slideset on the public BDE Flickr account