Elsevier

Computers & Geosciences

Volume 81, August 2015, Pages 12-19
Computers & Geosciences

Virtual integration of sensor observation data

https://doi.org/10.1016/j.cageo.2015.04.006Get rights and content

Highlights

  • Virtual observation data integration vs. prevailing data warehouse approaches.

  • Flexible combination of Mediator/Wrapper architecture with OGC SWE interfaces.

  • In situ and remote static and mobile sensors producing vector and raster data.

  • Multi-thread implementation to leverage current hardware multicore architectures.

  • Under validation in two Spanish public meteorological and oceanographic agencies.

Abstract

This paper discusses the design, implementation and evaluation of a framework that enables the virtual integration of heterogeneous observation data sources through a Sensor Observation Service (SOS) standard interface. Currently available SOS implementations follow a data warehouse design approach for data integration. Contrary to this, the present framework uses a well-known Mediator/Wrapper virtual data integration architecture, enabling the direct access to the current data supplied by the data sources. Currently, the framework is being validated as the OGC compliant technology to publish the meteorological and oceanographic observation data generated by two public agencies of the regional government of Galicia (Northwest of Spain).

Introduction

The success or failure of many environmental applications and tools is greatly determined by the availability and effective management of appropriate observation data. The amount of such data that is currently being produced is huge and the heterogeneity of the processes that generate those data is wide. Therefore important barriers exist caused by the use of proprietary data formats and interfaces that make difficult data accessibility and systems interoperability. According to Horsburgh et al. (2011), the components of any environmental observatory information system must include (among others) data acquisition and communication (Villarroya et al., 2013) and data publication and interoperability. The great importance of the latter component is supported by regulations such as the European Commission directive INSPIRE that aims at the creation of the European Spatial Data Infrastructure. Important technological solutions to be applied in the development of data publication and interoperability components come from the area of web services and service oriented architectures.

In line with this, the Open Geospatial Consortium (OGC) proposes in its Sensor Web Enablement (SWE) initiative various standard web service interfaces and data formats that enable interoperable access to sensor data in environmental data infrastructures. Among the proposed interfaces, the Sensor Observation Service (SOS) (Bröring et al., 2012b) is having an important impact in the development of current environmental information systems. This service enables standardized access to collections of observation data generated by different processes, which are in most cases physical sensors. Observations inside a SOS are organized in Offerings, which resemble layers of Web Map Services (WMS) and feature types of Web Feature Services (WFS).

Providing integrated open access through SOS interfaces to heterogeneous observation data sources is currently a challenge for many organizations. For example, in environmental management agencies, air temperature data may come from both meteorological and oceanographic stations. Similarly, precipitation data may be generated both by meteorological stations and by weather surveillance radar. Various alternatives are possible in order to tackle the above problem.

  • Client-side data integration: Each data source is accessed through a different SOS, specifically adapted to its own characteristics. Data integration clients are complex and demand much expertise from their end-users. Besides, very similar data integration functionality is replicated in many clients. Client-side data integration would be required even if each data source is accessed through a different Offering of a single SOS.

  • Data warehouse: All the data are logically integrated in a central Database Management System (DBMS) with a common data model. Periodically, Extract, Transform and Load (ETL) processes obtain data from each data source to feed the data warehouse. This is a good solution to perform efficient data analysis over historical relational data, since queries to the data warehouse are isolated from the operational transactions of the data sources and ETL may be done in moments of low system activity. On the other hand, it requires additional storage and management infrastructure. Besides, if near real time data access is required then ETL tasks would have to be executed too frequently. Finally, some sensors produce raster like observation data which does not fit well current DBMS technologies.

  • Virtual data integration: The common repository with the common data model is now virtual and data sources are accessed directly in each query, avoiding the need of new data storage infrastructure, minimizing the overall impact in the organization. Each data source maintains its own repository adapted to its type of data (including vector or raster spatial data). On the other hand, accessing data sources in each query to construct the common data model is expensive in terms of performance. However, ETL processes are no longer needed which is specially interesting when near real time data access is required.

  • Service-side complex solutions: Both data warehouse and virtual data integration might be combined into complex decision support architectures that adapt to the specific needs of each organization.

Based on the fact that most of currently available SOS implementation follow a data warehouse approach, this paper describes the design, implementation and evaluation of a framework that performs virtual data integration of heterogeneous observation data sources. The classical Mediator/Wrapper data integration architecture (Wiederhold, 1992) used by the framework together with the use of OGC SWE standard interface specifications makes it very flexible in the incorporation of new data sources. The proposed solution is also flexible in the sense that observation data produced by any kind of process is supported, including in situ and remote static and mobile sensors. The use of multi-thread software architecture enables the framework to leverage the currently available multi-core hardware architectures. Finally, semantic data integration is out of the scope of the currently available version of the framework and left as a problem to tackle in the near future.

The remainder of this paper is organized as follows. Section 2 describes some pieces of work related to the present one. Section 3 describes how global and local concepts are mapped by the virtual data integration approach undertaken in the frameworks mediator. The components of the software architecture are briefly described in Section 4. The evaluation of the framework and its validation in two real scenarios related to meteorological and oceanographic data is shown in Section 5. Finally, Section 6 concludes the paper and discusses some issues of further work.

Section snippets

Related work

The Sensor Web Enablement1 initiative of the Open Geospatial Consortium provides a series standard specifications for the interoperability of sensor data related web services. A Sensor Observation Service (SOS) (Bröring et al., 2012b) provides web access to collections of observations. Each observation has a value (such as 15 °C) of an Observed Property (such as air temperature) for a given time instant. Besides, the observation

Observation data integration model

The data that is published through version 1.0.0 of SOS must be organized into possibly overlapping collections of observations called Offerings, which resemble layers of well known Web Map Services (WMSs). Offerings should be dense, in the sense that the probability of issuing a query with empty result should be minimized. Both the global data published by the present framework and the one offered by each data source is organized in Offerings, since all of them implement the SOS 1.0.0

Mediator/Wrapper frameworks architecture

The architecture of the framework (see Fig. 3) is based on the well-known Mediator/Wrapper paradigm (Wiederhold, 1992). The following general interactions between components take place during the evaluation of each SOS request.

  • 1.

    The SOSDIService receives the request and invokes the SOSDIMediatorCore through the ISOS interface.

  • 2.

    SOSRequestParse is used to parse the request XML.

  • 3.

    ODIMManager is used to access the ODIM and obtains required definitions of relevant global Offerings.

  • 4.

    The request is

Framework validation and evaluation

The framework is being validated by experts of two public agencies of the Spanish region of Galicia (northwest of Spain), namely MeteoGalicia6 and Intecmar.7 MeteoGalicia is a meteorological agency with a wide range of meteorological and oceanographic observation Processes, including the following.

Meteorological stations (Fig. 4(a)). A network of more than 80 automatic stations equipped with a total of 693 different physical sensors. Around 120

Conclusions and further work

The design, implementation and evaluation of a framework was described that provides a real solution to the problem of virtual integration of heterogeneous observation data sources in environmental application domains. The framework is currently being validated in two real scenarios with meteorological and oceanographic data. The source code is licensed under GPL version 3 and available at https://gitlab.citius.usc.es/cograde/sosvdi. Advantages of the approach are the following.

  • Server-side data

Acknowledgments

This work was partially supported by Xunta de Galicia (Ref. 09MDS034522PR) and Ministerio de Ciencia e Innovación, Gobierno de España (Ref. TIN2010-21246-c02-02). The authors also thank MeteoGalicia (Consellería de Medio Ambiente Territorio e Infraestrucuras, Xunta de Galicia) and Intecmar for their support during the validation of the framework with real data sets.

References (39)

  • T. Uslander et al.

    Designing environmental software applications based upon an open sensor service architecture

    Environ. Model. Softw.

    (2010)
  • S. Villarroya et al.

    Geodadisa framework for the development of geographic data acquisition and dissemination servers

    Comput. Geosci.

    (2013)
  • Botts, M., Robin, A., 2014. OGC SenorML: Model and XML Encoding Standard. Open Geospatial Consortium (OGC)....
  • Boucelma, O., Essid, M., Lacroix, Z., 2002. A WFS-based mediation system for GIS interoperability. In: Proceedings of...
  • S. Bowers et al.

    A conceptual modeling framework for expressing observational data semantics

  • J.E. Bresenham

    Algorithm for computer control of a digital plotter

    IBM Syst. J.

    (1965)
  • Bröring, A., Maue, P., Malewski, C., Janowicz, K., 2012a. Semantic mediation on the sensor web. In: IEEE International...
  • Bröring, A., Stasch, C., Echterhoff, J., 2012b. OGC Sensor Observation Service Interface Standard. Open Geospatial...
  • Chawathe, S., Garcia-Molina, H., Hammer, J., Ireland, K., Papakonstantinou, Y., Ullman, J., Widom, J., 1994. The...
  • Cited by (16)

    • Semantic mediation of observation datasets through Sensor Observation Services

      2017, Future Generation Computer Systems
      Citation Excerpt :

      Raster observation data of remote sensors are supported only by [15]. The SOSVDI framework described in [2] supports virtual integration of vector and raster observation data, but semantic conflict resolution is only addressed partially just for Properties. An ontology provides a formal description of the knowledge (concepts and relationships among them) of a specific domain.

    • Designing the Distributed Model Integration Framework – DMIF

      2017, Environmental Modelling and Software
      Citation Excerpt :

      Provide the URL of the WSDL of web services to be linked. Note that the participating services can be models or data sets organized in various ways, for example, sensor observation data (Regueiro et al., 2015). Individual web services can have one or more functions and the user has to select which functions are to be involved in the integration.

    • BUILDING A DIGITAL TWIN OF THE ITALIAN COASTS

      2022, International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives
    View all citing articles on Scopus
    View full text