Evolving Gaussian process models for prediction of ozone concentration in the air
Introduction
Ozone (O3), a form of oxygen, is a highly unstable and poisonous gas that can form and react under the action of light and that is present in two layers of the atmosphere. The ozone is a very specific air substance, which is present in the whole Earth’s atmosphere – from the ground level to the top of the atmosphere. The stratospheric ozone prevents the harmful solar ultraviolet radiation to reach the Earth’s surface. However, in the tropospheric layer, which is at ground level, the ozone is an air pollutant, which damages human health and the ecosystem equilibrium. Exposure to ozone can cause serious health problems in plants and people, thus ozone pollution is a major problem in some regions of the world. It tends to increase during periods of high temperatures and sunny skies. The ozone content changes in the troposphere and the complexity of the processes defining these changes are the reasons why the atmospheric ozone dynamics is an object of intensive research.
The most direct way to obtain accurate air quality information is from measurements made at surface monitoring stations across countries. Fixed measurements of hourly ozone concentrations in compliance with the European Directive on ambient air quality and cleaner air for Europe [1] give continuous information about the evolution of surface ozone pollution at a large number of sites across Europe. In several Member States, they are more and more supplemented by numerical model outputs delivered at a regional or local scale, in keeping with the European Directive. The European standards that guarantee human-health protection are as follows: health protection level, 120 μg/m3 8 h mean concentration; informing the public level, 180 μg/m3 1 h mean concentration; and warning the public level, 240 μg/m3 1 h mean concentration. Therefore, predicting the ozone concentration and informing the population when the air-quality standards are not being met are important tasks.
Ozone concentration has a pronounced daily cycle [22], which can be modelled and forecasted using a variety of methods, and methods that describe the non-linear dynamics from available data are particularly useful. Thus, there exists a number of methods for ozone concentration prediction based on various modelling techniques, e.g. based on neural network NARX models [2], [15], [31], polynomial NARX models [26], fuzzy systems [20], [21], support vector machines [7], ARIMA stochastic models [9], Gaussian processes (GP) [15], [14], [13]. There are also methods which are based on a combination of some of the mentioned techniques, e.g. the approach in [10] combines the use of neural networks, support vector machines and genetic algorithms.
In this paper, we focus on the use of GP modelling techniques for development and comparison of various models for prediction of ozone concentration in the air. The GP model is a probabilistic, non-parametric model based on the principles of Bayesian probability. It differs from most of the other black-box identification approaches in that it does not try to approximate the modelled system by fitting the parameters of the selected basis functions, but rather by searching for relationships among the measured data. The output of the GP model is a normal distribution, expressed in terms of the mean and the variance. The mean value represents the most likely output and the variance can be interpreted as a measure of its confidence. The obtained variance, which depends on the amount and the quality of the available identification data, is important information when it comes to distinguishing the GP models from other computational intelligence methods. Because of their properties GP models are especially suitable for modelling of uncertain processes or when modelling data are unreliable, noisy or missing. GP models fit well for modelling of environmental systems as well as for ozone pollution modelling. Thus, GP models have been developed for prediction of ozone concentration in the air of Bourgas, which is among the regions in Bulgaria with the highest levels of ozone pollution in the air. In [14] first-order GP models based on measurements of the air-pollutant concentrations are identified and verified for one-step-ahead predictions of the ozone concentration in the air of Bourgas. Furthermore, in [13] high-order GP models by using measurements of both the air pollutants and the meteorological parameters are identified and verified. In both cases GP models are trained off-line using only a subset of the available data due to the high computational burden of modelling GP models. However, this limitation and, consequently, the quality of GP models can be improved with on-line updating using the most recent measurements.
A noticeable drawback of system identification with GP models is the computation time necessary for the modelling. Regression based on GP models involves several matrix computations in which the computational complexity increases with the third power of the number of input data, such as matrix inversion and the calculation of the log-determinant of the used covariance matrix. This computational greed restricts the amount of training data, to at most a few thousand cases. To overcome the computational-limitation issues and to also make use of the method for large-scale dataset applications, numerous authors have suggested various sparse approximations [27], [28]. All sparse approximate methods try to retain the bulk of the information contained in the full training dataset, but reduce the size of the covariance matrix to facilitate a less computationally demanding implementation of the GP model. The special kind of sparse approximate method is on-line modelling method Sparse On-line Gaussian Processes (OGP) [8] which tries to incorporate all information of the data by projecting to the reduced covariance matrix.
The OGP method was already implemented for modelling the ozone concentration in the air [25]. As the weather and its characteristics are constantly changing, the model should be updated and adjusted as well. That means it should not only update the model with information contained in streaming data, but should concurrently optimize hyperparameter values as well. As we experienced the OGP method has problems with numerical instability, therefore we propose, by our opinion, more robust method for on-line updating (evolving) of a GP model and compare its performance with an off-line trained GP models. The proposed method is based on the concept described in [24], but it is implemented differently as the method used in the experimental part of the paper.
The paper is structured as follows. In Section 2, the use and properties of Gaussian processes for modelling are reviewed. In Section 3, first- and high-order GP models for prediction of ozone concentration in the air of Bourgas, Bulgaria are identified off-line. A method for prediction of ozone concentration based on an on-line updated (evolving) GP model is proposed and evaluated in Section 4. The concluding remarks end the paper.
Section snippets
Modelling of dynamic systems with Gaussian processes
A GP model is a flexible, probabilistic, non-parametric model with uncertainty predictions. Its uses and properties for modelling are reviewed in [29]. The use of Gaussian processes for modelling dynamic systems is a relatively recent development [6], [12], [18]. A retrospective review can be found in [17].
A Gaussian process is a collection of random variables which have a joint multivariate Gaussian distribution (Fig. 1). Assuming a relationship of the form y = f(x) between input x and output y,
Off-line GP models for prediction of ozone concentration in the air of Bourgas
Measurement data for the year 2008, collected at the automatic measurement station in the centre of Bourgas, Bulgaria, are used. The data includes hourly measurements of the concentrations of ozone, sulphur dioxide, nitrogen dioxide, phenol and benzene. The meteorological parameters have not been measured at this station. However, in order to study how these parameters would influence the prediction of ozone concentration in the air of Bourgas, their measurements at two other stations in the
Evolving GP models
In the previous section, data through all period of interest, in our case year 2008, is available for training and validation.1 Therefore all characteristics of the data from the whole period can be trained. In this section we will test an alternative approach of training GP models that is needed when the training data is not available through the whole period of interest and
Results
To assess the effectiveness of proposed method we divided data in four periods: astronomical winter, astronomical spring, astronomical summer and astronomical fall,2 as data is available only for the year 2008 and it can be shown (Fig. 3) that selected seasons have different characteristics.
From Fig. 3
Conclusions
In this paper, various first- and high-order GP models for prediction of the ozone concentration in the air of Bourgas, Bulgaria are identified off-line and compared. The results show that for the same model type, the high-order models are more accurate than first-order models. The best model is the second-order model, whose input parameters are values of the ozone concentration and the meteorological parameters at two previous hours.
Furthermore, as an alternative approach an on-line updating
Acknowledgements
This work was financed by the National Science Fund of the Ministry of Education, Youth and Science of Republic of Bulgaria, Contract DO02-94/14.12.2008 and the Slovenian Research Agency, Contract BI-BG/09–10-005 (“Application of Gaussian processes to the modelling and control of complex stochastic systems”).
References (31)
- et al.
Combining principal component regression and artificial neural-networks for more accurate predictions of ground-level ozone
Environmental Modelling & Software
(2008) - et al.
Application of Gaussian processes for black-box modelling of biosystems
ISA Transactions
(2007) - et al.
Stochastic model to forecast ground-level ozone concentration at urban and rural areas
Chemosphere
(2005) - et al.
Ozone concentration forecast method based on genetic algorithm optimized back propagation neural networks and support vector machine data classification
Atmospheric Environment
(2011) Growing cell structures – a self-organizing network for unsupervised and supervised learning
Neural Networks
(1994)- et al.
Explicit stochastic predictive control of combustion plants based on Gaussian process models
Automatica
(2008) - et al.
Gas–liquid separator modelling and simulation with Gaussian-process models
Simulation Modelling Practice and Theory
(2008) - et al.
Fuzzy system models combined with nonlinear regression for daily ground-level ozone predictions
Atmospheric Environment
(2007) - et al.
Forecasting peak air pollution levels using narx models
Engineering Applications of Artificial Intelligence
(2009) - D. 2008/50/EC, Directive 2008/50/EC of the european parliament and of the council of 21 may 2008 on ambient air quality...
Evolving Intelligent Systems: Methodology and Applications
Computer Controlled Systems: Theory and Design
Prediction of daily maximum ground ozone concentration using support vector machine
Environmental Monitoring and Assessment
Sparse online gaussian processes
Neural Computation
Cited by (58)
A simplified potential source density function based on predefined discretization
2024, Journal of Engineering Research (Kuwait)A novel soft sensor based warning system for hazardous ground-level ozone using advanced damped least squares neural network
2020, Ecotoxicology and Environmental SafetyHandling Big Datasets in Gaussian Processes for Statistical Wind Vector Prediction
2019, IFAC-PapersOnLineA novel approach to forecast urban surface-level ozone considering heterogeneous locations and limited information
2018, Environmental Modelling and SoftwareA space mapping method based on Gaussian process model for variable fidelity metamodeling
2018, Simulation Modelling Practice and TheoryCitation Excerpt :Gaussian process (GP) model has been wildly used to formulate a Bayesian framework for emulating various forms of functions, especially for expensive simulations or physical experiments [1–3].