An ensemble multi-step-ahead forecasting system for fine particulate matter in urban areas

Abstract

In recent years, growing air pollution has become a significant issue due to its detrimental effects on the environment and different living organisms. Providing accurate and reliable forecasts of air quality over a long future horizon is an effective way to mitigate health risks. In this paper, the problem of urban PM2.5 forecasts for several days ahead is considered. An ensemble multi-step-ahead forecasting system is introduced for this problem, which combines different multi-step-ahead strategies (including single-output and multi-output approaches). The proposed hybrid framework consists of three parts. In the first part, the Ensemble Empirical Mode Decomposition (EEMD) technique is combined with a prediction tool and multi-step-ahead strategies. Boosting idea is considered in the second part of the algorithm. Finally, the stacked ensemble of boosted hybrid structures is developed to provide the final multi-step-ahead forecasts. Least Square Support Vector Regression (LSSVR), and Long Short-Term Memory neural network (LSTM) are employed as the prediction tools in the proposed hybrid framework. Through real PM2.5 data examples from Mashhad, Iran, the proposed ensemble model is investigated for 1-day-ahead to 10-days-ahead. The results reveal the effectiveness of the ensemble model in comparison with the multi-step-ahead strategies in all time-steps. The proposed model with LSSVR prediction tool shows the smallest mean RMSE, MAE, and MAPE values of 7.810, 5.562, and 18.104% over all time-steps, and RMSE improvement rates of more than 35% compared to simply combining different multi-step-ahead strategies with LSSVR approach.

Introduction

One of the most serious environment-related issues is air pollution, attracting worldwide attention more and more. The impacts of air pollution are attributable to at least seven million fatalities in a year (Mannucci and Franchini, 2017). In urbanized areas, PM2.5 is a significant air pollutant which has a diameter of 2.5 μm or less. When these fine particles are inhaled, they can easily penetrate into lungs and bring about harmful health effects (Dockery, 2009) such as respiratory (Guaita et al., 2011) and cardiovascular (Lippmann, 2014) problems. In order to mitigate health risks, it is imperative to have access to accurate and reliable forecasts in advance. If people become aware of severe pollution episodes several days ahead of time, open-air activities and thus air pollutant exposure can be avoided. This is especially crucial for sensitive groups like pregnant women, newborns, children, the elderly, and people with cardiovascular or respiratory diseases (Zhang et al., 2012). From another point of view, providing air pollution forecasts information is beneficial for decision-makers to protect public health through control policies such as traffic limitations, school closure, reminding major industrial units to reduce pollutants discharge, etc. Developing a precise and robust air quality warning system, providing long-term predictions, is an effective way to improve public awareness, supporting management decisions, and therefore avoiding severe pollution episodes or at least reducing health risks.

Understanding the need for warning air quality systems has led to a substantial increase in the number of PM2.5 prediction researches. Among different methods employed for forecasting PM2.5, artificial intelligence-based approaches have attracted great interest and are considered as powerful tools in this field due to their ability for dealing with non-linear features of air quality time series (Li et al., 2019a; Xu et al., 2017a; Zhai and Chen, 2018). Multi-Layer Perceptron (MLP) (de Mattos Neto et al., 2014; Li and Jin, 2018; Ordieres et al., 2005; Perez and Menares, 2018), Elman Neural Network (ENN) (Hao and Tian, 2019; Xu et al., 2017a; Yang and Wang, 2017), Radial Basis Function (RBF) (Ganesh et al., 2018; Zou et al., 2015), general regression neural network (Ausati and Amanollahi, 2016; Zhou et al., 2014), Extreme Learning Machines (ELM) (Li, et al., 2018b; Peng et al., 2017), adaptive neuro-fuzzy inference system (Li, et al., 2018b; Wang et al., 2018), Support Vector Machines (SVM) (Niu et al., 2016; Xu et al., 2017b; Zhu et al., 2018), LSTM (Huang and Kuo, 2018; Li et al., 2017), etc. are some of the examples of artificial intelligence-based methods employed for PM2.5 predictions.

The literature on PM2.5 forecasting has mostly focused on one-step-ahead prediction. One-step-ahead prediction involves estimating the response variable for the time step following the last observation, while multi-step-ahead prediction consists of predicting the next two or more steps. Forecasting PM2.5 for several days ahead is highly crucial and beneficial to provide information for policymakers, emergency planning, and early warnings so that the health risks will be mitigated.

One of the important issues in this field is how to generate multi-step-ahead forecasts for machine learning models (Kline and Zhang, 2004). To this aim, five approaches, have been proposed in the literature, including recursive (Rec), direct (Dir), combination of direct and recursive (DirRec), multi-input multi-output (MIMO) and combination of direct and MIMO (DIRMO) (Taieb et al., 2012). The details of these approaches will be discussed in Section 3.2. In 2003, multi-step-ahead forecasting of monthly data on various US economic time series was considered using Dir and Rec strategies (Kang, 2003). Hamzaçebi et al. compared the performance of Rec and Dir strategies using artificial neural network method (Hamzaçebi et al., 2009). A comparison study of different existing multi-step-ahead strategies can be found in (Taieb et al., 2012). They employed lazy learning as the forecasting technique and a large experimental benchmark to compare various strategies. Multi-step-ahead prediction of crude oil price was addressed by Xiong et al. using a hybrid feed-forward neural network combined with Rec, Dir, and MIMO strategies (Xiong et al., 2013). Bao et al. proposed MIMO strategy for SVR prediction model and showed that MIMO strategy achieves better results in comparison with Rec and Dir strategies (Bao et al., 2014). Lijuan and Guohua suggested using an optimized SVR to predict 12-steps-ahead of monthly inbound tourist flow by Dir strategy (Lijuan and Guohua, 2016). In (Wang et al., 2016), a review of multi-step-ahead wind speed forecasting is presented considering Rec, Dir, DirRec, MIMO, and DIRMO strategies. The results show higher accuracy for DIRMO models. Liu et al. put forward multi-step-ahead wind speed prediction by combining empirical wavelet transform, LSTM, and ENN models. In this study, Rec strategy was considered to generate multi-step forecasts (Liu et al., 2018).

Despite the need for multi-step-ahead forecasting, there have been few studies considering this problem, particularly in the context of PM2.5 predictions. As an example, Biancofiore et al. employed a neural network with recursive architecture for three days ahead forecasting of PM2.5 and PM10 in a case study in Italy (Biancofiore et al., 2017). Li et al. proposed a support vector regression optimized by quantum-behaved Particle Swarm Optimization (PSO) algorithm to predict PM2.5 and NO2 for the next 4 h in Beijing, using Dir and Rec approaches (Li et al., 2018a). In (Kalateh Ahani et al., 2019), PM2.5 forecasting for the next ten days was considered using different multi-step-ahead strategies combined with MLP and ARIMAX models. The results revealed that ARIMAX with Rec strategy using LASSO feature selection achieves the best performance among all compared models. A dynamic evaluation model was proposed in (Li et al., 2019b) to predict PM2.5 for three days in advance. The developed model consisted of improved Complementary EEMD (CEEMD) as a method of noise filtering, improved sine and cosine algorithm for selection of parameters and Least Square SVM (LSSVM) as a forecasting technique. Zhou et al. presented multi-output SVM for predictions of PM2.5 concentrations up to 4 h at five air quality monitoring stations in Taipei city (Zhou et al., 2019b). They showed that multi-output SVM takes underlying non-linear spatial relationships among five stations and provides reliable and accurate forecasting in comparison with developing a single-output SVM for each monitoring station. In another study by Zhou et al. the same problem was tackled by using a deep multi-output LSTM approach (Zhou et al., 2019a). Liu et al. developed a hybrid method based on a Back-Propagation Neural Network (BPNN) optimized by PSO and Adaboost algorithm combined with wavelet packet decomposition method. In this study, three-steps-ahead PM2.5 forecasting of four cities in China was considered using an iterative approach (Liu et al., 2019a). In Table 1, a summary of multi-step-ahead prediction studies is given categorized by application and strategy.

One of the approaches to improve the performance of machine learning models is ensemble learning. In ensemble methods, multiple algorithms are combined to achieve better results. Ensemble learning can be categorized into cooperative and competitive methods (Qiu, 2018). In cooperative methods, the original task is decomposed into several sub-tasks which are easier to be modelled. The results of the sub-tasks are aggregated to produce the final output. Different kinds of decomposition methods like Ensemble Empirical Mode (EMD), wavelet, etc. are included in this category. These approaches have been widely used in PM2.5 prediction area. As an example, Qin et al. considered particulate matter prediction problem in China, using an optimized BPNN in combination with EEMD technique (Qin et al., 2014). Niu et al. established a hybrid model for PM2.5 forecasting based on CEEMD and SVR optimized by grey wolf optimizer (GWO). In this study, the predicted IMFs were integrated using another optimized SVR (Niu et al., 2016). In (Wang et al., 2017a), two-phase decomposition, including wavelet transform and variational mode decomposition, together with BPNN optimized by differential evolution (DE), was used to forecast day-ahead PM2.5 concentrations. A hybrid model based on improved CEEMD with adaptive noise and ELM optimized by imperialist competitive algorithm was addressed by Li and Zhu to model daily pollution contaminants of six major cities in China (Li and Zhu, 2018). Competitive methods, another category of ensemble learning, combines the outputs of different base models which are trained with different datasets, or similar datasets but with different parameter settings. Three major kinds of competitive methods include bagging (Breiman, 1996), boosting (Friedman, 2001; Schapire and Freund, 2013), and stacking (Wolpert, 1992). There have been little attempts at using competitive ensemble learning in PM2.5 forecasting until now. For instance, in (Liu and Li, 2015), the results of ARIMA, neural network, and exponential smoothing method were combined based on entropy weighting method to forecast PM2.5 in Guangzhou, China. An ensemble prediction of PM2.5 was presented in (Wang et al., 2017b). In this study, a hybrid-Garch method was utilized to combine outputs of ARIMA and SVM models. In (Zhai and Chen, 2018), the results of LASSO, AdaBoost, XGBoost, and an optimized MLP were integrated by SVR via stacked generalization to forecast daily PM2.5. Liu et al. proposed a stacked ensemble method using ENN with different architectures as base models and outlier-robustness ELM as meta-learner in order to predict PM2.5 concentrations (Liu et al., 2019b).

As stated before, despite the importance and urgent need for PM2.5 forecasting multiple steps of time into the future, most of them only focus on one-step-ahead prediction. In practical applications, multi-step-ahead forecast of PM2.5 is of greater value, especially when it comes to decision making and planning for preventive actions, placing restrictions on traffic and industry, etc. Predicting multiple steps of time ahead, makes more information available about the future, and thus allowing better and more accurate decision making and developing plans in advance. For example, if air quality data become available to environmental managers a few days in advance, related organizations will be informed sooner, and efficient decisions will be made concerning closing schools, placing restrictions on major industrial units for reducing pollutant discharge, traffic limitations, etc. Most importantly, providing information to the public in advance allows them to plan their daily outdoor activities better. This is especially necessary for pregnant women, elderly people, asthmatics, children, etc. to avoid or at least reduce the need for medication or hospital treatment. This paper aims to comprehensively explore multi-step-ahead prediction of fine particulate matter in Mashhad, Iran. Moreover, enhancing performance of multi-step-ahead forecasting is considered by employing the ideas of ensemble techniques (both cooperative and competitive ensemble learning), which have been ignored in most of the previous studies.

Therefore, given the limitations of previous studies and the current need for predicting PM2.5 multiple days ahead in Mashhad, an ensemble multi-step-ahead framework is proposed in this study. The developed framework is composed of three major parts. In the first part, five multi-step-ahead strategies with EEMD decomposition technique are employed. The second part aims to improve the results of the previous stage by using the idea of boosting procedure. In the last part, a stacked ensemble of different boosted multi-step-ahead strategies is considered in order to take advantage of all types of multi-step-ahead strategies in the final prediction. The proposed framework is implemented using LSSVR and LSTM forecasting tools, and the hyperparameters of each prediction technique are optimized using Bayesian Optimization Algorithm (BOA).

The main contributions of this article are as follows:

• An ensemble multi-step-ahead forecasting system is developed to effectively conduct PM2.5 predictions in Mashhad for 1-day-ahead, 2-days-ahead, …, 10-days-ahead. The proposed method comprises decomposition-ensemble, boosting, and stacking modules, and the effectiveness of each part is explored.

• A novel idea is proposed in the boosting part, in which iteratively fitting of residuals using decomposition-ensemble procedure is considered to improve the results.

• Five widely used multi-step-ahead strategies are employed to generate predictions over the forecasting horizon, while most studies only select one or two strategies without considering the performance of other types of multi-step-ahead strategies.

• In the stacking module, the results of these strategies are combined. According to the obtained results, the stacked ensemble of multi-step-ahead strategies provides better results than each of the techniques alone. This is the first attempt to integrate the results of different multi-step-ahead approaches. This idea suggests that instead of choosing one strategy to provide multi-step-ahead predictions, take advantage of different strategies to enhance forecasting performance in all prediction time-steps. Besides, the performance of different multi-step-ahead techniques may vary depending on the forecasting method, but the proposed ensemble method outperforms different multi-step-ahead strategies regardless of the prediction method used.

The rest of the paper is structured as follows. Section 2 discusses the area of study and sources of data. Relevant methodologies are described in Section 3. In Section 4, the empirical study is conducted, and the results are analyzed. Section 5 provides the conclusion.