Short-term solar radiation forecasting using hybrid deep residual learning and gated LSTM recurrent network with differential covariance matrix adaptation evolution strategy

Highlights

• A new hybrid deep residual learning with gated LSTM is proposed for solar radiation.

• Cooperative architecture of GRU and LSTM enhanced Xception and ResNet performance.

• Adaptive evolutionary multivariate empirical mode decomposition method is introduced.

• Effective differential covariance matrix adaptation strategy proposed for tuning model.

• The proposed hybrid model outperformed 13 hybrid and popular prediction models.

Abstract

Developing an accurate and robust prediction of long-term average global solar irradiation plays a crucial role in industries such as renewable energy, agribusiness, and hydrology. However, forecasting solar radiation with a high level of precision is historically challenging due to the nature of this source of energy. Challenges may be due to the location constraints, stochastic atmospheric parameters, and discrete sequential data. This paper reports on a new hybrid deep residual learning and gated long short-term memory recurrent network boosted by a differential covariance matrix adaptation evolution strategy (ADCMA) to forecast solar radiation one hour-ahead. The efficiency of the proposed hybrid model was enriched using an adaptive multivariate empirical mode decomposition (MEMD) algorithm and 1+1EA-Nelder–Mead simplex search algorithm. To compare the performance of the hybrid model to previous models, a comprehensive comparative deep learning framework was developed consisting of five modern machine learning algorithms, three stacked recurrent neural networks, 13 hybrid convolutional (CNN) recurrent deep learning models, and five evolutionary CNN recurrent models. The developed forecasting model was trained and validated using real meteorological and Shortwave Radiation (SRAD1) data from an installed offshore buoy station located in Lake Michigan, Chicago, United States, supported by the National Data Buoy Centre (NDBC). As a part of pre-processing, we applied an autoencoder to detect the outliers in improving the accuracy of solar radiation prediction. The experimental results demonstrate that, firstly, the hybrid deep residual learning model performed best compared with other machine learning and hybrid deep learning methods. Secondly, a cooperative architecture of gated recurrent units (GRU) and long short-term memory (LSTM) recurrent models can enhance the performance of Xception and ResNet. Finally, using an effective evolutionary hyper-parameters tuner (ADCMA) reinforces the prediction accuracy of solar radiation.

Introduction

Solar power is one of the most abundant, accessible, and infinitely renewable energy sources yielded when energy from sunlight is transformed into electricity. Solar power is considered a top alternative source to fossil fuels with a high potential to meet global energy demands in the near future [1]. In solar energy technologies, the development of an accurate prediction short-term or long-term (day ahead) [2] solar radiation model plays a fundamental role in enhancing the scheduling and controlling the performance of photovoltaic power plants; having a reliable and robust plan for managing connection to smart grids [3]; and improving the gain margin of the energy suppliers in these markets.

However, predicting solar energy is challenging due to solar radiation’s intermittent and chaotic nature and atmospheric situations that are naturally ungovernable (i.e., clouds, shadows, the vapour of water, ice, air pollution or aerosols in the atmosphere) [4]. Another primary motivation for predicting solar radiation is that installing and maintaining solar radiation measurement devices is highly costly. This makes installing such instruments in every meteorological station financially challenging, especially in developing countries. As an example, there were around 1800 meteorological stations in Turkey in 2020; however, just 7% of them were equipped to register solar radiation data [5]. From this perspective, various technical models have been proposed to forecast solar radiation. The empirical model is one of the popular prediction models established on mathematical procedures. Its benefits include fast and straightforward calculations and is helpful techniques for predicting long-term (monthly or weekly) solar radiation data [6]. However, empirical models cannot accurately predict short-term solar radiation data due to changeable parameters in weather conditions such as cloud cover, rainy days, etc. Furthermore, extracting the intricate and nonlinear associations found in the dependent and independent variables is challenging for empirical models, particularly in humid subtropical climate areas when the weather is rainy with heavy clouds cover [7].

Previous solar energy research studies have proposed considerable number of data-driven techniques for short-term and long-term forecasting. These techniques fall into three main areas, physical techniques, statistical analysis, and machine-deep learning methods. In the physical forecasting models, the atmosphere’s dynamic motion and physical conditions are characterised using a set of mathematical formulas. The performance of physical models relies heavily on the quality and quantity of meteorological variables and astronomical dates (e.g., solar time and earth declination angle) [8]. The statistical approaches using statistical analysis of the various intake features for solar radiation prediction have been applied, including the auto-regressive models (AR), auto-regressive integrated moving average (ARIMA) [9], exponential smoothing, Markov Chain model [10] and Gaussian process [11]. Most of them show acceptable accuracy for predicting the ground solar radiation and cloud motion on different time horizons up to hours ahead. In the last decades, the application of artificial intelligence (AI)-based approaches has considerably developed in solar engineering fields [12]. Previous analyses represent that the AI-based approaches are able to provide more accurate forecasting of solar radiation results than those of the other models [13] such as supervised and unsupervised artificial neural networks (ANN) [14], deep learning models [15], support vector machines (SVM) [16], etc.

A comparative study [17] was done to clarify which one of the six machine learning models can perform best, including the gradient boosting tree (GBT), multi-layer perceptron (MLP), standard ANFIS, subtractive and fuzzy c-means clustering ANFIS, classification and regression tree (CART), and multivariate adaptive regression spline (MARS) to forecast solar irradiation in two sites. The Ref. [17] recommended applying the GBT model as a robust and reliable tool for predicting solar radiation.

A popular sequential deep learning model called long short-term memory (LSTM) is one of the most successful tools in handling the dependency between successive time series data with short-term intervals. One considerable early study was done by Qing and Niu [18] investigated hourly solar radiation forecasting using LSTM. The proposed LSTM [18] was $\text{\%}18$ more precise than BPNN in RMSE.

In short-term solar radiation forecasting, recurrent neural networks (RNN) promise high accuracy and robustness in relation to, long short-term memory (LSTM), bidirectional LSTM, and Gated Recurrent Units (GRU). However, initialising the hyper-parameters of RNNs is challenging due to the complex and non-linear relationships between the setting parameters and the topology and nature of the time-series data. To address these problems, Peng et al. [19] developed a hybrid deep learning model combination of BiLSTM, an adaptive ensemble decomposition method (CEEMDAN), and a sine cosine meta-heuristic algorithm (SCA) for predicting hourly stochastic historical time series solar radiation data. The comparative modelling results suggested that the proposed hybrid model [19] could conquer seven other machine learning models.

Nevertheless, increasing the time horizon for forecasting solar radiation is challenging [20] for AI-based methods because of decreased auto-correlation among the time series samples. One preliminary study in long-term global solar radiation by Jiang [21] applied traditional neural networks (feed-forward back-propagation) and compared them with different empirical regression methods. The findings [21] confirmed the superiority and high ability in generalising ANN models demonstrate in solar radiation forecasting. Multilayer perception (MLP) is one of the most popular and classic machine-learning techniques and has been applied in several studies in forecasting solar radiance [22]. In early work [23], Rodriguez et al. applied a combination of five Multilayer perceptron feed-forward neural networks (an ensemble model) developed by a Monte Carlo simulation to forecast global solar radiation, and the overall validation error and accuracy were considerable. However, the drawbacks of fully connected networks (e.g. network overfitting) were not considered in [23]. To deal with the long-term solar forecasting challenges, Kisi [24] analysed and compared the three AI-based methods: fuzzy genetic, ANN, and neuro-fuzzy models for estimating monthly solar radiations from the Mediterranean areas. The modelling results indicated that the fuzzy genetic model could perform better than the other two models. In another study [25] examining monthly solar radiation prediction, a traditional ANN and adaptive neuro-fuzzy inference system (ANFIS) were applied. The prediction results illustrated that ANFIS mostly outperformed other ANNs. However, the study did not discuss the importance of hyper-parameters initialisation [25].

One initial effort to apply deep learning models such as convolutional neural networks (CNN), was proposed by Kaba et al. [26] in order to estimate daily global solar radiation. Although the technical details of the CNN model are not clear, the estimation results show that the CNN model can be an appropriate alternative approach in long-term solar radiation forecasting.

To tackle the issues raised by traditional neural networks training, such as the exhaustive learning process, insufficient parameter preference, and the need for a large number of training samples, a new model of neural network called a Deep Belief Networks (DBN) [27] was proposed in 2007. A combined DBN [28] with a clustering idea was used to develop an accurate daily solar energy forecasting model based on 30 sites located in China. The DBN method [28] acquired more reasonable accuracy from the outcomes than empirical ML methods. Tuning the hyper-parameters of deep learning models is essential; nevertheless, it is frequently challenging. Meta-heuristic algorithms have been applied in order to optimise the hyper-parameters that lead to improving the average performance of the models. Wang et al. [29] proposed a primary hybrid solar radiation forecasting models consisting of an Extreme Learning Machine (ELM) and Cuckoo Search (CS). In order to reduce the computational runtime, the Ref. [29] applied a combination of Multiresponse Sparse Regression (MSR) and leave-one-out cross-validation (LOO-CV) to determine the priority of neurons and remove the lowest ones in Feed Forward Neural Networks. The CS played the role of weight coefficients optimiser.

Despite comprehensive studies in the last years, at least three research gaps remain in designing techniques/models for short- and long-term solar radiation prediction as follows:

• One of the most significant factors in improving the performance of prediction models hyper-parameters tuning.

• In most case studies, optimising the architecture of deep learning models using various recurrent neural networks did not consider substantially.

• The low performance of solar radiation predictors is due to an insufficient decomposition setting .

This study proposes a novel hybrid residual deep learning model for forecasting solar radiation one hour ahead based on real meteorological and Shortwave Radiation (SRAD1) data from an installed offshore buoy station located in Lake Michigan, Chicago, United States and supported by the National Data Buoy Centre (NDBC). In order to clean the data and improve accuracy, an outlier detection method (autoencoder) was applied. A hyper-parameter optimiser is also proposed to reinforce the model’s performance. The foremost contributions of this study are summed as follows:

• A novel hybrid solar radiation forecasting model is proposed composed of recurrent neural networks (GRU, LSTM and BiLSTM), and a convolutional ResNet50 model (deep residual learning) with adaptive decomposition technique and effective auto-tuner (ADCMA-ResNet50-GRU-2LSTM).

• An adaptive multivariate empirical mode decomposition (MEMD) algorithm is proposed to decompose solar radiation time-series data with a high level of nonlinearity and non-stationarity into intrinsic mode functions (IMFs) with minimum entropy using an evolutionary Nelder–Mead simplex search algorithm.

• In order to deal with the shortcomings of the hyper-parameters tuning initialisation, an effective and smart hyper-parameters tuner, adaptive differential covariance matrix evolutionary algorithm (ADCMA), was developed to improve prediction accuracy and reduce modelling bias.

• A comprehensive comparative framework was also designed to evaluate the performance of various canonical and hybrid machine learning and deep learning models with regard to developing an accurate and reliable solar radiation forecasting model.

The principle sections of this article are organised as follows. The technical details of the involved methods are exemplified in the next Section 2. The following presents the case study’s attributes and their statistical analysis of the dataset used in this study in Section 3. In order to develop a systematic comparison framework for the short-term solar radiation forecast, various models are evaluated and compared with the proposed model in Section 4. Eventually, in Section 5, the acquired results of this investigation and future research plans are outlined. All the acronyms and symbols utilised in this study can be seen in Table 1, Table 2.