Assessing artificial neural networks and statistical methods for infilling missing soil moisture records
Introduction
Moisture in the upper layers of the soil is a vital component of the total water balance in the Earth-atmosphere system, playing a crucial role in several hydrological processes. Soil moisture is one of the main factors influencing the partitioning of rainfall into infiltration and runoff (Mahmood, 1996, Thornthwaite, 1961), controlling the exchange of water and energy between the land surface and the atmosphere (Legates et al., 2010, Berg and Mulroy, 2006, Trenberth and Guillemot, 1998, Houser et al., 1998, Reynolds et al., 2002), and the subsurface water drainage that influences the leaching of contaminants to groundwater (Langevin and Panday, 2012, Legates et al., 2010). The reliability of the above mentioned applications usually depends on the availability of a continuous time series of soil moisture record. Typically, soil moisture data acquired through ground (or in situ) measurements have missing values due to equipment malfunction, logger storage overruns, data retrieval problems, and/or severe weather conditions (Dumedah and Coulibaly, 2011, Coulibaly and Evora, 2007). Consequently, the infilling of missing soil moisture values becomes a necessary procedure to generate a continuous time series record.
Several studies have infilled hydrologic variables including precipitation (Mwale et al., 2012, Nkuna and Odiyo, 2011, Coulibaly and Evora, 2007, French et al., 1992, Luck et al., 2000, Abebe et al., 2000, ASCE Task Committee on Application of Artificial Neural Networks in Hydrology, 2000b), streamflow (Mwale et al., 2012, Ng and Panu, 2010, Ng et al., 2009, Elshorbagy et al., 2000, ASCE Task Committee on Application of Artificial Neural Networks in Hydrology, 2000b), evapotranspiration (Abudu et al., 2010), air temperature (Coulibaly and Evora, 2007, Schneider, 2001), and soil moisture (Gao et al., 2013, Wang et al., 2012, Dumedah and Coulibaly, 2011). The infilling methods employed in the above studies ranged from statistical methods (Gao et al., 2013, Wang et al., 2012, Dumedah and Coulibaly, 2011) to artificial neural networks (Mwale et al., 2012, Nkuna and Odiyo, 2011, Coulibaly and Evora, 2007), with varying levels of accuracy. While several studies have explored different infilling approaches, very few studies have been undertaken to actually reconstruct soil moisture records using both statistical and artificial neural network methods. As a result, this study investigates 5 statistical and 9 artificial neural network methods, a total of 14 methods to estimate missing soil moisture records. The soil moisture monitoring network located in the Yanco region of southeast Australia (Smith et al., 2012) is used as the demonstration data set.
The statistical methods include monthly replacement, weighted Pearson correlation, station relative difference, and a weighted merger of the three statistical methods. Moreover, a method based on the concept of rough sets (Pawlak, 1997, Pawlak et al., 1995, Pawlak, 1982) was used to determine patterns of temporal stability of soil moisture to account for different moisture conditions. The artificial neural networks (ANNs) evaluated in this stud are broadly categorized into feedforward group, dynamic group and radial basis group. Detailed descriptions for the statistical and ANN methods are provided in the methods section. The selected approaches constitute a varied range of methodologies to facilitate a comprehensive inter-comparison between a range of statistical and ANNs with the potential to identify high performing methods to infill missing soil moisture. The infilling methods have been evaluated for their estimation accuracy across 13 soil moisture monitoring stations independently at three different soil layer depths in the Yanco area. Moreover, an evaluation of the soil moisture across the 13 monitoring stations in space and their persistence of relative moisture conditions over several time periods was demonstrated. These space–time distributions are presented for the entire period of the chosen soil moisture data, and also on a month-by-month basis.
Section snippets
Study area and soil moisture data
The Yanco area shown in Fig. 1 is a area, located in the western plains of the Murrumbidgee Catchment in southeast Australia where the topography is flat with very few geological outcroppings. Soil texture types are predominantly sandy loams, scattered clays, red brown earths, transitional red brown earth, sands over clay, and deep sands. According to the Digital Atlas of Australian Soils, the dominant soil is characterized by plains with domes, lunettes, and swampy depressions, and
Methods for infilling missing soil moisture records
Using the raw soil moisture data, a complete data set was retrieved by removing all periods with missing records, such that all soil moisture records were temporally consistent (or common) to all stations for a specific soil layer (Dumedah and Coulibaly, 2011, Coulibaly and Evora, 2007). In other words, the complete data set is spatially complete in a way that each record in the data set at any one station has corresponding records available across the remaining 12 stations for the specified
Results and discussion
The application of the 14 infilling methods is presented in two stages. The first stage illustrates the spatial–temporal variation of soil moisture for the 13 monitoring stations; whereas the second stage presents the evaluation of the infilling methods to estimate the soil moisture records. The infilling methods are evaluated in three phases: surface soil layer, second soil layer, and the third soil layer independently.
Conclusions
This study has evaluated 14 infilling methods including artificial neural network and statistical techniques, to estimate missing soil moisture records at 13 monitoring stations independently for three different soil layers. An evaluation of the estimated soil moisture values against known records showed that the top three highest performing methods are the nonlinear autoregressive neural network, the rough sets method, and the monthly replacement. The high estimation accuracy (RMSE of 0.03
Acknowledgements
This work was supported by funding from the Australian Research Council (DP0879212). The program for the infilling methods used herein is available upon request from the first author at: [email protected].
References (53)
- et al.
Soil moisture temporal stability over experimental areas in central italy
Geoderma
(2009) - et al.
Temporal stability of surface soil moisture in the little washita river watershed and its applications in satellite soil moisture product validation
J. Hydrol.
(2006) - et al.
Temporal persistence and stability of surface soil moisture in a semi – arid watershed
Remote Sens. Environ.
(2008) - et al.
Comparison of neural network methods for infilling missing daily weather records
J. Hydrol.
(2007) - et al.
Rainfall forecasting in space and time using a neural network
J. Hydrol.
(1992) - et al.
Estimation of spatial soil moisture averages in a large gully of the loess plateau of china through statistical and modeling solutions
J. Hydrol.
(2013) - et al.
Mean soil moisture estimation using temporal stability analysis
J. Hydrol.
(2005) - et al.
Infilling of missing rainfall and streamflow data in the Shire River basin, Malawi - A self organizing map approach
Phys. Chem. Earth
(2012) - et al.
Comparisons of traditional and novel stochastic models for the generation of daily precipitation occurrences
J. Hydrol.
(2010) - et al.
Filling of missing rainfall data in Luvuvhu River Catchment using artificial neural networks
Phys. Chem. Earth
(2011)
Rough set approach to knowledge-based decision support
Eur. J. Oper. Res.
Towards a general equation for frequency domain reflectometers
J. Hydrol.
A three-dimensional gap filling method for large geophysical datasets: application to global satellite soil moisture observations
Environ. Model. Softw.
Application of adaptive fuzzy rule – based models for reconstruction of missing precipitation events
Hydrol. Sci. J.
Infilling missing daily evapotranspiration data using neural networks
J. Irrig. Drain. Eng.
Neural Networks: Algorithms and Applications
Artificial neural networks in hydrology I: preliminary concepts
J. Hydrol. Eng.
Artificial neural networks in hydrology II: hydrologic applications
J. Hydrol. Eng.
Neural Network Toolbox: Users Guide
Streamflow predictability given macro-scale estimates of the initial soil moisture status
Hydrol. Sci. J.
Spatial–temporal variability of soil moisture and its estimation across scales
Water Resour. Res.
Multivariable functional interpolation and adaptive networks
Complex Syst.
Nonlinear autoregressive integrated neural network del for short-term load forecasting
IEEE Proc. – Gener., Trans. Distrib
Evaluation of statistical methods for infilling missing values in high-resolution soil moisture data
J. Hydrol.
Minimizing the effects of inaccurate sediment description in borehole data using rough set theory and transition probability
J. Geogr. Syst.
Minimizing effects of scale distortion for spatially grouped census data using rough sets
J. Geogr. Syst.
Cited by (26)
In-use measurement of ultrasonic flowmeter based on Machine learning
2023, Measurement: Journal of the International Measurement ConfederationA critical review of real-time modelling of flood forecasting in urban drainage systems
2022, Journal of HydrologyComparison of gap-filling techniques applied to the CCI soil moisture database in Southern Europe
2021, Remote Sensing of EnvironmentCitation Excerpt :There were hardly any differences between the RMSE and cRMSE due to the low bias; its medians range from 0.036 to 0.026 m3m−3 for SVM with PCA and SVM-Tdh, respectively. These results are in line with those reported by Dumedah et al. (2014). The simplest methods resulted in the lowest accuracy, and the best results were obtained using a nonlinear autoregressive neural network, comparable to SVM-Tdh.
Application of artificial neural network modeling techniques to signal strength computation
2021, HeliyonCitation Excerpt :Neural networks have been very effective for modeling and for characterisation of complex systems for a number of applications [29, 30]. The successful application of ANN to a number of scientific and engineering problems abounds [31, 32, 33, 34, 35], including the prediction of signal strength in the VHF and UHF bands [2, 36, 37, 38, 39, 40, 41]. One of the most widely used type of ANN is the feedforward network.
Using machine learning to produce near surface soil moisture estimates from deeper in situ records at U.S. Climate Reference Network (USCRN) locations: Analysis and applications to AMSR-E satellite validation
2016, Advances in Water ResourcesCitation Excerpt :In addition to isolating locations that are consistently wetter or drier than the spatial average within a test watershed, these analyses can provide indices for temporal patterns of soil moisture to be utilized in subsequent models (Brocca et al., 2010). While the current analysis aims to fill missing soil moisture values in the vertical dimension, other works have deployed various regression techniques (e.g. Dumedah et al., 2011) or artificial neural networks (Dumedah et al., 2014) to fill missing values in the temporal dimension. Meng and Quiring (2008) examined model accuracy for replacing soil moisture at in situ stations.