Skip to main content

Advertisement

Springer Nature Link
Log in

The Use of Attention-Enhanced CNN-LSTM Models for Multi-Indicator and Time-Series Predictions of Surface Water Quality

  • Published:
Water Resources Management Aims and scope Submit manuscript

Abstract

Deep learning (DL) has recently been applied to surface water quality prediction, whereas its online monitoring data consists of multiple indicators and time series, which are challenging for prediction models due to complex temporal dependencies and inter-indicator mechanisms. Convolutional neural network (CNN) and long short term memory (LSTM) can be used for indicator and temporal domains respectively, but still lack the ability to represent complex patterns in surface water quality. Since attention mechanisms are designed to effectively focus on the most crucial information, spatial attention mechanism (SAM) and temporal attention mechanism (TAM) are suitable for dealing with the above multi-indicator and time series issues. This work incorporates SAM and TAM into the CNN-LSTM model to form 4 DL models for water quality prediction including CNN-LSTM, SAM-enhanced CNN-LSTM, TAM-enhanced CNN-LSTM, and the CNN-LSTM enhanced by both attention mechanisms. Four surface water online monitoring sites are used as case studies to examine the models in predicting three water quality indicators including dissolved oxygen (DO), ammonia nitrogen (NH3-N), and total organic carbon (TOC). According to the case results of the 4 models after training with similar training epochs, the prediction accuracies of attention-enhanced models are better than the CNN-LSTM model, and the model with both attention mechanisms generally achieves the best performance among the 4 models. The prediction NSE of DO by the four models are 0.817, 0.948, 0.952, and 0.967 respectively in a representative case Jiujiang. The results demonstrate that spatial and temporal attention can analyze correlations from multiple indicators and time series of water quality data respectively, to improve the accuracy of surface water quality prediction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

Data Availability

The data that support the findings of this study are available on request.

Abbreviations

DL:

Deep Learning

CNN:

Convolution Neural Network

RNN:

Recurrent Neural Network

LSTM:

Long Short Term Memory

AM:

Attention Mechanism

SAM:

Spatial Attention Mechanism

TAM:

Temporal Attention Mechanism

DO:

Dissolved Oxygen

NH3-N:

Ammonia Nitrogen

TOC:

Total Organic Carbon

SA:

Spatial Attention module

SE:

Squeeze-and-Excitation Networks

CBAM:

Convolutional Block Attention Module

CLM:

Convolution neural network – Long short term Memory

CLSE:

CNN-SE-LSTM

CLSA:

CNN-SA-LSTM

CLCB:

CNN-CBAM-LSTM

MSE :

Mean Square Error

NSE :

Nash-Sutcliffe Efficiency

References

  • Alehu BA, Bitana SG (2023) Assessment of Climate Change Impact on Water Balance of Lake Hawassa Catchment. Environ Processes 10(1):14

    Article  Google Scholar 

  • Antanasijević D, Pocajt V, Povrenović D, Perić-Grujić A, Ristić M (2013) Modelling of dissolved oxygen content using artificial neural networks: Danube River, North Serbia, case study. Environ Sci Pollut Res 20(12):9006–9013

    Article  Google Scholar 

  • Asgari G, Abdipour H, Shadjou AM (2023) A review of novel methods for Diuron removal from aqueous environments. Heliyon 9(12), e23134

  • Bai J, Zhu J, Song Y, Zhao L, Hou Z, Du R, Li H (2021) A3T-GCN: attention temporal graph Convolutional Network for Traffic forecasting. ISPRS Int J Geo-Information 10(7):485

    Article  Google Scholar 

  • Barzegar R, Aalami MT, Adamowski J (2020) Short-term water quality variable prediction using a hybrid CNN–LSTM deep learning model. Stoch Env Res Risk Assess 34(2):415–433

    Article  Google Scholar 

  • Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Networks 5(2):157–166

    Article  CAS  Google Scholar 

  • Chen K, Chen H, Zhou C, Huang Y, Qi X, Shen R, Liu F, Zuo M, Zou X, Wang J, Zhang Y, Chen D, Chen X, Deng Y, Ren H (2020a) Comparative analysis of surface water quality prediction performance and identification of key water parameters using different machine learning models based on big data. Water Res 171:115454

    Article  CAS  Google Scholar 

  • Chen X, Wang Y, Cai Z, Zhang M, Ye C (2020b) Response of the nitrogen load and its driving forces in estuarine water to dam construction in Taihu Lake, China. Environ Sci Pollut Res 27(25):31458–31467

    Article  CAS  Google Scholar 

  • Chen Y, Song L, Liu Y, Yang L, Li D (2020c) A review of the Artificial Neural Network Models for Water Quality Prediction. Appl Sci 10(17):5776

    Article  CAS  Google Scholar 

  • Chen J, Wang H, Yin W, Wang Y, Lv J, Wang A (2024) Deciphering carbon emissions in urban sewer networks: bridging urban sewer networks with city-wide environmental dynamics. Water Res 256:121576

    Article  CAS  Google Scholar 

  • Cleeremans A, Servan-Schreiber D, McClelland JL (1989) Finite State Automata and simple recurrent networks. Neural Comput 1(3):372–381

    Article  Google Scholar 

  • Fuladipanah M, Azamathulla HM, Kisi O, Kouhdaragh M, Mandala V (2024) Quantitative forecasting of bed sediment load in river engineering: an investigation into machine learning methodologies for complex phenomena. Water Supply 24(2):585–600

    Article  Google Scholar 

  • Goodarzi S, Torabideh M, Parsaseresht G, Abdipour H, Kamani H, Zomorrodi Jangaee T (2024) Penicillin removal from the aqueous environment based on AOPs/challenges and outlook. A review. Appl Water Sci 14(7):164

    Article  Google Scholar 

  • Han Y, Bu H (2023) The impact of climate change on the water quality of Baiyangdian Lake (China) in the past 30 years (1991–2020). Sci Total Environ 870:161957

    Article  CAS  Google Scholar 

  • Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  CAS  Google Scholar 

  • Hu J, Shen L, Sun G (2018) Squeeze-and-Excitation Networks, pp. 7132–7141

  • Jaderberg M, Simonyan K, Zisserman A, Kavukcuoglu K (2015) Spatial transformer networks. MIT Press, Montreal, Canada, pp 2017–2025

    Google Scholar 

  • Khorram S, Jehbez N (2023) A hybrid CNN-LSTM Approach for Monthly Reservoir inflow forecasting. Water Resour Manage 37(10):4097–4121

    Article  Google Scholar 

  • Lagogiannis S, Papadopoulos A, Dimitriou E (2024) Development of an Automatic Water Monitoring Network by using Multi-criteria Analysis and a GIS-Based fuzzy process. Environ Processes 11(3):36

    Article  Google Scholar 

  • Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  • Li R, Zhu G, Lu S, Sang L, Meng G, Chen L, Jiao Y, Wang Q (2023) Effects of urbanization on the water cycle in the Shiyang River basin: based on a stable isotope method. Hydrol Earth Syst Sci 27(24):4437–4452

    Article  CAS  Google Scholar 

  • Li J, Zhao Y, Chen D, Zhao P, Zhang C, Wang Y (2024) The quantitative role of moisture and Vertical Motion in Shaping Summer Heavy Rainfall over North China under two distinct large-Scale Weather patterns. J Clim 37(8):2655–2672

    Article  Google Scholar 

  • Liao Z, Zhang M, Chen Y, Zhang Z, Wang H (2024) A Prediction - Detection - Judgment framework for sudden water contamination event detection with online monitoring. J Environ Manage 355:120496

    Article  Google Scholar 

  • Liu Y, Zhang Q, Song L, Chen Y (2019) Attention-based recurrent neural networks for accurate short-term and long-term dissolved oxygen prediction. Comput Electron Agric 165:104964

    Article  Google Scholar 

  • Liu Y, Liu P, Wang X, Zhang X, Qin Z (2021) A study on water quality prediction by a hybrid dual channel CNN-LSTM model with attention mechanism, SPIE

  • Livieris IE, Pintelas E, Pintelas P (2020) A CNN–LSTM model for gold price time-series forecasting. Neural Comput Appl 32(23):17351–17360

    Article  Google Scholar 

  • Makubura R, Meddage DPP, Azamathulla HM, Pandey M, Rathnayake U (2022) A simplified Mathematical Formulation for Water Quality Index (WQI): a Case Study in the Kelani River Basin, Sri Lanka. Fluids 7(5):147

    Article  Google Scholar 

  • Mampitiya L, Rathnayake N, Leon LP, Mandala V, Azamathulla HM, Shelton S, Hoshino Y, Rathnayake U (2023) Machine learning techniques to predict the Air Quality Using Meteorological Data in two urban areas in Sri Lanka. Environments 10(8):141

    Article  Google Scholar 

  • Mei P, Li M, Zhang Q, Li G, song L (2022) Prediction model of drinking water source quality with potential industrial-agricultural pollution based on CNN-GRU-Attention. J Hydrol 610:127934

    Article  CAS  Google Scholar 

  • Mnih V, Heess NMO, Graves A, Kavukcuoglu K (2014) Recurrent models of visual attention. ArXiv abs/1406.6247.

  • Niazkar M, Zakwan M, Goodarzi MR, Hazi MA (2024) Editorial: Assessment of Climate Change Impact on Water resources using machine learning algorithms. J Water Clim Change 15(6):iii–vi

    Article  Google Scholar 

  • Niu Z, Zhong G, Yu H (2021) A review on the attention mechanism of deep learning. Neurocomputing 452:48–62

    Article  Google Scholar 

  • Pang J, Luo W, Yao Z, Chen J, Dong C, Lin K (2024) Water Quality Prediction in Urban Waterways based on Wavelet Packet Denoising and LSTM. Water Resour Manage 38(7):2399–2420

    Article  Google Scholar 

  • Pelletier GJ, Chapra SC, Tao H (2006) QUAL2Kw – a framework for modeling water quality in streams and rivers using a genetic algorithm for calibration. Environ Model Softw 21(3):419–425

    Article  Google Scholar 

  • Qiao J, Hu Z, Li W (2016) Soft measurement modeling based on Chaos Theory for biochemical oxygen demand (BOD). Water 8(12):581

    Article  CAS  Google Scholar 

  • Seibold VC, Stepper MY, Rolke B (2020) Temporal attention boosts perceptual effects of spatial attention and feature-based attention. Brain Cogn 142:105570

    Article  Google Scholar 

  • Shih S-Y, Sun F-K, Lee H-y (2019) Temporal pattern attention for multivariate time series forecasting. Mach Learn 108(8):1421–1441

    Article  Google Scholar 

  • Singh KP, Basant A, Malik A, Jain G (2009) Artificial neural network modeling of the river water quality—A case study. Ecol Model 220(6):888–895

    Article  CAS  Google Scholar 

  • Stollenga MF, Masci J, Gomez FJ, Schmidhuber J (2014) Deep Networks with Internal Selective Attention through Feedback Connections

  • Talukdar S, Shahfahad, Ahmed S, Naikoo MW, Rahman A, Mallik S, Ningthoujam S, Bera S, Ramana GV (2023) Predicting lake water quality index with sensitivity-uncertainty analysis using deep learning algorithms. J Clean Prod 406:136885

    Article  CAS  Google Scholar 

  • Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) attention is all you need, pp. 6000–6010, Curran Associates Inc., Long Beach, California, USA

  • Vatanchi SM, Etemadfard H, Maghrebi MF, Shad R (2023) A comparative study on forecasting of long-term Daily Streamflow using ANN, ANFIS, BiLSTM and CNN-GRU-LSTM. Water Resour Manage 37(12):4769–4785

    Article  Google Scholar 

  • Vijayakumar CR, Balasubramani DP, Azamathulla HM (2021) Assessment of groundwater quality and human health risk associated with chromium exposure in the industrial area of Ranipet, Tamil Nadu, India. J Water Sanitation Hygiene Dev 12(1):58–67

    Article  Google Scholar 

  • Wang X, Tian W, Liao Z (2022) Framework for Hyperparameter Impact Analysis and Selection for Water Resources Feedforward Neural Network. Water Resour Manage 36(11):4201–4217

    Article  Google Scholar 

  • Woo S, Park J, Lee J-Y, Kweon I (2018) pp. 3–19

  • Yan L, Chen C, Hang T, Hu Y (2021) A stream prediction model based on attention-LSTM. Earth Sci Inf 14(2):723–733

    Article  Google Scholar 

  • Yang Y, Xiong Q, Wu C, Zou Q, Yu Y, Yi H, Gao M (2021) A study on water quality prediction by a hybrid CNN-LSTM model with attention mechanism. Environ Sci Pollut Res 28(39):55129–55139

    Article  CAS  Google Scholar 

  • Yin L, Wang L, Li T, Lu S, Tian J, Yin Z, Li X, Zheng W (2023) U-Net-LSTM: Time Series-enhanced Lake Boundary Prediction Model. Land 12(10):1859

    Article  Google Scholar 

  • Zhang Q, You X-y (2024) Recent advances in Surface Water Quality Prediction using Artificial Intelligence models. Water Resour Manage 38(1):235–250

    Article  CAS  Google Scholar 

  • Zhang Z, Li M, Lin X, Wang Y, He F (2019) Multistep speed prediction on traffic networks: a deep learning approach considering spatio-temporal dependencies. Transp Res Part C: Emerg Technol 105:297–322

    Article  Google Scholar 

  • Zhang L, Jiang Z, He S, Duan J, Wang P, Zhou T (2022) Study on Water Quality Prediction of Urban Reservoir by coupled CEEMDAN decomposition and LSTM neural network model. Water Resour Manage 36(10):3715–3735

    Article  Google Scholar 

  • Zhang S, Liu Z, Chen Y, Jin Y, Bai G (2023) Selective kernel convolution deep residual network based on channel-spatial attention mechanism and feature fusion for mechanical fault diagnosis. ISA Trans 133:369–383

    Article  Google Scholar 

  • Zheng H, Liu Y, Wan W, Zhao J, Xie G (2023) Large-scale prediction of stream water quality using an interpretable deep learning approach. J Environ Manage 331:117309

    Article  CAS  Google Scholar 

  • Zhou G, Su S, Xu J, Tian Z, Cao Q (2023) Bathymetry Retrieval from Spaceborne Multispectral Subsurface Reflectance. IEEE J Sel Top Appl Earth Observations Remote Sens 16:2547–2558

    Article  Google Scholar 

Download references

Funding

This research was supported by the National Natural Science Foundation of China (Grant No. 52170102).

Author information

Authors and Affiliations

  1. Key Laboratory of Yangtze River Water Environment, Ministry of Education, Tongji University, Shanghai, 200092, People’s Republic of China

    Minhao Zhang, Zhiyu Zhang & Zhenliang Liao

  2. College of Environmental Science and Engineering, Tongji University, Shanghai, 200092, People’s Republic of China

    Minhao Zhang, Zhiyu Zhang & Zhenliang Liao

  3. School of Environmental and Municipal Engineering, Xi’an University of Architecture and Technology, Xi’an, 710055, People’s Republic of China

    Xuan Wang

  4. College of Civil Engineering and Architecture, Xinjiang University, Urumqi, 830046, People’s Republic of China

    Zhenliang Liao

  5. College of Ecology, Lishui University, Lishui, Zhejiang, 323000, People’s Republic of China

    Lijin Wang

Authors
  1. Minhao Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Zhiyu Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Xuan Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Zhenliang Liao
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Lijin Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Conceptualization: Minhao Zhang; Methodology: Minhao Zhang, Zhiyu Zhang; Formal analysis and investigation: Minhao Zhang, Zhiyu Zhang; Writing - original draft preparation: Minhao Zhang; Writing - review and editing: Minhao Zhang, Zhiyu Zhang, Zhenliang Liao; Funding acquisition: Zhenliang Liao; Resources: Xuan Wang, Lijin Wang; Supervision: Zhenliang Liao.

Corresponding authors

Correspondence to Zhenliang Liao or Lijin Wang.

Ethics declarations

Ethical Approval

There are no relevant waivers or approvals.

Consent to Participate

The authors declare that they are aware of and consent to their participation in this paper.

Consent to Publish

The authors declare that they consent to the publication of this paper.

Competing Interests

No potential conflict of interest was reported by the authors(s).

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, M., Zhang, Z., Wang, X. et al. The Use of Attention-Enhanced CNN-LSTM Models for Multi-Indicator and Time-Series Predictions of Surface Water Quality. Water Resour Manage 38, 6103–6119 (2024). https://doi.org/10.1007/s11269-024-03946-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11269-024-03946-1

Keywords

Profiles

  1. Zhiyu Zhang View author profile