Skip to main content
SpringerLink
Log in

Nonlinear MIMO System Identification with Echo-State Networks

  • Published:
Journal of Control, Automation and Electrical Systems Aims and scope Submit manuscript

Abstract

This paper presents a formalization of steps for identification of echo-state network (ESN)-based models for multiple-input and multiple-output (MIMO) nonlinear dynamic systems models, employing the best practices found in the literature. The ESN is a recurrent neural network capable to model nonlinear dynamics and it can be trained using computationally efficient algorithms. A simplified ESN architecture from the literature for process identification tasks is used and a procedure to create the model and tune the main hyperparameters is proposed. The proposed method is evaluated using both a simulated pH neutralization process and a real refrigeration compressor test rig. For each case study, the procedure to obtain the data series, the model creation, and the influence of each hyperparameter in the model performance are discussed. In both cases, the proposed model architecture presented better results than the linear model and the two nonlinear models used as baseline, namely an extreme learning machine-based Hammerstein model and a long short-term memory network model. The results show that the proposed ESN-based system identification method can be used to obtain MIMO models for nonlinear processes using a computationally efficient training algorithm. For the ESN architecture considered in the case studies, the training time is about 80 s, while a prediction can be obtained in about 15 ms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Antonelo, E. A., Camponogara, E., & Foss, B. (2017). Echo state networks for data-driven downhole pressure estimation in gas-lift oil wells. Neural Networks, 85, 106–117.

  • Ayoubi, M. (1994). Nonlinear dynamic systems identification with dynamic neural networks for fault diagnosis in technical processes. Proceedings of IEEE International Conference on Systems, Man and Cybernetics, 3, 2120–2125.

  • Bessa, R., & Barreto, G. A. (2019). Robust echo state network for recursive system identification. In I. Rojas, G. Joya, & A. Catala (Eds.), Advances in computational intelligence (pp. 247–258). Springer.

  • Björk, E., & Palm, B. (2006). Performance of a domestic refrigerator under influence of varied expansion device capacity, refrigerant charge and ambient temperature. International Journal of Refrigeration, 29(5), 789–798. https://doi.org/10.1016/j.ijrefrig.2005.11.008

    Article  Google Scholar 

  • Boussaada, Z., Curea, O., Remaci, A., Camblong, H., & Mrabet Bellaaj, N. (2018). A nonlinear autoregressive exogenous (NARX) neural network model for the prediction of the daily direct solar radiation. Energies, 11(3), 620.

    Article  Google Scholar 

  • Chen, Y., He, Z., Shang, Z., Li, C., Li, L., & Xu, M. (2019). A novel combined model based on echo state network for multi-step ahead wind speed forecasting: A case study of NREL. Energy conversion and management, 179, 13–29.

    Article  Google Scholar 

  • Fernandez, B., Parlos, A., & Tsai, W. (1990). Nonlinear dynamic system identification using artificial neural networks (ANNs). In 1990 IJCNN international joint conference on neural networks, (IEEE).

  • Ganguli, S., Huh, D., & Sompolinsky, H. (2008). Memory traces in dynamical systems. Proceedings of the National Academy of Sciences, 105(48), 18970–18975. https://doi.org/10.1073/pnas.0804451105

    Article  Google Scholar 

  • Gomez, J., Jutan, A., & Baeyens, E. (2004). Wiener model identification and predictive control of a pH neutralisation process. IEE Proceedings-Control Theory and Applications, 151(3), 329–338.

    Article  Google Scholar 

  • Guo, Y., Wang, F., Chen, B., & Xin, J. (2017). Robust echo state networks based on correntropy induced loss function. Neurocomputing, 267, 295–303. https://doi.org/10.1016/j.neucom.2017.05.087

    Article  Google Scholar 

  • Han, M., & Xu, M. (2018). Laplacian echo state network for multivariate time series prediction. IEEE Transactions on Neural Networks and Learning Systems, 29(1), 238–244. https://doi.org/10.1109/tnnls.2016.2574963

    Article  MathSciNet  Google Scholar 

  • Henson, M. A., & Seborg, D. E. (1994). Adaptive nonlinear control of a pH neutralization process. IEEE Transactions on Control Systems Technology, 2(3), 169–182.

    Article  Google Scholar 

  • Hochreiter, S. (1998). The vanishing gradient problem during learning recurrent neural nets and problem solutions. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 6(2), 107–116.

    Article  MathSciNet  Google Scholar 

  • Inubushi, M., & Yoshimura, K. (2017). Reservoir computing beyond memory-nonlinearity trade-off. Scientific Reports. https://doi.org/10.1038/s41598-017-10257-6

    Article  Google Scholar 

  • Isermann, R., & Münchof, M. (2011). Identification of dynamic systems. Springer.

  • Jaeger, H. (2001). The “echo state” approach to analysing and training recurrent neural networks. Tech. Rep. 34, German National Research Center for Information Technology, Bonn, Germany.

  • Jaeger, H. (2003). Adaptive nonlinear system identification with echo state networks. Advances in Neural Information Processing Systems (NIPS 2003) (pp. 609–616).

  • Jaeger, H., Lukoševičius, M., Popovici, D., & Siewert, U. (2007). Optimization and applications of echo state networks with leaky: Integrator neurons. Neural Networks, 20(3), 335–352. https://doi.org/10.1016/j.neunet.2007.04.016

    Article  MATH  Google Scholar 

  • Li, D., Han, M., & Wang, J. (2012). Chaotic time series prediction based on a novel robust echo state network. IEEE Transactions on Neural Networks and Learning Systems, 23(5), 787–799. https://doi.org/10.1109/tnnls.2012.2188414

    Article  Google Scholar 

  • Ljung, L., Andersson, C., Tiels, K., & Schön, T. B. (2020). Deep learning and system identification. In Proc IFAC Congress.

  • Lukoševičius, M. (2012). A practical guide to applying echo state networks. In G. Montavon, G. B. Orr, & K. R. Müller (Eds.), Neural networks tricks of the trade (pp. 659–686). Springer.

  • Lukoševičius, M., & Jaeger, H. (2009). Reservoir computing approaches to recurrent neural network training. Computer Science Review, 3(3), 127–149.

    Article  Google Scholar 

  • Lun, S. X., Yao, X. S., Qi, H. Y., & Hu, H. F. (2015). A novel model of leaky integrator echo state network for time-series prediction. Neurocomputing, 159, 58–66. https://doi.org/10.1016/j.neucom.2015.02.029

    Article  Google Scholar 

  • Nelles, O. (2001). Nonlinear system identification. Springer.

  • Qiao, J., Wang, L., & Yang, C. (2018). Adaptive lasso echo state network based on modified Bayesian information criterion for nonlinear system modeling. Neural Computing and Applications (pp. 1–15).

  • Rodan, A., & Tino, P. (2010). Minimum complexity echo state network. IEEE Transactions on Neural Networks, 22(1), 131–144.

    Article  Google Scholar 

  • Schoukens, J., & Ljung, L. (2019). Nonlinear system identification: A user-oriented road map. IEEE Control Systems Magazine, 39(6), 28–99. https://doi.org/10.1109/MCS.2019.2938121

    Article  MathSciNet  MATH  Google Scholar 

  • Schwedersky, B. B., Flesch, R. C. C., & Dangui, H. A. C. (2018). Practical nonlinear model predictive control using an echo state network model. In 2018 International Joint Conference on Neural Networks (IJCNN) (pp. 1–8). https://doi.org/10.1109/IJCNN.2018.8489446

  • Schwedersky, B. B., Flesch, R. C. C., & Dangui, H. A. (2019). Practical nonlinear model predictive control algorithm for long short-term memory networks. IFAC-PapersOnLine, 52(1), 468–473. https://doi.org/10.1016/j.ifacol.2019.06.106

    Article  Google Scholar 

  • Shi, Z., & Han, M. (2007). Support vector echo-state machine for chaotic time-series prediction. IEEE Transactions on Neural Networks, 18(2), 359–372. https://doi.org/10.1109/tnn.2006.885113

    Article  Google Scholar 

  • Tang, Y., Li, Z., & Guan, X. (2014). Identification of nonlinear system using extreme learning machine based Hammerstein model. Communications in Nonlinear Science and Numerical Simulation, 19(9), 3171–3183.

  • Verzelli, P., Alippi, C., & Livi, L. (2019). Echo state networks with self-normalizing activations on the hyper-sphere. Scientific Reports. https://doi.org/10.1038/s41598-019-50158-4

    Article  Google Scholar 

  • Wainrib, G., & Galtier, M. N. (2016). A local echo state property through the largest Lyapunov exponent. Neural Networks, 76, 39–45. https://doi.org/10.1016/j.neunet.2015.12.013

    Article  MATH  Google Scholar 

  • Yang, C., Qiao, J., Ahmad, Z., Nie, K., & Wang, L. (2019). Online sequential echo state network with sparse RLS algorithm for time series prediction. Neural Networks, 118, 32–42.

  • Yildiz, I. B., Jaeger, H., & Kiebel, S. J. (2012). Re-visiting the echo state property. Neural Networks, 35, 1–9.

    Article  Google Scholar 

  • Yu, W. (2004). Nonlinear system identification using discrete-time recurrent neural networks with stable learning algorithms. Information Sciences, 158, 131–147. https://doi.org/10.1016/j.ins.2003.08.002

    Article  MathSciNet  MATH  Google Scholar 

  • Zhou, H., Huang, J., Lu, F., Thiyagalingam, J., & Kirubarajan, T. (2018). Echo state kernel recursive least squares algorithm for machine condition prediction. Mechanical Systems and Signal Processing, 111, 68–86. https://doi.org/10.1016/j.ymssp.2018.03.047

    Article  Google Scholar 

Download references

Acknowledgements

An early version of paper was presented at XXIII Congresso Brasileiro de Automática (CBA 2020). This study was funded in part by the Brazilian National Council for Scientific and Technological Development (CNPq) under Grants 140283/2018-8 and 309244/2018-8, and in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior-Brasil (CAPES)-Finance Code 001.

Author information

Authors and Affiliations

  1. Programa de Pós-Graduação em Engenharia de Automação e Sistemas, Universidade Federal de Santa Catarina, 88040-900, Florianópolis, SC, Brazil

    Bernardo Barancelli Schwedersky & Hiago Antonio Sirino Dangui

  2. Department of Automation and Systems Engineering, Universidade Federal de Santa Catarina, 88040-900, Florianópolis, SC, Brazil

    Rodolfo César Costa Flesch

Authors
  1. Bernardo Barancelli Schwedersky
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rodolfo César Costa Flesch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hiago Antonio Sirino Dangui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rodolfo César Costa Flesch.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Schwedersky, B.B., Flesch, R.C.C. & Dangui, H.A.S. Nonlinear MIMO System Identification with Echo-State Networks. J Control Autom Electr Syst 33, 743–754 (2022). https://doi.org/10.1007/s40313-021-00874-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40313-021-00874-y

Keywords

Search

Navigation