Skip to main content
SpringerLink
Log in

Deep Neural Networks and Hybrid GMDH-Neuro-fuzzy Networks in Big Data Analysis

  • Chapter
  • First Online:
Big Data: Conceptual Analysis and Applications

Part of the book series: Studies in Big Data ((SBD,volume 58))

Abstract

Deep neural networks (DNN) and their generalizations—hybrid DNN are considered which represent the efficient tools for BD analysis. The properties and drawbacks of deep learning are considered and analyzed. Encoders—decoders and restricted Boltzmann machines are described and their applications for Deep learning implementation are presented. Methods of regularization of DL: penalty functions, Dropout and Bagging are presented. New class of deep learning networks are suggested and presented. so-called GMDH-neo-fuzzy networks representing a combination of self-organization method GMDH and fuzzy neural networks. Due to principle of self-organization and small number of tuning parameters GMDH enables to simplify and accelerate the training of DN. Several variants of this class hybrid networks are considered and algorithms of their structure synthesis based on GMDH are suggested and analyzed. The application of GMDH enables to reduce dimensionality of training DN and accelerate the convergence of training and by this solve some problems of Big Data Analysis. Experimental investigations of hybrid GMDH-neo-fuzzy networks are carried out and their results are presented and analyzed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. G. Hinton, S. Osindero, Y.-W. Teh, A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)

    Article  MathSciNet  Google Scholar 

  2. I. Goodfellow, Y. Bengio, A. Courville, Deep Learning (MIT Press, 2016)

    Google Scholar 

  3. Y. Bengio, Y. LeCun, G. Hinton, Deep learning. Nature 521, 436–444 (2015)

    Article  Google Scholar 

  4. J. Schmidhuber, Deep learning in neural networks: an overview. Neural Netw. 61

    Google Scholar 

  5. E. Lughofer, Evolving Fuzzy Systems—Methodologies, Advanced Concepts and Applications (Springer, Berlin, 2011, 2015), pp. 85–117

    Google Scholar 

  6. Z. Hu, Y.V. Bodyanskiy, O.K. Tyshchenko, A cascade deep neuro-fuzzy system for high-dimensional online possibilistic fuzzy clustering, in Proceedings of the XI-th International Scientific and Technical Conference “Computer Science and Information Technologies” (CSIT 2016) (2016), pp. 119–122. https://doi.org/10.1109/stc-csit.2016.7589884

  7. P. Angelov, D. Filev, N. Kasabov, Evolving Intelligent Systems: Methodology and Applications (Willey, 2010)

    Google Scholar 

  8. Y.V. Bodyanskiy, O.A. Vynokurova, A.I. Dolotov, Self-learning cascade spiking neural network for fuzzy clustering based on group method of data handling. J. Autom. Inform. Sci. 45(3), 23–33 (2013)

    Google Scholar 

  9. Y. Bodyanskiy, O. Vynokurova, A. Dolotov, O. Kharchenko, Wavelet-neuro-fuzzy network structure optimization using GMDH for the solving forecasting tasks, in Proceedings of the 4th International Conference on Inductive Modelling ICIM 2013, Kyiv (2013), pp. 61–67

    Google Scholar 

  10. Y. Bodyanskiy, O. Vynokurova, N. Teslenko, Cascade GMDH-wavelet-neuro-fuzzy network, in Proceedings of the 4th International Workshop on Inductive Modeling «IWIM 2011» , Kyiv, Ukraine (2011), pp. 22–30

    Google Scholar 

  11. Y. Bodyanskiy, Y. Zaychenko, E. Pavlikovskaya, M. Samarina, Y. Viktorov, The neo-fuzzy neural network structure optimization using the GMDH for the solving forecasting and classification problems, in Proceedings of the International Workshop on Inductive Modeling, Krynica, Poland (2009), pp. 77–89

    Google Scholar 

  12. A.G. Ivakhnenko, Heuristic self-organization in problems of engineering cybernetics. Automatica 6(2), 207–219 (1970)

    Article  Google Scholar 

  13. T. Yamakawa, E. Uchino, T. Miki, H. Kusanagi, A neo fuzzy neuron and its applications to system identification and prediction of the system behavior, in Proceedings of the 2nd International Conference on Fuzzy Logic and Neural Networks (1992), pp. 477–483

    Google Scholar 

  14. E. Uchino, T. Yamakawa, Soft computing based signal prediction, restoration and filtering, in Intelligent Hybrid Systems: Fuzzy Logic, Neural Networks and Genetic Algorithms (Kluwer Academic Publisher, Boston, 1997), pp. 331–349

    Google Scholar 

  15. T. Miki, T. Yamakawa, Analog implementation of neo-fuzzy neuron and its on-board learning, in Computational Intelligence and Applications (WSES Press, Piraeus, 1999), pp. 144–149

    Google Scholar 

  16. M. Sugeno, G.T. Kang, Structure identification of fuzzy model. Fuzzy Sets Syst. 28, 15–33 (1998)

    Article  MathSciNet  Google Scholar 

  17. T. Takagi, M. Sugeno, Fuzzy identification of systems and its applications to modeling and control. IEEE Trans. Syst. Man Cybern. 15, 116–132 (1985)

    Google Scholar 

  18. N. Kasabov, Evolving Connectionist Systems (Springer, London, 2003)

    Book  Google Scholar 

  19. E. Lughofer, Evolving Fuzzy Systems—Methodologies, Advanced Concepts and Applications (Springer, Berlin, 2011)

    Book  Google Scholar 

  20. R.J.-S. Jang, ANFIS: adaptive-network-based fuzzy inference systems. IEEE Trans. Syst. Man Cybern. 23, 665–685 (1993)

    Google Scholar 

  21. R.J.-S. Jang, C.-T. Sun, E. Mizutani, Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence (Prentice Hall, Upper Saddle River, 1997)

    Google Scholar 

  22. S. Osowski, Sieci neuronowe do przetwarzania informacji (Oficyna Wydawnicza Politechniki Warszawskiej, Warszawa, 2006)

    Google Scholar 

  23. A.G. Ivakhnenko, Long-Term Forecasting and Control of Complex Systems (Technica, Kiev, 1975)

    Google Scholar 

  24. A.G. Ivakhnenko, Polynomial theory of complex systems. IEEE Trans. Syst. Man. Cybern. 1(4), 364–378 (1971)

    Google Scholar 

  25. A.G. Ivakhnenko, Self-Learning Systems of Recognition and Automatic Control (Technica, Kiev, 1969)

    Google Scholar 

  26. A.G. Ivakhnenko, D. Wuensch, G.A. Ivakhnenko, Inductive sorting-out GMDH algorithms with polynomial complexity for active neurons of neural networks. Neural Netw. 2, 1169–1173 (1999)

    Google Scholar 

  27. A.G. Ivakhnenko, G.A. Ivakhnenko, J.A. Mueller, Self-organization of the neural networks with active neurons. Pattern Recognit. Image Anal. 4(2), 177–188 (1994)

    Google Scholar 

  28. G.A. Ivakhnenko, Self-organization of neuronet with active neurons for effects of nuclear test explosions forecasting. Syst. Anal. Model. Simul. 20, 107–116 (1995)

    Google Scholar 

  29. K.S. Narendra, K. Parthasarathy, Identification and control of dynamical systems using neural networks. IEEE Trans. Neural Netw. 1, 4–26 (1990)

    Article  Google Scholar 

  30. T. Kondo, Identification of radial basis function networks by using revised GMDH-type neural networks with a feedback loop, in Proceedings of the SICE Annual Conference, Tokyo, Japan (2002), pp. 2882–2887

    Google Scholar 

  31. T. Ohtani, Automatic variable selection in RBF network and its application to neurofuzzy GMDH, in Proceedings of the Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies, vol. 2 (2000), pp. 840–843

    Google Scholar 

  32. Yu. Zaychenko, The fuzzy group method of data handling and its application for economical processes forecasting. Sci. Inq. 7(1), 83–96 (2006)

    MathSciNet  Google Scholar 

  33. T. Ohtani, H. Ichihashi, T. Miyoshi, K. Nagasaka, Y. Kanaumi, Structural learning of neurofuzzy GMDH with Minkowski norm, in Proceedings of the 1998 Second International Conference on Knowledge-Based Intelligent Electronic Systems, vol. 2 (1998), pp. 100–107

    Google Scholar 

  34. Y. Bodyanskiy, O. Vynokurova, I. Pliss, Hybrid GMDH-neural network of computational intelligence, in Proceedings of the 3rd International Workshop on Inductive Modeling, Krynica, Poland (2009), pp. 100–107

    Google Scholar 

  35. A.G. Ivakhnenko, V.S. Stepashko, Disturbance Tolerance of Modeling (Naukova Dumka, Kiev, 1985)

    Google Scholar 

  36. Y. Bodyanskiy, N. Teslenko, P. Grimm, Hybrid evolving neural network using kernel activation functions, in Proceedings 17th Zittau East-West Fuzzy Colloquium, Zittau/Goerlitz, HS (2010), pp. 39–46

    Google Scholar 

  37. D.T. Pham, X. Liu, Neural Networks for Identification, Prediction and Control (Springer, London, 1995)

    Book  Google Scholar 

  38. A. Bifet, Adaptive Stream Mining: Pattern Learning and Mining from Evolving Data Streams (IOS Press, Amsterdam, 2010)

    MATH  Google Scholar 

  39. C.C. Aggarwal, Data Streams: Models and Algorithms (Advances in Database Systems) (Springer, New York, 2007)

    Book  Google Scholar 

  40. Y. Bodyanskiy, O. Tyshchenko, D. Kopaliani, An extended neo-fuzzy neuron and its adaptive learning algorithm. Int. J. Intell. Syst. Appl. (IJISA) 7(2), 21–26 (2015)

    Google Scholar 

  41. L.-X. Wang, Adaptive Fuzzy Systems and Control. Design and Statistical Analysis (Prentice Hall, Upper Saddle River, 1994)

    Google Scholar 

  42. L.-X. Wang, J.M. Mendel, Fuzzy basis functions, universal approximation, and orthogonal least-squares learning. IEEE Trans. Neural Netw. 3(5), 807–814 (1992)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv, Ukraine

    Michael Z. Zgurovsky

  2. National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv, Ukraine

    Yuriy P. Zaychenko

Authors
  1. Michael Z. Zgurovsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuriy P. Zaychenko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Z. Zgurovsky .

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Zgurovsky, M.Z., Zaychenko, Y.P. (2020). Deep Neural Networks and Hybrid GMDH-Neuro-fuzzy Networks in Big Data Analysis. In: Big Data: Conceptual Analysis and Applications. Studies in Big Data, vol 58. Springer, Cham. https://doi.org/10.1007/978-3-030-14298-8_2

Download citation

Publish with us

Policies and ethics

Search

Navigation