Abstract
Big Data is being touted as the next big thing arousing technical challenges that confront both academic research communities and commercial IT deployment. The root sources of Big Data are founded on infinite data streams and the curse of dimensionality. It is generally known that data which are sourced from data streams accumulate continuously making traditional batch-based model induction algorithms infeasible for real-time data mining. In the past many methods have been proposed for incrementally data mining by modifying classical machine learning algorithms, such as artificial neural network. In this paper we propose an incremental learning process for supervised learning with parameters optimization by neural network over data stream. The process is coupled with a parameters optimization module which searches for the best combination of input parameters values based on a given segment of data stream. The drawback of the optimization is the heavy consumption of time. To relieve this limitation, a loss function is proposed to look ahead for the occurrence of concept-drift which is one of the main causes of performance deterioration in data mining model. Optimization is skipped intermittently along the way so to save computation costs. Computer simulation is conducted to confirm the merits by this incremental optimization process for neural network.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
P. McCullagh, J.A. Nelder, Generalized Linear Models, 2nd edn. (Chapman & Hall, London, 1989)
P.-F. Pai, T.-C. Chen, Rough set theory with discriminant analysis in analyzing electricity loads. Expert Syst. Appl. 36, 8799–8806 (2009)
M.M. Gaber, A. Zaslavsky, S. Krishnaswamy, Mining data streams: a review. ACM SIGMOD Rec. 34(2), 18–26 (2005)
W. Fan, A. Bifet, Mining big data: current status, and forecast to the future. SIGKDD Explor. 14(2), 1–5 (2012)
A. Murdopo, Distributed decision tree learning for mining big data streams. Master of Science Thesis, European Master in Distributed Computing, July 2013
B. Yoshua, A. Courville, P. Vincent, Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)
G. Zhou, K. Sohn, H. Lee, Online incremental feature learning with denoising autoencoders. AISTATS, 2012
O. Aran, E. Alpaydın, An incremental neural network construction algorithm for training multilayer perceptrons. ICANN/ICONIP‘03, 2003
A.G. Ivakhnenko, Heuristic self-organization in problems of engineering cybernetics. Automatica 6, 207–219 (1970)
A. Fang, F. Ramos, Tuning online neural networks with reinforcement learning. 2014 Research Conversazione, The School of Information Technologies, University of Sydney, 2014
N. Shiraga, S. Ozawa, S. Abe, A reinforcement learning algorithm for neural networks with incremental learning ability. Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP ‘02, vol. 5, November 2002, pp. 2566–2570
G.E. Hinton, S. Osindero, Y.-W. Teh, A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)
A. Carlevarino, R. Martinotti, G. Metta, G. Sandini, An incremental growing neural network and its application to robot control. Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks, 2000. IJCNN 2000, vol. 5, pp. 323–328
M. Ring, Sequence learning with incremental higher-order neural networks. Technical Report, University of Texas at Austin, Austin, TX, USA, 1993
C. Constantinopoulos, A. Likas, An incremental training method for the probabilistic RBF network. IEEE Trans. Neural Netw. 17(4), 966–974 (2006)
M. Tscherepanow, Incremental on-line clustering with a topology-learning hierarchical ART neural network using hyperspherical categories, Advances in data mining, poster and industry. Proceedings of the 12th Industrial Conference on Data Mining (ICDM2012) (ibai-publishing, Fockendorf 2012), p. 22
F. Costa, P. Frasconi, V. Lombardo, G. Soda, Learning incremental syntactic structures with recursive neural networks. Proceedings of the Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies, 2000, pp. 458–461
C. MacLeod, G.M. Maxwell, Incremental evolution in ANNs: neural nets which grow. Artif. Intell. Rev. 16(3), 201–224 (2001)
T. Seipone, J.A. Bullinaria, Evolving improved incremental learning schemes for neural network systems. The 2005 I.E. Congress on Evolutionary Computation, vol. 3, pp. 2002–2009
J.-F. Connolly, E. Granger, R. Sabourin, Incremental adaptation of fuzzy ARTMAP neural networks for video-based face classification. Proceedings of the 2009 I.E. Symposium on Computational Intelligence in Security and Defense Applications (CISDA 2009), July 2009, pp. 1–8
R. Kohavi, Wrappers for Performance Enhancement and Oblivious Decision Graphs (Department of Computer Science, Stanford University, Stanford, 1995)
S. Fong, S. Deb, X.-S. Yang, J. Li, Feature selection in life science classification: metaheuristic swarm search. IT Prof. 16(4), 24–29 (2014)
R. Kohavi, The power of decision tables. 8th European Conference on Machine Learning, pp. 174–189, 1995b
S. Ioannou, L. Kessous, G. Caridakis, K. Karpouzis, V. Aharonson, S. Kollias, Adaptive on-line neural network retraining for real life multimodal emotion recognition. Artificial Neural Networks—ICANN 2006. Lecture notes in computer science, vol. 4131, 2006, pp. 81–92
Acknowledgement
The authors are thankful for the financial support from the research grant “Rare Event Forecasting and Monitoring in Spatial Wireless Sensor Network Data,” Grant no. MYRG2014-00065-FST, offered by the University of Macau, FST, and RDAO.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Fong, S., Fang, C., Tian, N., Wong, R., Yap, B.W. (2016). Self-Adaptive Parameters Optimization for Incremental Classification in Big Data Using Neural Network. In: Hung, P. (eds) Big Data Applications and Use Cases. International Series on Computer Entertainment and Media Technology. Springer, Cham. https://doi.org/10.1007/978-3-319-30146-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-30146-4_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-30144-0
Online ISBN: 978-3-319-30146-4
eBook Packages: Computer ScienceComputer Science (R0)