Abstract
Classifier ensemble is a main direction of incremental learning researches, and many ensemble-based incremental learning methods have been presented. Among them, Learn++, which is derived from the famous ensemble algorithm, AdaBoost, is special. Learn++ can work with any type of classifiers, either they are specially designed for incremental learning or not, this makes Learn++ potentially supports heterogeneous base classifiers. Based on massive experiments we analyze the advantages and disadvantages of Learn++. Then a new ensemble incremental learning method, Bagging++, is presented, which is based on another famous ensemble method: Bagging. The experimental results show that Bagging ensemble is a promising method for incremental learning and heterogeneous Bagging++ has the better generalization and learning speed than other compared methods such as Learn++ and NCL.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Giraud-Carrier, C.: A Note on the Utility of Incremental Learning. AI Communications 13(4), 215–223 (2000)
Polikar, R., Udpa, L., Udpa, S.S., Honavar, V.: Learn++: An incremental learning algorithm for supervised neural networks. IEEE Transactions on Systems, Man, and Cybernetics - Part C: Applications and Reviews 31(4), 497–508 (2001)
Utgoff, P.E.: Incremental induction of decision trees. Machine Learning 4, 161–186 (1989)
Cauwenberghs, G., Poggio, T.: Incremental and Decremental Support Vector Machine Learning. In: Advances in Neural Information Processing Systems, vol. 12, pp. 409–415. MIT Press, Cambridge (2000)
Carpenter, G.A., Grossberg, S., Reynolds, J.H.: ARTMAP: Supervied real-time learning and classification of nonstationary data by a self organizing neural network. Neural Networks 4(5), 565–588 (1991)
Kasabov, N.: Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning. IEEE Transactions on Systems, Man and Cybernetics - Part B: Cybernetics 31(6), 902–918 (2001)
Sewell, M.: Ensemble learning (2008), http://machine-learning.martinsewell.com/ensembles/ensemble-learning.pdf, (unpublished)
Seipone, T., Bullinaria, J.: Evolving improved incremental learning schemes for neural network systems. In: Proceedings of the 2005 IEEE Congress on Evolutionary Computing (CEC 2005), Piscataway, NJ, pp. 273–280 (2005)
Inoue, H., Narihisa, H.: Self-organizing neural grove and its applications. In: Proceedings of the 2005 International Joint Conference on Neural Networks (IJCNN 2005), Montreal, Canada, pp. 1205–1210 (2005)
Minku, F.L., Inoue, H., Yao, X.: Negative correlation in incremental learning. Natural Computing 8, 280–320 (2009)
Breiman, L.: Bagging Predictors. Machine Learning 24(2), 123–140 (1996)
Schwenk, H., Bengio, Y.: Boosting neural networks. Neural Computation 12, 1869–1887 (2000)
Asuncion, D.N.A.: UCI machine learning repository (2007), http://www.ics.uci.edu/mlearn/MLRepository.html (unpublished)
Tang, K., Lin, M., Minku, F.L., Yao, X.: Selective Negative Correlation Learning Approach to Incremental Learning. Neurocomputing 72(13-15), 2796–2805 (2009)
Riedmiller, M., Braun, H.: RPROP- A fast adaptive learning algorithm. In: Proc. of ISCIS VII (1992)
Lin, C.J.: LIBSVM: A Library for Support Vector Machines (2009), http://www.csie.ntu.edu.tw/~cjlin/libsvm/ (unpublished)
Demsar, J.: Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1–30 (2006)
Maloof, M.A., Michalski, R.S.: Incremental learning with partial instance memory. Artificial Intelligence 154(1-2), 95–126 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhao, Q.L., Jiang, Y.H., Xu, M. (2010). Incremental Learning by Heterogeneous Bagging Ensemble. In: Cao, L., Zhong, J., Feng, Y. (eds) Advanced Data Mining and Applications. ADMA 2010. Lecture Notes in Computer Science(), vol 6441. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17313-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-17313-4_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17312-7
Online ISBN: 978-3-642-17313-4
eBook Packages: Computer ScienceComputer Science (R0)