Abstract
Ensemble pruning is a technique to reduce ensemble size and increase its accuracy by selecting an optimal or suboptimal subset as subensemble for prediction. Many ensemble pruning algorithms via greedy search policy have been recently proposed. The key to the success of these algorithms is to construct an effective metric to supervise the search process. In this paper, we contribute a new metric called DBM for greedy ensemble pruning. This metric is related not only to the diversity of base classifiers, but also to the prediction details of current ensemble. Our experiments show that, compared with greedy ensemble pruning algorithms based on other advanced metrics, DBM based algorithm induces ensembles with much better generalization ability.
The work is supported by the National Science Foundation of China (No. 60901078).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. John Wiley and Sons, Chichester (2004)
Breiman, L.: Bagging predictors. Machine Learning, 123–140 (1996)
Freund, Y., Schapire, R.F.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55, 119–139 (1997)
Breiman, L.: Random forests. Machine learning 45, 5–32 (2001)
Zhang, D., Chen, S., Zhou, Z., Yang, Q.: Constraint projections for ensemble learning. In: 23rd AAAI Conference on Artificial Intelligence (AAAI 2008), pp. 758–763 (2008)
Zhou, Z.H., Wu, J., Tang, W.: Ensembling neural networks: Many could be better than all. Artificial intelligence 137, 239–263 (2002)
Zhang, Y., Burer, S., Street, W.N.: Ensemble pruning via semi-definite programming. The Journal of Machine Learning Research, 1315–1338 (2006)
Chen, H., Tino, P., Yao, X.: Predictive ensemble pruning by expectation propagation. IEEE Transactions on Knowledge and Data Engineering, 999–1013 (2009)
Caruana, R., Niculescu-Mizil, A., Crew, G., Ksikes, A.: Ensemble Selection from Librariries of Models. In: 21st International Conference on Machine Learning (2004)
Martinez-Muverbnoz, G., Suarez, A.: Aggregation ordering in bagging. International conference on artificial intelligence and applications (IASTED), pp. 258–263. Calgary: Acta press (2004)
Banfield, R.E., Hall, L.O., Bowyer, K.W., Kegelmeyer, W.P.: Ensemble diversity measures and their application to thinning. Information Fusion, 49–62 (2005)
Partalas, I., Tsoumakas, G., Vlahavas, I.P.: Focused Ensemble Selection: A Diversity-Based Method for Greedy Ensemble Selection. In: 18th European Conference on Artificial Intelligence, Patras, Greece, July 21-25, pp. 117–121 (2008)
Martínez-Munoz, G., Hernández-Lobato, D., Suárez, A.: An analysis of ensemble pruning techniques based on ordered aggregation. IEEE Transactions on Pattern Analysis and Machine Intelligence 31, 245–259 (2009)
Partalas, I., Tsoumakas, G., Vlahavas, I.P.: An ensemble uncertainty aware measure for directed hill climbing ensemble pruning. Machine Learning, 257–282 (2010)
Lu, Z.Y., Wu, X.D., Zhu, X.Q., Bongard, J.: Ensemble Pruning via Individual Contribution Ordering. In: 16th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (2010)
Asuncion, D.N.A.: UCI machine learning repository (2007)
Quinlan, J.R.: C4.5: programs for machine learning. Morgan Kaufmann, San Francisco (1993)
Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Guo, H., Zhi, W., Han, X., Fan, M. (2011). A New Metric for Greedy Ensemble Pruning. In: Deng, H., Miao, D., Lei, J., Wang, F.L. (eds) Artificial Intelligence and Computational Intelligence. AICI 2011. Lecture Notes in Computer Science(), vol 7003. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23887-1_80
Download citation
DOI: https://doi.org/10.1007/978-3-642-23887-1_80
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23886-4
Online ISBN: 978-3-642-23887-1
eBook Packages: Computer ScienceComputer Science (R0)