Abstract
In this paper, we propose a comparative study between 3 random forest pruning measures using simultaneously or separately performance and diversity. The measures will be used with a Sequential Forward Selection (SFS) path to reduce the number of initial trees. The methods are applied on a dataset from the UCI Repository and a diabetic monitoring application. The results allow obtaining ensembles of smaller sizes with similar or even exceeding, in some cases, performance of the initial forest with considerable improvements in the case of use of performance and diversity.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Chapman & Hall, New York (1984)
Breiman, L.: Random forests. Mach. Learn. 45, 5–32 (2001)
Partalas, I., Tsoumakas, G., Katakis, I., Vlahavas, I.: Ensemble pruning using reinforcement learning. In: Antoniou, G., Potamias, G., Spyropoulos, C., Plexousakis, D. (eds.) SETN 2006. LNCS (LNAI), vol. 3955, pp. 301–310. Springer, Heidelberg (2006)
Partalas, I., Tsoumakas, G., Vlahavas, I.: An ensemble uncertainty aware measure for directed hill climbing ensemble pruning. Mach. Learn. 81, 257–282 (2010)
Partalas, I., Tsoumakas, G., Vlahavas, I.: A study on greedy algorithms for ensemble pruning. Technical report TR-LPIS-360-12 (2012)
Kulkarni, V.Y., Sinha, P.K.: Pruning of random forest classifiers: a survey and future directions. In: International Conference on Data Science & Engineering (ICDSE), Cochin, Kerala, India (2012)
Bernard, S., Heutte, L., Adam, S.: On the selection of decision trees in random forest. In: IJCNN, pp. 302–307 (2009)
Bernard, S., Adam, S., Heutte, L.: Dynamic random forests. Pattern Recogn. Lett. 33(12), 1580–1586 (2012)
TalebZouggar, S., Adla, A.: EMnGA: Entropy Measure and Genetic Algorithms Based Method for Heterogeneous Ensembles Selection. IDEAL (2018)
Zouggar, S.T., Adla, A.: New function for ensemble pruning. In: ICDSST 2018, pp. 181–190 (2018)
Zouggar, S.T., Adla, A.: A diversity-accuracy measure for homogenous ensemble selection. Int. J. Interact. Multimedia Artif. Intell. (IJMAI) 5(5), 63–70 (2018)
Asuncion, A., Newman, D.: UCI machine learning repository (2007). http://www.ics.uci.edu/»mlearn/MLRepository.html
TalebZouggar, S., Atmani, B., Adla, A.: On generating and simplifying decision trees using tree automata models. INFOCOMP J. Comput. Sci. 12(2), 32–43 (2013)
Breiman, L., Cutler, A.: Random forests, Berkeley (2005). http://www.stat.berkeley.edu/users/breiman/RandomForests/
Ferri, F.J., Pudil, P., Hatef, M., Kittler, J.: Comparative study of techniques for large-scale feature selection. Pattern Recogn. Pract. 4, 403–413 (1994)
Yang, F., Lu, W.H., Luo, L.K., Li, T.: Margin optimization based pruning for random forest. Neurocomputing 94, 54–63 (2012)
Nan, F., Wang, J., Saligrama, V.: Pruning random forests for prediction on a budget. In: NIPS (2016)
Jiang, X., Wu, C., Guo, H.: Forest pruning based on branch importance. Comput. Intell. Neurosci. 2017, 1–11 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Taleb Zouggar, S., Adla, A. (2020). Measures of Random Forest Pruning: Comparative Study and Experiment on Diabetic Monitoring. In: Ezziyyani, M. (eds) Advanced Intelligent Systems for Sustainable Development (AI2SD’2019). AI2SD 2019. Advances in Intelligent Systems and Computing, vol 1103. Springer, Cham. https://doi.org/10.1007/978-3-030-36664-3_30
Download citation
DOI: https://doi.org/10.1007/978-3-030-36664-3_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36663-6
Online ISBN: 978-3-030-36664-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)