Abstract
The receiver operating characteristics (ROC) analysis has gained increasing popularity for analyzing the performance of classifiers. In particular, maximizing the convex hull of a set of classifiers in the ROC space, namely ROCCH maximization, is becoming an increasingly important problem. In this work, a new convex hull-based evolutionary multi-objective algorithm named ETriCM is proposed for evolving neural networks with respect to ROCCH maximization. Specially, convex hull-based sorting with convex hull of individual minima (CH-CHIM-sorting) and extreme area extraction selection (EAE-selection) are proposed as a novel selection operator. Empirical studies on 7 high-dimensional and imbalanced datasets show that ETriCM outperforms various state-of-the-art algorithms including convex hull-based evolutionary multi-objective algorithm (CH-EMOA) and non-dominated sorting genetic algorithm II (NSGA-II).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abbass HA (2003) Speeding up backpropagation using multiobjective evolutionary algorithms. Neural Comput 15(11):2705–2726
Asafuddoula M, Ray T, Sarker R (2015) A decomposition-based evolutionary algorithm for many objective optimization. Evolutionary Computation, IEEE Transactions on 19(3):445–460
Bettinger R (2003) Cost-sensitive classifier selection using the ROC convex hull method. Comput Sci Stat 35:142–153
Bhowan U, Johnston M, Zhang M, Yao X (2013) Evolving diverse ensembles using genetic programming for classification with unbalanced data. Evolut Comput IEEE Trans 17(3):368–386
Boyce JE, Dobkin DP, Drysdale III RLS, Guibas LJ (1982) Finding extremal polygons. In: Proceedings of the fourteenth annual ACM symposium on Theory of computing. ACM, pp 282–289
Chang CC, Lin CJ (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol 2:389–396
Chatelain C, Adam S, Lecourtier Y, Heutte L, Paquet T (2010) A multi-model selection framework for unknown and/or evolutive misclassification cost problems. Pattern Recog 43(3):815–823
Cococcioni M, Ducange P, Lazzerini B, Marcelloni F (2007) A new multi-objective evolutionary algorithm based on convex hull for binary classifier optimization. Evolut Comput 2007. CEC 2007. IEEE Congress on. IEEE, pp 3150–3156
Das I (1997) Nonlinear multicriteria optimization and robust optimality. PhD thesis, Rice University, Houston, Texas
Das I (1999) On characterizing the “knee” of the Pareto curve based on normal-boundary intersection. Struct Optim 18(2–3):107–115
Deb K, Pratap A, Agarwal S, Meyarivan TAMT (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. Evolut Comput IEEE Trans 6(2):182–197
Ducange P, Lazzerini B, Marcelloni F (2010) Multi-objective genetic fuzzy classifiers for imbalanced and cost-sensitive datasets. Soft Comput 14(7):713–728
Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27(8):861–874
Fawcett T (2008) PRIE: a system for generating rulelists to maximize ROC performance. Data mining and knowledge discovery 17(2):207–224
Ferri C, Flach P, Hernández-Orallo J (2002) Learning decision trees using the area under the ROC curve. ICML 2:139–146
Flach PA (2003) The geometry of ROC space: understanding machine learning metrics through ROC isometrics. ICML, pp 194–201
Flach PA, Wu S (2005) Repairing concavities in ROC curves. IJCAI, pp 702–707
Graham RL (1972) An efficient algorith for determining the convex hull of a finite planar set. Inf Process Lett 1(4):132–133
Gräning L, Jin Y, Sendhoff B (2006) Generalization improvement in multi-objective learning. Neural Networks, 2006. IJCNN’06. International Joint Conference on. IEEE. pp 4839–4846
Hong W, Lu G, Yang P, Wang Y, Tang K (2015) A new evolutionary multi-objective algorithm for convex hull maximization. Evolutionary Computation, 2015. CEC 2015. IEEE Congress on. IEEE, pp 931–938
Lévesque J, Durand A, Gagné C, Sabourin R (2012) Multi-objective evolutionary optimization for generating ensembles of classifiers in the ROC space. In: Proceedings of the 14th annual conference on Genetic and evolutionary computation. ACM. pp 879–886
Majnik M, Bosnic Z (2013) ROC analysis of classifiers in machine learning: a survey. Intell Data Anal 17(3):531–558
Martin D, Rosete A, Alcala-Fdez J, Herrera F (2014) A new multiobjective evolutionary algorithm for mining a reduced set of interesting positive and negative quantitative association rules. Evolut Comput IEEE Trans 18(1):54–69
Mukhopadhyay A, Maulik U, Bandyopadhyay S, Coello Coello C (2014) A survey of multiobjective evolutionary algorithms for data mining: Part I. Evolut Comput IEEE Trans 18(1):4–19
Prati RC, Flach PA (2005) ROCCER: an algorithm for rule learning based on ROC analysis. Proc Int Joint Conf Artif Intell 26(2):823–828
Provost F, Fawcett T (2001) Robust classification for imprecise environments. Mach Learn 42(3):203–231
Seada H, Deb K (2015) A unified evolutionary optimization procedure for single, multiple, and many objectives. Evolutionary Computation, IEEE Transactions on. doi:10.1109/TEVC.2015.2459718
UCI (2015) UC Irvine machine learning respository, [Online]. Available: http://archive.ics.uci.edu.c/ml
Wang P, Emmerich M, Li R, Tang K, Bäck T, Yao X (2015) Convex hull-based multi-objective genetic programming for maximizing receiver operating characteristic performance. Evolut Comput IEEE Trans 19(2):188–200
Wang P, Tang K, Weise T, Tsang EPK, Yao X (2014) Multiobjective genetic programming for maximizing ROC performance. Neurocomputing 125:102–118
Zapotecas Martínez S, Coello Coello CA (2010) A novel diversification strategy for multi-objective evolutionary algorithms. In: Proceedings of the 12th annual conference companion on Genetic and evolutionary computation. ACM. pp 2031–2034
Zhang X, Tian Y, Cheng R, Jin Y (2015) An efficient approach to nondominated sorting for evolutionary multiobjective optimization. Evolut Comput IEEE Trans 19(2):201–213
Zhao J, Fernandes VB, Jiao L, Yevseyeva I, Maulana A, Li R, Bäck T, Emmerich M (2014) Multiobjective optimization of classifiers by means of 3-D convex hull based evolutionary algorithm. arXiv preprint arXiv:1412.5710
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China (Grants Nos. 61329302 and 61175065), and the Program for New Century Excellent Talents in University (Grant No. NCET-12-0512).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hong, W., Tang, K. Convex hull-based multi-objective evolutionary computation for maximizing receiver operating characteristics performance. Memetic Comp. 8, 35–44 (2016). https://doi.org/10.1007/s12293-015-0176-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12293-015-0176-8