An out-of-sample framework for TOPSIS-based classifiers with application in bankruptcy prediction
Introduction
Multi-criteria decision analysis (MCDA) methodologies are widely used for addressing a variety of problems; namely, selection problems, ranking problems, sorting problems, classification problems, clustering problems, and description problems, where selection problems are concerned with identifying the best alternative or a subset of best alternatives; ranking problems are concerned with constructing a rank ordering of alternatives from best to worst; sorting problems are concerned with classifying alternatives into pre-defined and ordered homogenous groups or classes; classification problems are concerned with classifying alternatives into pre-defined and unordered homogenous classes; clustering problems are concerned with classifying alternatives into not pre-defined and not ordered homogenous classes; and description problems are concerned with identifying major distinguishing features of alternatives and perform their description based on these features. In this paper, we are focusing on the solution of classification problems, or equivalently predicting class belonging. To be more specific, we are concerned with the implementation of classifiers and their performance evaluation both in-sample and out-of-sample.
One popular MCDA methodology is Technique for Order Performance by the Similarity to Ideal Solution (TOPSIS) proposed by Hwang and Yoon (1981) and used in many application areas – see Behzadian et al. (2012) for a review including a sample of application areas. This methodology was originally designed for solving ranking problems. In fact, TOPSIS provides a ranking of alternatives based on similarity scores, where the similarity score of each alternative is a function of the distances between the alternative and a couple of benchmarks commonly referred to as the positive and the negative ideal solutions. Later on, TOPSIS has been adapted for solving classification problems. However, to the best of our knowledge, TOPSIS classifiers and their performance evaluation has so far been restricted to in-sample analyses only (e.g., Tansel and Yurdakul, 2010, Abd-El Fattah et al., 2013). In sum, an out-of-sample framework for TOPSIS as a classifier is lacking. The aim of this paper is to fill this gap by proposing a new integrated framework for implementing a full classification analysis; namely, in-sample classification and out-of-sample classification. The proposed framework is intended to make TOPSIS classifiers real contenders in practice and to increase confidence in their use in a variety of critical application areas such as the prediction of risk class belonging (e.g., bankruptcy prediction, distress prediction, fraud detection, credit scoring).
The remainder of this paper unfolds as follows. In Section 2, we provide a detailed description of the proposed integrated in-sample and out-of-sample framework for TOPSIS classifiers and discuss implementation decisions. In Section 3, we empirically test the performance of the proposed framework in bankruptcy prediction of companies listed on the London Stock Exchange (LSE) and report on our findings. Finally, Section 4 concludes the paper.
Section snippets
An integrated in-sample – Out-of-sample framework for TOPSIS classifiers
In the forecasting literature, nowadays prediction models – whether designed for predicting a continuous variable (e.g., the level or volatility of the price of a strategic commodity such as crude oil) or a discrete one (e.g., risk class belonging of companies listed on a stock exchange) – have to be implemented both in-sample and out-of-sample to assess their ability to reproduce or forecast the response variable in the training sample and to forecast the response variable in the test sample,
Empirical results
In order to assess the performance of the proposed framework, we considered a sample of 6605 firm-year observations consisting of non-bankrupt and bankrupt UK firms listed on the London Stock Exchange (LSE) during 2010–2014 excluding financial firms and utilities as well as those firms with less than 5 months lag between the reporting date and the fiscal year. The source of our sample is DataStream. The list of bankrupt firms is however compiled from London Share Price Database (LSPD) – codes 16
Conclusions
The validation of prediction models requires both in-sample and out-of-sample evaluation of their performance. TOPSIS classifiers however lack a proper framework for performing their out-of-sample evaluation. In this paper, we filled this gap by proposing an instance of the case-based reasoning methodology; namely, k-nearest neighbour, trained on the outcome of a TOPSIS classifier. We assessed the performance of the proposed framework using a UK dataset on bankrupt and non-bankrupt firms. Our
Acknowledgments
This work was conducted while Prof. Pérez-Gladish was a visitant researcher at the Business School of The University of Edinburgh. She would like to thank the Spanish Ministry of Education Culture and Sport for its financial support within the framework of its International Mobility Program for Senior Researchers “Salvador de Madariaga” (Reference PRX16-0169).
Jamal Ouenniche is a Reader in Management Science at the University of Edinburgh. His research encompasses a broad range of applications and methodologies in descriptive, predictive and prescriptive analytics. He has published in “Operations Research”, "European Journal of Operational Research", "Computers and Operations Research", “American Journal of Operations Research”, “International Journal of Operational Research”, “Journal of the Operational Research Society”, "International Journal of
References (40)
- et al.
A hybrid multi-criteria decision-making model for firms competence evaluation
Expert Syst. Appl.
(2009) Performance measurement model for Turkish aviation firms using the rough-AHP and TOPSIS methods under fuzzy environment
Expert Syst. Appl.
(2011)- et al.
Integrating fuzzy C-means and TOPSIS for performance evaluation: an application and comparative analysis
Expert Syst. Appl.
(2014) - et al.
A state-of the-art survey of TOPSIS applications
Expert Syst. Appl.
(2012) - et al.
Using fuzzy number for measuring quality of service in the hotel industry
Tour. Manag.
(2007) - et al.
Domestic open-end equity mutual fund performance evaluation using extended TOPSIS method with different distance approaches
Expert Syst. Appl.
(2010) - et al.
Fuzzy Rasch model in TOPSIS: a new approach for generating fuzzy numbers to assess the competitiveness of the tourism industries in Asian countries
Tour. Manag.
(2012) - et al.
Using a multi-criteria decision making approach to evaluate mobile phone alternatives
Comput. Stand. Interfaces
(2007) - et al.
An algorithmic method to extend TOPSIS for decision-making problems with interval data
Appl. Math. Comput.
(2006) - et al.
Risk assessment model selection in construction industry
Expert Syst. Appl.
(2011)
Ranking the strategic actions of Iran mobile cellular telecommunication using two models of fuzzy QFD
Telecommun. Policy
Evaluating vehicle telematics system by using a novel MCDM technique with dependence and feedback
Expert Syst. Appl.
Comparison of weights in TOPSIS models
Math. Comput. Model.
An empirical study of classification algorithm evaluation for financial risk prediction
Appl. Soft Comput.
Fuzzy performance evaluation in Turkish banking sector using analytic hierarchy process and TOPSIS
Expert Syst. Appl.
COTS evaluation using modified TOPSIS and ANP
Appl. Math. Comput.
A performance evaluation model by integrating fuzzy AHP and fuzzy TOPSIS methods
Expert Syst. Appl.
Using fuzzy TOPSIS method for evaluating the competitive advantages of shopping websites
Expert Syst. Appl.
Empirical models for the monitoring of UK corporations
J. Bank. Financ.
A multi-criteria interval-valued intuitionistic fuzzy group decision making with Choquet integral-based TOPSIS
Expert Syst. Appl.
Cited by (41)
Extreme gradient boosting trees with efficient Bayesian optimization for profit-driven customer churn prediction
2024, Technological Forecasting and Social ChangeA modified CRITIC with a reference point based on fuzzy logic and hamming distance
2022, Knowledge-Based SystemsCitation Excerpt :The main goal is to transform the evaluation scores into standard scales ranging between the values 0 and 1. A linear scale transformation [2] is used to normalize the values of the decision matrix in several MCDM methods, including the CRITIC [3,4]. The classical CRITIC approach mandates the attribution of each criterion as beneficial or non-beneficial [5].
Measuring the territorial effort in research, development, and innovation from a multiple criteria approach: Application to the Spanish regions case
2022, Technology in SocietyCitation Excerpt :For the sake of simplicity, let us consider a decision problem where all the criteria are of the type “the greater the better”. Let us note that this does not detract from the problem's generality since we can always make all criteria “the greater the better” type by normalizing the data (see Ref. [24]). In this phase, the analyst will be able to identify the relative importance of each decision criterion in determining the position of the alternative in the rank.
Managing expert knowledge in water network expansion project implementation
2021, IFAC-PapersOnLineEWT-SMOTE to improve default prediction performance in imbalanced data: Analysis of Chinese data
2024, Journal of Forecasting
Jamal Ouenniche is a Reader in Management Science at the University of Edinburgh. His research encompasses a broad range of applications and methodologies in descriptive, predictive and prescriptive analytics. He has published in “Operations Research”, "European Journal of Operational Research", "Computers and Operations Research", “American Journal of Operations Research”, “International Journal of Operational Research”, “Journal of the Operational Research Society”, "International Journal of Production Economics", "International Journal of Production Research", “Expert Systems with Applications”, “International Review of Financial Analysis”, “Applied Financial Economics”, “Finance Letters”, “Applied Economics Letters”, “Energy Economics”, “The Journal of Developing Areas” and “Journal of Applied Business Research”.
Blanca Pérez-Gladish is a Full Time Professor at the University of Oviedo in Spain. Her main interest research fields are multiple criteria decision making and fuzzy sets theory. She has published her research in “European Journal of Operational Research”, “Annals of Operations Research”, “Applied Mathematics and Computation”, “Journal of the Operational Research Society”, “OMEGA”, “Research in International Business and Finance”, “International Journal of Operational Research”, “Corporate Social Responsibility and Environmental Management”, “International Journal of Business Science and Applied Management”, “Review of Behavioral Finance”, “Australian Journal of Management”, “Applied Mathematics and Information Sciences” and “INFOR: Information Systems and Operational Research”.
Kais Bouslah is a Lecturer in Banking and Finance at the University of St Andrews. He received his PhD from the Université du Québec à Montréal (UQAM), Canada. He is a member of the Centre for Responsible Banking & Finance, School of Management, University of St Andrews. His research interests are socially responsible finance and alternative investments. His research has been published in “Journal of Business Ethics”, “Journal of Banking & Finance” and "The British Accounting Review".