A fairer assessment of DMUs in a generalised two-stage DEA structure

Highlights

• Developed modelling approaches to attain fairness in a network DEA structure.

• Minimax secondary model highlights the best behaviour of the worst performing DMUs.

• CRITIC as a mechanism to highlight minority opinions within a network DEA context.

• Envisaged applications include job rotation systems and sustainable supply chains.

Abstract

In Data Envelopment Analysis (DEA), a variety of approaches have been used in the context of single-stage and basic serial two-stage systems to attain fairness in the evaluation of decision-making units (DMUs). Little work, however, has been done to address this challenge in a generalised two-stage structure featuring additional inputs in the second stage and a proportion of first-stage outputs as final outputs. In this paper, we argue that in this context, fairness is enhanced by increasing measures related to the discriminatory power and the weighting scheme of the method. We describe a mechanism that gives prominence to a more contemporary concept of fairness, incorporating diversity and inclusion of minority opinions. These aspects have, to our knowledge, not yet received explicit attention in the methodological development of DEA. We propose a novel combination of an additive self-efficiency aggregation model, a minimax secondary goal model, and the CRiteria Importance Through Inter-criteria Correlation (CRITIC) method, in order to promote these aspects of fairness, and thus achieve a better degree of cooperation between the stages of a DMU and among DMUs. The additive aggregation model is chosen over the alternative multiplicative approach for a variety of reasons relating to the emphasis on the intermediate products exchanged and the simplification. The minimax model offers peer evaluation in which each DMU aims to evaluate the worst of the others in the best possible light. Application of the CRITIC method to DEA addresses the aggregation problem within the cross-efficiency concept. Practical applications of this approach could include supporting the determination of training needs in job rotation manufacturing, or evaluation of sustainable supply chains. The paper includes a description of a numerical experiment, illustrating the approach.

Introduction

Data Envelopment Analysis (DEA) is a non-parametric approach for evaluating the performance of Decision-Making Units (DMUs) that use inputs to produce outputs (Cook et al., 2014). DEA was developed by Charnes et al. (1978) (CCR) for the constant returns-to-scale assumption. Traditional DEA does not model the internal processes in a DMU. As a result, a relatively large proportion of DMUs emerge as DEA-efficient, without a means to distinguish them (Ma et al., 2017). To enable the study of internal structures, research has extended DEA models to consider network structures (Chen and Zhu, 2017, Guo et al., 2017, Kao, 2009, Kao, 2014, Kao and Hwang, 2011, Örkcü et al., 2019, Wanke and Barros, 2014). In a two-stage process in particular, inputs used by a DMU feed into a first stage, producing intermediate outputs that feed into a second stage, producing the final outputs of the entire system. Such a structure facilitates the measurement of both the overall system and its individual stages’ efficiencies (Mahdiloo et al., 2016).

Measuring the performance can be challenging when inputs and outputs are shared among different processes and are not easily distinguished (Zha & Liang, 2010). Yu and Shi (2014) examine a two-stage structure with additional inputs in the second stage and some of the intermediate products as final outputs, towards building cooperative and leader–follower models. Jianfeng (2015) considers a network DEA model, in which the inputs are classified into those that are entirely integrated into one stage and those that are shared between the two stages. Ma et al. (2017) propose a parallel–series hybrid two-stage DEA model utilising the principles of additive and multiplicative efficiency decomposition.

While two-stage DEA models have the potential to increase managerial insight into the sources of inefficiency, two major problems similar to those in a single-stage emerge. The first one concerns the lack of discrimination power due to a high number of efficient DMUs (Mahdiloo et al., 2016). The second challenge relates to an ‘unrealistic’ weighting scheme. Indeed, it is allowed for high relative-importance weights to be assigned to ‘less important’ inputs or outputs, and/or low weights to significant factors. This choice of weights could turn a DMU into an efficient unit (Ghasemi et al., 2014).

In this paper we are interested in methods which aim to avoid a low degree of discrimination, unrealistic weight schemes, and to use a system of ranking that encourages cooperation by the units being evaluated. While doing so, we also wish to provide a mechanism that gives a voice for minority opinions. This aspect has, to our knowledge, not yet received explicit attention in the methodological development of DEA. In short, we say that we intend to tweak DEA methodology to improve the fairness¹ in the evaluation outcomes. We summarise the core literature, relevant to fairness evaluation in DEA, in Table 1.

Among those methods tested towards fairness is cross-efficiency (CE), which adds peer-evaluation to the self-evaluation principle (Sexton et al., 1986). As stressed by Anderson et al. (2002), CE improves the probability of obtaining a unique ranking. A critical drawback of CE is the non-uniqueness of optimal weights, which leads to the non-uniqueness of cross-efficiencies. To alleviate this, Doyle and Green (1994) recommended the adoption of alternative secondary goals in an aim to select unique optimal multipliers. In particular, they introduced an aggressive and a benevolent model, while the secondary objective functions in Liang et al. (2008) reflected the minimisation of total deviation, maximum deviation, and mean absolute deviation from an ‘ideal’ point. The interested reader could also check(Wang and Chin, 2010a, Wang and Chin, 2011, Wu, Chu, Sun, Zhu and Liang, 2016, Wu et al., 2012a, Wu et al., 2012b) and (Li et al., 2018). The non-uniqueness issue is also critical in a network system. Kao and Liu (2019) developed an aggressive CE model to measure the efficiency in two basic network structures. Örkcü et al. (2019) came up with a neutral CE model in a two-stage system, which is indifferent to the preference choice between the aggressive and benevolent formulations.

The aggregation of the cross-efficiency scores is another issue in CE. An appropriate aggregation strategy can enable the DMUs to accept their ranking. Although the average method has proven effective in ensuring a credible ranking (Liang et al., 2008, Wang and Chin, 2010b), it loses sight of the weights assigned to scores (Wang & Wang, 2013). To accommodate this issue, Wu et al. (2011) utilised the Shannon entropy, allocating a fixed but different weight to each DMU. Wu et al. (2012a) highlighted that this is problematic, since it ignores the primary role of the self-evaluated efficiency of each DMU. They, thus, embedded the Shannon entropy into the CE by considering the association among the self and the peer-evaluation values. For more recent work on this, see Song and Liu, 2018, Wang and Chin, 2011, and Wang and Wang (2013).

Fairness in the evaluation outcomes has been achieved even via the integration of game theoretic concepts within traditional single-stage and two-stage DEA networks. For instance, Zhou et al. (2013) introduced a Nash bargaining game model to obtain a unique efficiency decomposition for the two constituent sub-stages of the centralised model. Their approach leads to a fair context, in that it reflects how the two sub-stages bargain with each other for better efficiencies. An et al. (2017) also used Nash bargaining, but introduced a framework for setting fair target values for intermediate products of two-stage systems, so that the two stages are encouraged to collaborate with each other within a pre-agreed range of fair outcomes. Wu, Chu, Sun and Zhu (2016) proposed a CE evaluation approach based on Pareto improvement. A merit of their approach is that it always generates a set of Pareto optimal cross-efficiencies for the DMUs. Li (2017) introduced a sequence of leader–follower procedures as to ensure a fair evaluation in the sense that it guarantees that the same result is obtained for the second ( $=$ follower) stage of a DMU as would be obtained applying the standard DEA model to the second stage independently. A number of studies have been reported in this direction, such as Li et al., 2018, Ma et al., 2014, and Yu and Shi (2014).

In summary, fairness in the evaluation of DMUs has been extensively explored via CE towards single-stage and basic network structures. Nevertheless, when the discussion shifts to more complex structures where inputs and outputs are shared among different processes, there is limited attention to how to achieve more meaningful results for the DMUs. This intricacy is due to the additional inputs in the second stage obtained from the external environment and the dual role of the intermediate products. There are several enlightening applications, especially in logistics, supply chain, and manufacturing, that could justify the necessity of exploring fairness in the performance evaluation of a generalised two-stage structure. These are discussed in more depth with an example in Section 3 and in the implications of Section 4.2.

In our paper, we firstly introduce an additive self-efficiency aggregation model that can highlight the strength of each sub-stage and obtain the most favourable efficiency for the DMU overall. Since the optimal set of weights derived from the aggregation model may not be unique, we employ a minimax secondary goal model. The reasons for the adoption of this model are twofold: (i) it corresponds to cooperative situations (Liang et al., 2008), since sub-stages behave benignly, and (ii) it is compatible with multi-stage systems where the individual sub-stages pursue mutual cooperation via the maximisation of the overall efficiency (Yu & Shi, 2014). The multi-objective model is converted using the Compromise Programming methodology as a means to identify a good solution that balances the objectives.

On the aggregation of the individual CE, existing frameworks (Wang and Chin, 2011, Wu et al., 2012a) pay attention to the reasonable allocation of the weights by limiting the range between self and peer-assessment efficiencies. This condition may indicate consistency from the perspective of the majority opinion. However, considering that many organisations are moving towards systems of evaluation in which also the opinions of minorities are valued (Park & DeShon, 2010), we introduce an aggregation method that rewards contrast. We rely upon the CRiteria Importance Through Inter-criteria Correlation (CRITIC) method (Diakoulaki et al., 1995), an objective method for eliciting weights in multi-criteria problems. With the exception of He and Ma (2015), our paper is the first to apply the CRITIC method in the context of DEA. Its novel function and meaning as deployed in the paper is further described in Section 3.3.2, and differences with the above study are discussed in Section 4.2. Besides, CRITIC would be compatible with the minimax model introduced herein; this is justified by the model’s nature to highlight the best behaviour of the worst-performing unit, while the scores of the other better-performing units might decrease.

The remainder of the paper is organised as follows. Section 2 describes the methodological background. In Section 3, we develop the alternative modelling approach for the generalised two-stage DEA structure. Section 4 illustrates the methods with a numerical example. Section 5 presents conclusions and further research.