A three-stage DEA model to evaluate learning-teaching technical efficiency: Key performance indicators and contextual variables
Introduction
The efficiency of university higher education is crucial to the development and growth of countries. Specifically, the production of human capital and the creation of new knowledge are fundamental factors for national economies that must compete at an international level. Therefore, studies such as this one, determining which aspects of higher education should be improved in order to achieve greater efficiency, are quite useful.
Over recent years, the growing importance of undergraduate and post graduate degree studies in tourism in Spain has justified the analysis of teaching efficiency in tourism studies (considering the fact that between the academic course years of 1988/89 and 2008/09, two and a half times the students pursued tourism degrees during a period in which, overall, diploma and degree studies decreased by approximately 25%, National Institute of Statistics [INE, in Spanish], 2010). This work focuses specifically on the tourism degree of the University of Alicante (Spain) during the 2011/12, 2012/13 and 2013/14 academic course years.
The aim of this work is first, to evaluate the efficiency of the learning-teaching process in higher education, specifically in the tourism degree and second, to select the correct indicators that permit an adequate evaluation of the performance and efficiency of education. The identification and subsequent study of the variables used to monitor the progress and success of the teaching process (Key Performance Indicators, KPIs) is a fundamental issue. According to the expert systems perspective, the methodology used in this study facilitates and improves the identification and quantification of potential improvements in terms of reduction of resources and/or improvement in academic results.
Since the work of Charnes, Cooper, and Rhodes (1978), Data Envelopment Analysis (DEA) has been widely used to analyze efficiency in diverse areas, specifically, in higher education. It is ideal for analyzing activities in sectors that require multiple resources in their production process in order to generate different types of products. Thus, DEA has become one of the most frequently used methods for determining which variables contribute to improving higher education performance (Agasisti and Dal Bianco, 2009, Johnes, 2006a, Joumady and O. Ris, 2005). DEA has enabled the assessment of the relative efficiency of the units in higher education institutions and has permitted the determination of which inputs and outputs contribute to the achievement of optimum performance.
The methodology selected for this study was implemented in three stages. First, the DEA method developed by Fried and Lovell (1996) and subsequently modified by Muñiz (2002) was applied. This method considers the contextual variables that affect the teaching process; second, super efficiency was analyzed, leading to the prioritization of the efficient units; and, finally, a sensitivity analysis was conducted to determine the contribution of each variable in terms of the efficiency level without the need to omit any variables. A significant theoretical contribution of this study is that it improves the manner in which the key variables were selected in previous studies on teaching efficiency using DEA, such as those by Montoneri, Lee, Lin, and Huang (2011, 2012), since it takes advantage of the information provided by contextual variables, super efficiency and the influence of variables (KPIs) on technical efficiency.
This study has been organized as follows: Section 2 presents a literature review in order to support the selection of the analysis model and variables. Section 3 presents the methodological model to be justified and described. The data from the study is presented in Section 4 and the results of the same are presented and discussed in Section 5. Finally, Section 6 offers our conclusions and suggests the main ideas that may be implemented in order to improve the learning-teaching efficiency analysis.
Section snippets
Literature review on efficiency in higher education
Assessing the efficiency of higher education institutions is not a simple task given that these are complex organizations having multiple inputs and outputs (Abd Aziz et al., 2013, Johnes, 2006b). Although efficiency in higher education has also been analyzed using parametric and OLS (Ordinary Least Square) regression methods (Johnes and Taylor, 1990, Zoghbi et al., 2013), ever since Johnes and Johnes (1993) the most widely used methodology have been frontier methods such as Data Envelopment
Method
Based on the reviewed studies, this section presents the methodology used in our analysis, the inputs and outputs used and an outline of our reasons for selecting them.
The literature revealed that Data Envelopment Analysis (DEA) was the most frequently used method for analyzing efficiency in the context of higher education, although other methods such as the Stochastic Frontier Analysis (SFA) have also been used. DEA offers a number of advantages that make it ideal for analyzing efficiency in
Data
The data required for the analysis was obtained from the 2011/12, 2012/13 and 2013/14 academic course years, during which a survey was completed by students in the "Practical Introduction to Economics" course held during the first year of the Tourism degree program at the University of Alicante (Spain). The course is designed to familiarize students with theoretical and practical concepts of economics applied to the tourism industry.
633 students completed the survey, supposedly all of the
Results
The objective of this section is to describe and comment on the results obtained from the application of the previously described method (seen in Section 3). All of the results of the efficiency analysis were obtained using Lingo 12.0 software and these results appear in Table 3.
The first two columns of Table 3 reflect the groups (DMUs) and the academic course (Period). Then, column three indicates the efficiency values of each group having four values per cell. The first three values refer to
Conclusions
This paper examines teaching efficiency in higher education based on Data Envelopment Analysis (DEA) methodology. It has dual objectives: first, to examine the performance of students' educational process in tourism degree courses, and second, to attempt to identify the KPIs that may help to optimize the quality of the teaching process.
Based on expert and intelligent systems, the identification procedure for KPIs proposed in this study offer advances as compared to past research for a variety
References (53)
- et al.
Technical and allocative efficiency in a chain of small hotels
Hospitality Management
(2005) - et al.
Two-stage performance model for evaluating the managerial efficiency of higher education: application by the Taiwanese tourism and leisure department
Journal of Hospitality, Leisure, Sport & Tourism Education
(2012) - et al.
Measuring the efficiency of decision making units
European Journal of Operational Research
(1978) - et al.
Determining the relative efficiency of MBA programs using DEA
European Journal of Operational Research
(2000) - et al.
Economies of scope in research and teaching: a non-parametric investigation
Omega
(2013) Data Envelopment Analysis and non-discretionary inputs to select the most suitable model using multi-criteria decision analysis
Expert Systems with Applications
(2015)Measuring teaching efficiency in higher education: An application of data envelopment analysis to economics graduates from UK Universities 1993
European Journal of Operational Research
(2006)Data envelopment analysis and its application to the measurement of efficiency in higher education
Economics of Education Review
(2006)- et al.
Measuring the research performance of Chinese higher education institution using data envelopment analysis
China Economic Review
(2008) - et al.
Efficiency analysis of university departments: an empirical study
Omega
(2008)
A learning performance evaluation with benchmarking concept for English writing courses
Expert Systems with Applications
Application of data envelopment analysis on the indicators contributing to learning and teaching performance
Teaching and Teacher Education
Separating managerial inefficiency and external conditions in data envelopment analysis
European Journal of Operational Research
Evaluating alternative DEA models used to control for non-discretionary inputs
Computers & Operations Research
Education production efficiency: evidence from Brazilian universities
Economic Modelling
The efficiency of Australian universities: a data envelopment analysis
Economics of Education Review
Comparative departmental efficiency analysis within a university: a DEA approach
Procedia. Social and Behavioral Sciences
Data envelopment analysis to the Italian university system: theoretical issues and policy implications
International Journal of Business Performance Management
Measuring efficiency of higher education institutions
International Journal of Management and Decision Making
Beyond frontiers: comparing the efficiency of higher education decision-making units across more than one country
Education Economics
Evaluating the efficiency of research in academic departments
Higher Education Quarterly
A procedure for ranking efficient units in DEA
Management Science
Assessing the comparative efficiency of higher education institutions in the UK by the means of data envelopment analysis
Education Economics
Time-staged outputs in DEA
Omega
Investigating technical and scale efficiencies of Australian universities through envelopment analysis
Socio-Economics Planning Sciences
Efficiency analysis for exogenously fixed inputs and outputs
Operations Research
Cited by (43)
What causes green efficiency losses in Chinese agriculture? A perspective based on input redundancy
2023, Technological Forecasting and Social ChangeA recommendation system for effective learning strategies: An integrated approach using context-dependent DEA
2023, Expert Systems with ApplicationsCitation Excerpt :Many scholars study problems in the field of education based on the DEA method, specifically, the potency of analysing teaching efficiency with the help of DEA has been proven to be capable (Johnes et al., 2017). On the basis of previous algorithm related researches, DEA and its mutation methods, such as multistage DEA (Fuentes et al., 2016; Wolszczak-Derlacz, 2017), and fuzzy theory- related DEA (Aparicio et al., 2019; Nojavan et al., 2020), have been widely used in the current research field of education in various aspects. However, the research objects of efficiency in the field of education are mostly focused on macro individuals, such as schools, colleges or regions, and research on student individuals is lacking.
Multi-output regression using polygon generation and conditional generative adversarial networks
2022, Expert Systems with ApplicationsCitation Excerpt :A hierarchical analytical model is used in (Vlachos, 2014) to estimate retail supply chain performance. The KPIs are estimated in (Fuentes, Fuster, & Lillo-Bañuls, 2016) using a non-parametric method called multiple-stage data envelopment analysis (DEA). However, these prediction methods rely on assumptions that are heavily based on human expertise and the application at hand.
A methodological proposal to determine the optimal levels of inter-municipal cooperation in the organization of solid waste management systems
2020, Waste ManagementCitation Excerpt :Cordero-Ferrera et al. (2011) apply Muñiz’s procedure to purge the efficiency scores of a group of 97 Spanish primary care centers in the Spanish region of Extremadura from the effect of a set of purely external variables (gross birth rate, dependence rate, maternity rate, ageing rate, replace rate, population density, percentage of population working in agriculture, and percentage of population working in services). Fuentes et al. (2016) evaluate the technical efficiency of the learning-teaching process in higher education in Spain, taking into account the impact of two external variables (socio-economic and cultural level of students’ families and students’ education prior to university). While the main aim of Muñiz’s approach is that of finding significant and comparable efficiency scores, the purpose of this study is to attain a measure of the impact of the dimensional variable on those efficiency scores.
Assessment of variables determining the health management departments’ efficiency with analytical hierarchy process
2024, Health Services Management Research