Elsevier

Expert Systems with Applications

Volume 48, 15 April 2016, Pages 89-99
Expert Systems with Applications

A three-stage DEA model to evaluate learning-teaching technical efficiency: Key performance indicators and contextual variables

https://doi.org/10.1016/j.eswa.2015.11.022Get rights and content

Highlights

Abstract

This study evaluates the technical efficiency of the learning-teaching process in higher education using a three-stage procedure that offers advances in comparison to previous studies and improves the quality of the results. First, it utilizes a multiple stage Data Envelopment Analysis (DEA) with contextual variables. Second, the levels of super efficiency are calculated in order to prioritize the efficiency units. And finally, through sensitivity analysis, the contribution of each key performance indicator (KPI) is established with respect to the efficiency levels without omission of variables. The analytical data was collected from a survey completed by 633 tourism students during the 2011/12, 2012/13 and 2013/14 academic course years. The results suggest that level of satisfaction with the course, diversity of materials and satisfaction with the teacher were the most important factors affecting teaching performance. Furthermore, the effect of the contextual variables was found to be significant.

Introduction

The efficiency of university higher education is crucial to the development and growth of countries. Specifically, the production of human capital and the creation of new knowledge are fundamental factors for national economies that must compete at an international level. Therefore, studies such as this one, determining which aspects of higher education should be improved in order to achieve greater efficiency, are quite useful.

Over recent years, the growing importance of undergraduate and post graduate degree studies in tourism in Spain has justified the analysis of teaching efficiency in tourism studies (considering the fact that between the academic course years of 1988/89 and 2008/09, two and a half times the students pursued tourism degrees during a period in which, overall, diploma and degree studies decreased by approximately 25%, National Institute of Statistics [INE, in Spanish], 2010). This work focuses specifically on the tourism degree of the University of Alicante (Spain) during the 2011/12, 2012/13 and 2013/14 academic course years.

The aim of this work is first, to evaluate the efficiency of the learning-teaching process in higher education, specifically in the tourism degree and second, to select the correct indicators that permit an adequate evaluation of the performance and efficiency of education. The identification and subsequent study of the variables used to monitor the progress and success of the teaching process (Key Performance Indicators, KPIs) is a fundamental issue. According to the expert systems perspective, the methodology used in this study facilitates and improves the identification and quantification of potential improvements in terms of reduction of resources and/or improvement in academic results.

Since the work of Charnes, Cooper, and Rhodes (1978), Data Envelopment Analysis (DEA) has been widely used to analyze efficiency in diverse areas, specifically, in higher education. It is ideal for analyzing activities in sectors that require multiple resources in their production process in order to generate different types of products. Thus, DEA has become one of the most frequently used methods for determining which variables contribute to improving higher education performance (Agasisti and Dal Bianco, 2009, Johnes, 2006a, Joumady and O. Ris, 2005). DEA has enabled the assessment of the relative efficiency of the units in higher education institutions and has permitted the determination of which inputs and outputs contribute to the achievement of optimum performance.

The methodology selected for this study was implemented in three stages. First, the DEA method developed by Fried and Lovell (1996) and subsequently modified by Muñiz (2002) was applied. This method considers the contextual variables that affect the teaching process; second, super efficiency was analyzed, leading to the prioritization of the efficient units; and, finally, a sensitivity analysis was conducted to determine the contribution of each variable in terms of the efficiency level without the need to omit any variables. A significant theoretical contribution of this study is that it improves the manner in which the key variables were selected in previous studies on teaching efficiency using DEA, such as those by Montoneri, Lee, Lin, and Huang (2011, 2012), since it takes advantage of the information provided by contextual variables, super efficiency and the influence of variables (KPIs) on technical efficiency.

This study has been organized as follows: Section 2 presents a literature review in order to support the selection of the analysis model and variables. Section 3 presents the methodological model to be justified and described. The data from the study is presented in Section 4 and the results of the same are presented and discussed in Section 5. Finally, Section 6 offers our conclusions and suggests the main ideas that may be implemented in order to improve the learning-teaching efficiency analysis.

Section snippets

Literature review on efficiency in higher education

Assessing the efficiency of higher education institutions is not a simple task given that these are complex organizations having multiple inputs and outputs (Abd Aziz et al., 2013, Johnes, 2006b). Although efficiency in higher education has also been analyzed using parametric and OLS (Ordinary Least Square) regression methods (Johnes and Taylor, 1990, Zoghbi et al., 2013), ever since Johnes and Johnes (1993) the most widely used methodology have been frontier methods such as Data Envelopment

Method

Based on the reviewed studies, this section presents the methodology used in our analysis, the inputs and outputs used and an outline of our reasons for selecting them.

The literature revealed that Data Envelopment Analysis (DEA) was the most frequently used method for analyzing efficiency in the context of higher education, although other methods such as the Stochastic Frontier Analysis (SFA) have also been used. DEA offers a number of advantages that make it ideal for analyzing efficiency in

Data

The data required for the analysis was obtained from the 2011/12, 2012/13 and 2013/14 academic course years, during which a survey was completed by students in the "Practical Introduction to Economics" course held during the first year of the Tourism degree program at the University of Alicante (Spain). The course is designed to familiarize students with theoretical and practical concepts of economics applied to the tourism industry.

633 students completed the survey, supposedly all of the

Results

The objective of this section is to describe and comment on the results obtained from the application of the previously described method (seen in Section 3). All of the results of the efficiency analysis were obtained using Lingo 12.0 software and these results appear in Table 3.

The first two columns of Table 3 reflect the groups (DMUs) and the academic course (Period). Then, column three indicates the efficiency values of each group having four values per cell. The first three values refer to

Conclusions

This paper examines teaching efficiency in higher education based on Data Envelopment Analysis (DEA) methodology. It has dual objectives: first, to examine the performance of students' educational process in tourism degree courses, and second, to attempt to identify the KPIs that may help to optimize the quality of the teaching process.

Based on expert and intelligent systems, the identification procedure for KPIs proposed in this study offer advances as compared to past research for a variety

References (53)

  • MontoneriB. et al.

    A learning performance evaluation with benchmarking concept for English writing courses

    Expert Systems with Applications

    (2011)
  • MontoneriB. et al.

    Application of data envelopment analysis on the indicators contributing to learning and teaching performance

    Teaching and Teacher Education

    (2012)
  • MuñizM.A.

    Separating managerial inefficiency and external conditions in data envelopment analysis

    European Journal of Operational Research

    (2002)
  • MuñizM.A. et al.

    Evaluating alternative DEA models used to control for non-discretionary inputs

    Computers & Operations Research

    (2006)
  • ZoghbiA.C. et al.

    Education production efficiency: evidence from Brazilian universities

    Economic Modelling

    (2013)
  • AbbotM. et al.

    The efficiency of Australian universities: a data envelopment analysis

    Economics of Education Review

    (2003)
  • Abd AzizN. et al.

    Comparative departmental efficiency analysis within a university: a DEA approach

    Procedia. Social and Behavioral Sciences

    (2013)
  • AgasistiT. et al.

    Data envelopment analysis to the Italian university system: theoretical issues and policy implications

    International Journal of Business Performance Management

    (2006)
  • AgasistiT. et al.

    Measuring efficiency of higher education institutions

    International Journal of Management and Decision Making

    (2009)
  • AgasistiT. et al.

    Beyond frontiers: comparing the efficiency of higher education decision-making units across more than one country

    Education Economics

    (2009)
  • AgasistiT. et al.

    Evaluating the efficiency of research in academic departments

    Higher Education Quarterly

    (2011)
  • AndersenP. et al.

    A procedure for ranking efficient units in DEA

    Management Science

    (1993)
  • AthanassopoulosA.D. et al.

    Assessing the comparative efficiency of higher education institutions in the UK by the means of data envelopment analysis

    Education Economics

    (1997)
  • AvilésS. et al.

    Time-staged outputs in DEA

    Omega

    (2015)
  • AvrikanN.

    Investigating technical and scale efficiencies of Australian universities through envelopment analysis

    Socio-Economics Planning Sciences

    (2001)
  • BankerR. et al.

    Efficiency analysis for exogenously fixed inputs and outputs

    Operations Research

    (1986)
  • Cited by (43)

    • A recommendation system for effective learning strategies: An integrated approach using context-dependent DEA

      2023, Expert Systems with Applications
      Citation Excerpt :

      Many scholars study problems in the field of education based on the DEA method, specifically, the potency of analysing teaching efficiency with the help of DEA has been proven to be capable (Johnes et al., 2017). On the basis of previous algorithm related researches, DEA and its mutation methods, such as multistage DEA (Fuentes et al., 2016; Wolszczak-Derlacz, 2017), and fuzzy theory- related DEA (Aparicio et al., 2019; Nojavan et al., 2020), have been widely used in the current research field of education in various aspects. However, the research objects of efficiency in the field of education are mostly focused on macro individuals, such as schools, colleges or regions, and research on student individuals is lacking.

    • Multi-output regression using polygon generation and conditional generative adversarial networks

      2022, Expert Systems with Applications
      Citation Excerpt :

      A hierarchical analytical model is used in (Vlachos, 2014) to estimate retail supply chain performance. The KPIs are estimated in (Fuentes, Fuster, & Lillo-Bañuls, 2016) using a non-parametric method called multiple-stage data envelopment analysis (DEA). However, these prediction methods rely on assumptions that are heavily based on human expertise and the application at hand.

    • A methodological proposal to determine the optimal levels of inter-municipal cooperation in the organization of solid waste management systems

      2020, Waste Management
      Citation Excerpt :

      Cordero-Ferrera et al. (2011) apply Muñiz’s procedure to purge the efficiency scores of a group of 97 Spanish primary care centers in the Spanish region of Extremadura from the effect of a set of purely external variables (gross birth rate, dependence rate, maternity rate, ageing rate, replace rate, population density, percentage of population working in agriculture, and percentage of population working in services). Fuentes et al. (2016) evaluate the technical efficiency of the learning-teaching process in higher education in Spain, taking into account the impact of two external variables (socio-economic and cultural level of students’ families and students’ education prior to university). While the main aim of Muñiz’s approach is that of finding significant and comparable efficiency scores, the purpose of this study is to attain a measure of the impact of the dimensional variable on those efficiency scores.

    View all citing articles on Scopus
    1

    Tel./fax: +34 965 90 97 11.

    2

    Tel.: +34 965 90 34 00x2712.

    View full text