Abstract
Student evaluation of university teaching activity is now compulsory in Italy and a research group of the ItalianMinistry of Instruction, University, and Research proposed a questionnaire with items based on the four-point Likert scale and a traditional item-by-item analysis for the evaluation of classrooms, work load, course organization, lectures, and teaching aids. Three split-ballot experiments were carried out to test the differences between the four-point and five-point Likert scale. The traditional analysis is compared with the results of the fuzzy system set up to achieve the same purposes. The fuzzy system yielded scores that proved to be generally higher but sometimes also lower than those obtained using the five- or four-point Likert scale. Furthermore, an extension of standard procedures of the fuzzy system is suggested to obtain a fuzzy item-by-item analysis, thereby increasing the possibility of their use in social sciences.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Albanese, M., Prucha, C., Carnet, J. H. & Gjerde, C.L. (1977). The effect of right or left placement of the positive response on Likert-type scales used by medical students for rating instruction. Academic Medicine 72: 627–630.
Albaum, G. (1997). The Likert scale revisited: an alternate version. Journal of the Market Research Society 39(2): 331–348.
Bojadziev, G. & Bojadziev, M. (1997). Fuzzy Logic for Business, Finance and Management. Singapore: World Scientific.
Brody, C. J. & Dietz, J. (1997). On the dimensionality of two-question format Likert attitude scales. Social Science Research 26(2): 197–204.
Cacciola, S. & Marradi, A. (1988). Contributo al dibattito sulle scale Likert basato sull’analisi di interviste registrate. In: Marradi, A. (ed.), Costruire il Dato. Sulle Tecniche di Raccolta delle Informazioni nelle Scienze Sociali. Milan: Franco Angeli, pp. 63–102.
Questionario di base da utilizzare per l’attuazione di un programma per la valutazione della didattica da parte degli studenti. Rapporto finale del gruppo di Ricerca (RdR 1/00): MURST, Osservatorio per la valutazione del sistema universitario ( http://www.cnvsu.it).
Chan, J. C. (1991). Response-order Effects in Likert-type Scales. Educational Psychology Measurements 51: 531–540.
Coombs, C. H. (1953). Theory and method of social measurement. In: L. Festiger & D. Katz (eds), Research Methods in the Behavioral Sciences. New York: Dryden, pp. 471–535.
Cox, E. P. (1980). The optimal number of response alternatives for a scale: A review. Journal of Marketing Research 17: 407–422.
Dimitrov, V. & Kopra, K. (1998). Management of social complexity. In: L. Reznik, V. Dimitrov & J. Kacprzyk (eds), Fuzzy Systems Design: Social and Engineering Applications. Heidelberg: Physica-Verlag, pp. 117–131.
Dubois, D. & Prade, H. (eds.) (2000). Fundamentals of Fuzzy Sets. Boston: Kluwer Academic.
Duncan, O. D. & Stenbeck, M. (1987). Are Likert scales unidimensional? Social Science Research 16: 245–259.
Edvardsson, B. & Linden, M. (1976). A method for evaluation of metric properties of response scales, Quality & Quantity 10(3): 241–249.
Garland, R. (1991). The mid-point on a rating scale: Is it desirable? Marketing Bulletin 2: 66–70.
Guy, R. F. & Norvell, M. (1977). The neutral point on a Likert scale. The Journal of Psychology 95: 199–204.
Hand, D. J. & Crowder, M. J. (1996). Practical Longitudinal Data Analysis. London: Chapman & Hall.
Hofacker, C. F. (1984). Categorical judgment scaling with ordinal assumptions. Multivariate Behavioral Research 19(1): 91–106.
Kalton, J., Roberts, J. & Holt, D. (1980). The effect of offering a middle response option with opinion questions. The Statistician 29(1): 65–78.
Kasabov, N. K. (1996). Foundations of Neural Networks, Fuzzy Systems, and Knowledge Engineering. Cambridge, MA: MIT Press.
Lalla, M., Facchinetti, G. & Mastroleo, G. (2001). A fuzzy expert system for evaluating teaching efficiency. In: N. Reinhard, (2003), Modeling and Control of Economic System 2002, Oxford: Elsevier (Pergamon Press), pp. 395–400.
Laughlin, C. D. (1993). Fuzziness and phenomenology in ethnological research: Insights from fuzzy set theory. Journal of Anthropological Research 49(1): 17–37.
Marradi, A. (1992). L’analisi Monovariata. Milan: Franco Angeli.
Marradi, A. (1998). Termometri con vincolo di ordinabilità: il “gioco della torre” consente di aggirare la tendenza alla desiderabilità sociale? Sociologia e Ricerca Sociale 57: 49–59.
Matell, M. S. & Jacoby, J. (1971). Is there an optimal number of alternatives for Likert scale items? Study 1: Reliability and validity. Educational and Psychological Measurement 31: 657–674.
Matell, M. S. & Jacoby, J. (1972). Is there an optimal number of alternatives for Likert scale items? Effects of testing time and scale properties. Journal of Applied Psychology 56(6): 506–509.
Orvik, J. M. (1972). Social desirability for individual, his group, and society. Multivariate Behavioral Research 7: 3–32.
Pigalle, F. (1998). Evaluation stratégique des politiques publiques: Une approche par la logique floue. Revue d’Economie Régionale et Urbaine (3): 461–480.
Ragin, C. C. (2000). Fuzzy-Set Social Science. Chicago: University of Chicago Press.
Ray, J. J. (1990). Acquiescence and problems with forced-choice scales. Journal of Social Psychology 130 (3): 397–399.
Royes, G. F. & Bastos, R. C. (2001). Fuzzy sets in political science. 0-7803-7078-3/ 01/$10.00 (C)2001 IEEE: 935–940.
Russel, J. & Bullock, M. (1986). Fuzzy concepts and the perception of emotion in facial expressions. Social Cognition 4: 309–341.
Ryan, M. (1980). The Likert scale’s midpoint in communications research. Journalism Quarterly 57(2): 305–313.
San Luis Costas, C., Prieto Maranon, P. & Hernandez Cabrera, J. A. (1994). Application of diffuse measurement to the evaluation of psychological structures. Quality & Quantity 28(3): 305–313.
Schuman, H. & Presser, S. (1996). Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context. Thousand Oaks, CA: Sage Publications.
Smithson, M. J. (1988). Fuzzy set theory and the social science: the scope for applications. Fuzzy Sets and Systems 26: 1–21.
Takayanagi, S. & Cliff, N. (2000). An examination of graduate students’ statistical judgments: statistical and fuzzy set approaches. Psychological Reports 86: 243–259.
von Altrock, C. (1997). Fuzzy Logic and Neurofuzzy Applications in Business and Finance. New York: Prentice Hall.
Yamashita, T. (1997). On a support system for human decision making by the combination of fuzzy reasoning and fuzzy structural modeling. Fuzzy Sets and Systems 87: 257–263.
Wildt, A. R. & Mazis, M. B. (1978). Determinants of scale response: Label versus position. Journal of Marketing Research 15: 261–267.
Zadeh, L. A. (1965). Fuzzy sets. Information and Control 8: 338–353.
Zimmermann, H. J. (1996). Fuzzy Set Theory and Its Applications, 3rd edition. Boston: Kluwer Academic.
Author information
Authors and Affiliations
Additional information
This study is a part of the project “Methods and technology for innovating and re-organising teaching activity” supported by the University of Modena and Reggio Emilia.
Rights and permissions
About this article
Cite this article
Lalla, M., Facchinetti, G. & Mastroleo, G. Ordinal scales and fuzzy set systems to measure agreement: An application to the evaluation of teaching activity. Qual Quant 38, 577–601 (2005). https://doi.org/10.1007/s11135-005-8103-6
Issue Date:
DOI: https://doi.org/10.1007/s11135-005-8103-6