Abstract
Education and training interventions can be evaluated through the success of learning outcomes. Kirkpatrick's four-level model is a widely accepted and highly popular evaluation tool. However, some criticise the model's shortcomings. This article will examine the extent to which the four-level model can evaluate design and technology students' learning about aesthetics after an intervention by reporting our use of an augmented version of the four-level model. We examine the results in terms of students' reaction to the intervention, their long-term learning and their behaviour changes by studying their visual analyses and drawings through segment codes. We found that, in order to uncover the obscurities imbedded in aesthetics and to explicate the complexities, we could not use the four-level-model on its own, but had to revert to a more augmented version.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Market driven approach: aesthetic choices are quite often based on and motivated by market surveys, studies of competing products, and studies of defined target user groups, in order to make the product fit a favourable niche among other products (Parr 2004).
References
Alliger, G. M., Tannenbaum, S. I., Bennett, W., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50(1), 341–358.
Anderson, E. (1998). Enhancing Visual Literacy through Cognitive Activities. (Paper presented at the 2002 ASEE/SEF/TUB Colloquium, Glasgow).
Arnheim, R. (1954). Art and visual perception: a psychology of the creative eye. Berkeley, CA: University of California Press.
Arnheim, R. (1986). New essays on the psychology of art. Berkeley: University of California Press.
Arthur, W., Bennet, W., Edens, P. S., & Bell, S. T. (2003). Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology, 88(2), 234–245.
Bates, R. (2004). A critical analysis of evaluation practice: The Kirkpatrick Model and the principle of beneficence. Evaluation and Program Planning, 27, 341–347.
Bates, R. (2005). Kirkpatrick Four-level Evaluation Model. In S. Mattheson (Ed.), Encyclopedia of Evaluation (pp. 221–222). London: SAGE.
Clark, D. (2004). Evaluation phase–instructional systems. Retrieved 14 September, 2004, from www.nwlink.com/∼donclark/hrd/sat6.html
Clementz, A. R. (2002). Program level evaluation: Using kirkpatrick’s four levels of evaluation to conduct systemic evaluation of undergraduate college programs. 24 September 2003, from http://web.bryant.edu/∼assess/Program_Level_Evaluation.doc
Cohen, L., Manion, K., & Morrison, K. (2000). Research Methods in Education (5 ed.). London: Routledge & Farmer.
De Villiers, M. R. (2002). The dynamics of theory and practice in instructional systems design. Thesis, University of Pretoria.
De Vries, M. J. (2005). Teaching about technology. An introduction to the philosophy of technology for non-philosophers (Vol. 27). Dordrecht: Springer.
Department of Education. (2002). National curriculum statement for grades R–9 for technology. Pretoria: Department of Education.
Department of Education. (2003). White paper on e-Education. Pretoria: Department of Education.
Faber O. (1975). Aesthetic aspect of civil engineering design. In Institution of Civil Engineers (Ed.), Aesthetic Aspect of Civil Engineering Design (pp. 1–15). London: Institution of Civil Engineers.
Garner, S. W. (1993). The importance of graphic modelling in design activity in teaching and learning technology. In R. McCormick, P. Murphy & M. Harrison (Eds.). Workingham: Addison-Wesley Publishing Company.
Garratt, J. (1996). Design and technology (2 ed.). Cambridge: Cambridge University Press.
Goel, V. (1995). How sketches work. A bradford book. Cambridge: MIT Press.
Hannafin, M. J., & Peck, K. L. (1988). The design, development and evaluation of instructional software. New York: Macmillan.
Holden C. (1975). Aesthetic aspect of civil engineering design. In Institution of Civil Engineers (Ed.), Lecture at Seminar for Institution of Civil Engineers (pp. 41–50). London: Institution of Civil Engineers.
Holton-III, E. F. (1996). The flawed four-level evaluation model. Human Resource Development Quarterly, 7(1), 5–21.
Inglis, R. (1975). Aesthetic aspect of civil engineering design. London: Institution of Civil Engineers.
Kirkpatrick, D. L. (1994). Evaluating training programs: The four levels. San Francisco: Berrett Koehler.
Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating training programs: The four levels (3rd ed.). San Francisco: Berrett-Koehler Publishing.
Lauer, D. A. (1985). Design Basics (2nd ed.). New York: Holt, Rinehart and Winston.
McMillan, J. H., & Schumacher, S. (2001). Research in education: A conceptual introduction (5th ed.). London: Longman Inc.
Newstrom, J. W. (1995). Review of evaluation training programs: The four levels by D.L. Kirkpatrick. Human Resource Development Quarterly, 6(1), 317–319.
Osman, M. E., & Hannafin, M. J. (1992). Metacognition research and theory: Analysis and implications for instruction design. Educational Technology Research & Development, 40(2), 83–99.
Parr, J. W. (2004). Aesthetic intentions in product design market driven or alternative form. Retrieved 4 September, 2004, from http://folk.uio.no/parr/text/aesthetic.pdf.
Patton, M. (2002). Two decades of developments in qualitative inquiry. Qualitative Social Work, 1(3), 261–283.
Press, M., & Cooper, R. (2002). The design experience. Burlington: Ashgate.
Russ-Eft, D. F., & Preskill, H. S. (2005). Search of the Holy Grail: Return on investment evaluation in human resources development [Electronic Version]. Advances in Developing Human Resources, 7, 1:71–85. Retrieved 5 February 2007 from http://adh.sagepub.com/cgi/content/abstract/7/1/71.
Tversky, B. (1999). What does drawing reveal about thinking? Retrieved 20 January, 2005, from http://www.arch.usyd.edu.au/kcdc/books/VR99?Tversky.html.
Van den Akker, J. (1999). Principles and methods of development research. In J. Van den Akker, N. Nieveen, R. M. Branch, K. L. Gustafson & T. Plomp (Eds.), Design methodology and developmental research in education and training (pp. 1–14). The Netherlands: Kluwer Academic Publishers.
Winfrey, E. C. (2002). Kirkpatrick’s four levels of evaluation. Retrieved 24 September, 2003, from http://coe.sdsu.edu/eet/Articles/k4levels/index.htm.
Wong, W. (1993). Principles of form and design. New York: Van Nostrand Reinhold.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Haupt, G., Blignaut, S. Uncovering learning outcomes: explicating obscurity in learning of aesthetics in design and technology education. Int J Technol Des Educ 18, 361–374 (2008). https://doi.org/10.1007/s10798-007-9029-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10798-007-9029-1