Abstract
Given medical education’s longstanding emphasis on assessment, it seems prudent to evaluate whether our current research and development focus on testing makes sense. Since any intervention within medical education must ultimately be evaluated based upon its impact on student learning, this report seeks to provide a quantitative accounting of the learning gains attained through educational assessments. To approach this question, we estimate achieved learning within a medical school environment that optimally utilizes educational assessments. We compare this estimate to learning that might be expected in a medical school that employs no educational assessments. Effect sizes are used to estimate testing’s total impact on learning by summarizing three effects; the direct effect, the indirect effect, and the selection effect. The literature is far from complete, but the available evidence strongly suggests that each of these effects is large and the net cumulative impact on learning in medical education is over two standard deviations. While additional evidence is required, the current literature shows that testing within medical education makes a strong positive contribution to learning.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bekkink, M. O., Donders, R., van Muijen, G. N. P., & Ruiter, D. J. (2012). Challenging medical students with an interim assessment: A positive effect on formal examination score in a randomized controlled study. Advances in Health Sciences Education, 17, 27–37.
Cohen, J. (1973). Eta-squared and partial eta-squared in fixed factor ANOVA designs. Educational and Psychological Measurement, 33, 107–112.
Cohen, J. (1988). Statistical power for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
Cooper, H. M. (1982). Scientific guidelines for conducting integrative research reviews. Review of Educational Research, 52, 291–302.
Donnon, T., Paolucci, E. O., & Violato, C. (2007). The predictive validity of the MCAT for medical school performance and medical board licensing examinations: A meta-analysis of the published research. Academic Medicine, 82, 100–106.
Glover, J. A. (1989). The “testing” phenomenon: not gone but nearly forgotten. Journal of Educational Psychology, 81, 392–399.
Halpin, G., & Halpin, G. (1982). Experimental investigation of the effects of study and testing on student learning, retention, and rating of instruction. Journal of Educational Psychology, 74, 32–38.
Julian, E. R. (2005). Validity of the Medical College Admission Test for predicting medical school performance. Academic Medicine, 80, 910–917.
Kang, S. H. K., McDermott, K. B., & Roediger, H. L. (2007). Test format and corrective feedback modify the effect of testing on long-term retention. European Journal of Cognitive Psychology, 19, 528–558.
Kreiter, C. D., & Kreiter, Y. (2007). A validity generalization perspective on the ability of undergraduate GPA and the Medical College Admission Test to predict important outcomes. Teaching and Learning in Medicine, 19, 95–100.
Kromann, C. B., Bohnstedt, C., Jensen, M. L., & Ringsted, C. (2010). The testing effect on skill learning might last 6 months. Advances in Health Science Education Theory and Practice, 15, 395–401.
Kromann, C. B., Jensen, M. L., & Ringsted, C. (2009). The effect of testing on skills learning. Medical Education, 43, 21–27.
Kulik, J. A., & Kulik, C. C. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58, 79–97.
Larsen, D. P., Butler, A. C., & Roediger, H. L. I. I. I. (2009). Repeated testing improves long-term retention relative to repeated study: A randomized controlled trial. Medical Education, 43, 1174–1181.
Levine, T. R., & Hullett, C. R. (2002). Eta squared, partial eta squared, and misreporting of effect size communication research. Human Communication Research, 28, 612–625.
McDaniel, M. A., Anderson, J. L., Derbish, M. H., & Morrisette, N. (2007). Testing the testing effect in the classroom. European Journal of Cognitive Psychology, 19, 494–513.
McDaniel, M. A., & Fisher, R. P. (1991). Tests and test feedback as learning sources. Contemporary Educational Psychology, 16, 192–201.
Norman, G., & Eva, K. W. (2008). Quantitative research methods in medical education. Edinburgh: Association for the Study of Medical Education.
Phelps, R. P. (2011). The effect of testing on achievement: Meta-analyses and research summary, 1910–2010: Source list, effect sizes, and references for quantitative studies. Nonpartisan Education Review/Resources. Retrieved September, 2011 from. http://npe.educationnews.org/Review/Resources/QuantitativeList.htm.
Reibnegger, G., Caluba, H. C., Ithaler, D., Manhal, S., Neges, H. N., & Smolle, J. (2010). Progress of medical students after open admission or admission based on knowledge tests. Medical Education, 44, 205–214.
Robinson, P. (1972). Contingent systems of instruction, ED069704 (TM002156). Paper presented at the Rocky Mountain Psychological Association Convention.
Roediger, H. L., & Karpicke, J. D. (2006). Test enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17, 249–255.
Sechrest, L., & Yeaton, W. H. (1982). Magnitude of experimental effects in social science research. Evaluation Review, 6, 579–600.
Conflict of interest
The authors report no conflict of interests.
Author information
Authors and Affiliations
Corresponding author
Additional information
Research conducted as part of the Visiting Professorship sponsored by International Research Center for Medical Education (IRCME), University of Tokyo
Rights and permissions
About this article
Cite this article
Kreiter, C.D., Green, J., Lenoch, S. et al. The overall impact of testing on medical student learning: quantitative estimation of consequential validity. Adv in Health Sci Educ 18, 835–844 (2013). https://doi.org/10.1007/s10459-012-9395-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-012-9395-7