Abstract
Problem-based learning (PBL) is an instructional approach used in medical education that is characterized by solving problems in small groups with tutor guidance. More than 50 years since PBL’s inception, many questions remain to be addressed about its processes and learning outcomes. The purpose of the study was to examine the bibliometric characteristics of the 100 most cited articles on PBL in medical education and to identify landmark papers that have made significant contributions to PBL research. Results were systematically reviewed for citation frequency, publication year, journal, article type, article focus, authors, author collaboration, and country collaboration. The number of citations ranged from 81 to 3531 times cited with 31,041 total citations. The articles were contributed by 211 authors in 23 journals and most articles (68%) were published in Medical Education, Academic Medicine, and Medical Teacher. The majority of the articles (71%) originated from Netherlands, Canada, and the United States and six prolific authors were identified. Almost half of the articles are classified as empirical research. Article foci included theoretical foundations of PBL, curriculum design, learning outcomes and processes, tutors, assessment, guides to PBL implementation, commentaries, and student well-being. The strong author and country collaborations indicate continued global interest in the PBL instructional method, which is likely to remain an active topic of research as the evidence of its effectiveness over traditional instructional methods as well as its most impactful components is inconclusive.
Similar content being viewed by others
Data availability
The data that support the findings of this study are available from the corresponding author upon request.
References
Jonas HS, Etzel SI, Barzansky B. Educational programs in US medical schools. JAMA J Am Med Assoc. 1992;268(9):1083–90.
Kinkade S. A snapshot of the status of problem-based learning in U. S. medical schools, 2003–04. Acad Med. 2005;80(3):300–1.
Association of American Medical Colleges. Teaching formats by topic in required courses/clerkships [Internet]. Washington (DC): Association of American Medical Colleges; [cited 14 Dec 2022]. Available from: https://www.aamc.org/data-reports/curriculum-reports/interactive-data/teaching-formats-topic-required-courses/clerkships
Albanese M. Problem-based learning: why curricula are likely to show little effect on knowledge and clinical skills. Med Educ. 2000;34(9):729–38.
Dolmans DHJM, Loyens SMM, Marcq H, Gijbels D. Deep and surface learning in problem-based learning: a review of the literature. Adv Heal Sci Educ. 2016;21(5):1087–112.
Gijbels D, Dochy F, Van Den Bossche P, Segers M. Effects of problem-based learning: a meta-analysis from the angle of assessment. Rev Educ Res. 2005;75(1):27–61.
Kirschner PA, Sweller J, Clark RE. Why minimal guidance during instruction does not work: an analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educ Psychol. 2006;41(2):75–86.
Loyens SMM, Magda J, Rikers RMJP. Self-directed learning in problem-based learning and its relationships with self-regulated learning. Educ Psychol Rev. 2008;20(4):411–27.
Miflin BM, Campbell CB, Price DA. A conceptual framework to guide the development of self-directed, lifelong learning in problem-based medical curricula. Med Educ. 2000;34(4):299–306.
Norman GR, Schmidt HG. The psychological basis of problem-based learning: a review of the evidence. Acad Med. 1992;67(9):557–65.
Schmidt HG. Problem-based learning: rationale and description. Med Educ. 1983;17(1):11–6.
Schmidt HG. Foundations of problem-based learning: some explanatory notes. Med Educ. 1993;27(5):422–32.
Barrows HS. Problem-based learning in medicine and beyond: a brief overview. New Dir Teach Learn. 1996;1996(68):3–12.
Neufeld VR, Barrows HS. The mcmaster philosophy: an approach to medical education. J Med Educ. 1974;49(11):1040–50.
Barrows HS, Tamblyn RM. Problem-based learning: an approach to medical education. Springer Publishing Company. 1980.
Dolmans DHJM, De Grave W, Wolfhagen IHAP, van der Vleuten CPM. Problem-based learning: future challenges for educational practice and research. Med Educ. 2005;39(7):732–41.
Sampson M, Horsley T, Doja A. A bibliometric analysis of evaluative medical education studies: characteristics and indexing accuracy. Acad Med. 2013;88(3):421–7.
Azer SA. The top-cited articles in medical education: a bibliometric analysis. Acad Med. 2015;90(8):1147–61.
Azer SA, Azer S. Top-cited articles in medical professionalism: a bibliometric analysis versus altmetric scores. BMJ Open. 2019;9(7).
Young M, St-Onge C, Xiao J, Vachon Lachiver E, Torabi N. Characterizing the literature on validity and assessment in medical education: a bibliometric study. Perspect Med Educ. 2018;7(3):182–91.
Maggio LA, Costello JA, Norton C, Driessen EW, Artino AR Jr. Knowledge syntheses in medical education: a bibliometric analysis. Perspect Med Educ. 2021;10(2):79–87.
Maggio LA, Haustein S, Costello JA, Driessen EW, Artino AR Jr. Joining the meta-research movement: a bibliometric case study of the journal Perspectives on Medical Education. Perspect Med Educ. 2022;11(3):127–36.
Maggio LA, Ninkov A, Costello JA, Driessen EW, Artino AR. Knowledge syntheses in medical education: meta-research examining author gender, geographic location, and institutional affiliation. PLoS One. 2021;16(10 October).
Azer SA. Top-cited articles in problem-based learning: a bibliometric analysis and quality of evidence assessment. J Dent Educ. 2017;81(4):458–78.
Broadus RN. Toward a definition of “bibliometrics.” Scientometrics. 1987;12(5–6):373–9.
Okubo Y. Bibliometric indicators and analysis of research systems: methods and examples. Paris. 1997.
Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26(2):91–108.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.
Ioannidis JPA, Boyack K, Wouters PF. Citation metrics: a primer on how (not) to normalize. PLoS Biol. 2016;14(9).
Hmelo-Silver CE, Duncan RG, Chinn CA. Scaffolding and achievement in problem-based and inquiry learning: a response to Kirschner, Sweller, and Clark (2006). Educ Psychol. 2007;42(2):99–107.
Albanese MA, Mitchell S. Problem-based learning: a review of literature on its outcomes and implementation issues. Acad Med. 1993;68(1):52–81.
Dochy F, Segers M, Van den Bossche P, Gijbels D. Effects of problem-based learning: a meta-analysis. Learn Instr. 2003;13(5):533–68.
Hmelo-Silver CE. Problem-based learning: what and how do students learn? Educ Psychol Rev. 2004;16(3):235–66.
Colliver JA. Effectiveness of problem-based learning curricula: research and theory. Acad Med. 2000;75(3):259–66.
Wood DF. Problem based learning. BMJ. 2003;326(7384):328.
Spencer JA, Jordan RK. Learner centred approaches in medical education. Br Med J. 1999;318(7193):1280–3.
Norman GR, Schmidt HG. Effectiveness of problem-based learning curricula: theory, practice and paper darts. Med Educ. 2000;34(9):721–8.
Barrows HS. A taxonomy of problem-based learning methods. Med Educ. 1986;20(6):481–6.
Vernon DTA, Blake RL. Does problem-based learning work? A meta-analysis of evaluative research. Acad Med. 1993;68(7):550–63.
Schmidt HG, Moust JHC. What makes a tutor effective? A structural-equations modeling approach to learning in problem-based curricula. Acad Med. 1995;70(8):708–14.
Hmelo CE. Problem-based learning: effects on the early acquisition of cognitive skill in medicine. J Learn Sci. 1998;7(2):173–208.
Koh GC-H, Khoo HE, Wong ML, Koh D. The effects of problem-based learning during medical school on physician competency: a systematic review. C Can Med Assoc J. 2008;178(1):34–41.
Maudsley G. Do we all mean the same thing by “problem-based learning”? A review of the concepts and a formulation of the ground rules. Acad Med. 1999;74(2):178–85.
Koschmann TD, Myers AC, Feltovich PJ, Barrows HS. Using technology to assist in realizing effective learning and instruction: a principled approach to the use of computers in collaborative learning. J Learn Sci. 1994;3(3):227–64.
Neville AJ. Problem-based learning and medical education forty years on: a review of its effects on knowledge and clinical performance. Med Princ Pract. 2008;18(1):1–9.
Smits PBA, Verbeek JHAM, de Buisonjé CD. Problem based learning in continuing medical education: a review of controlled evaluation studies. Br Med J. 2002;324(7330):153–6.
Patel VL, Groen GJ, Norman GR. Effects of conventional and problem-based medical curricula on problem solving. Acad Med. 1991;66(7):380–9.
Mandin H, Jones A, Woloschuk W, Harasym P. Helping students learn to think like experts when solving clinical problems. Acad Med. 1997;72(3):173–9.
Maudsley G, Strivens J. Promoting professional knowledge, experiential learning and critical thinking for medical students. Med Educ. 2000;34(7):535–44.
Walton HJ, Matthews MB. Essentials of problem-based learning. Med Educ. 1989;23(6):542–58.
Dolmans DHJM, Gijselaers WH, Moust JHC, De Grave WS, Wolfhagen IHAP, van der Vleuten CPM. Trends in research on the tutor in problem-based learning: conclusions and implications for educational practice and research. Med Teach. 2002;24(2):173–80.
Dolmans DHJM, Wolfhagen IHAP, van der Vleuten CPM, Wijnen WHFW. Solving problems with group work in problem-based learning: Hold on to the philosophy. Med Educ. 2001;35(9):884–9.
Singh SP, Baxter H, Standen P, Duggan C. Changing the attitudes of “tomorrow’s doctors” towards mental illness and psychiatry: a comparison of two teaching methods. Med Educ. 1998;32(2):115–20.
Finucane PM, Johnson SM, Prideaux DJ. Problem-based learning: its rationale and efficacy. Med J Aust. 1998;168(9):445–8.
Schmidt HG, Rotgans JI, Yew EH. The process of problem-based learning: what works and why. Med Educ. 2011;45(8):792–806.
Hmelo-Silver CE, Barrows HS. Facilitating collaborative knowledge building. Cogn Instr. 2008;26(1):48–94.
Schmidt HG, van der Molen HT, te Winkel WWR, Wijnen WHFW. Constructivist, problem-based learning does work: a meta-analysis of curricular comparisons involving a single medical school. Educ Psychol. 2009;44(4):227–49.
Norman GR. Problem-solving skills, solving problems and problem-based learning. Med Educ. 1988;22(4):279–86.
Davis MH, Harden RM. AMEE Medical Education Guide No. 15: Problem-based learning: a practical guide. Med Teach. 1999;21(2):130–40.
Schmidt HG, Dauphinee WD, Patel VL. Comparing the effects of problem-based and conventional curricula in an international sample. J Med Educ. 1987;62(4):305–15.
Coles CR. Differences between conventional and problem-based curricula in their students’ approaches to studying. Med Educ. 1985;19(4):308–9.
Neufeld VR, Norman GR, Feightner JW, Barows HS. Clinical problem-solving by medical students: a cross-sectional and longitudinal analysis. Med Educ. 1981;15(5):315–22.
Frambach JM, Driessen EW, Chan L-C, van der Vleuten CPM. Rethinking the globalisation of problem-based learning: how culture challenges self-directed learning. Med Educ. 2012;46(8):738–47.
Hartling L, Spooner C, Tjosvold L, Oswald A. Problem-based learning in pre-clinical medical education: 22 years of outcome research. Med Teach. 2010;32(1):28–35.
Dolmans DHJM, Snellen-Balendong H, Wolfhagen IHAP, van der Vleuten CPM. Seven principles of effective case design for a problem-based curriculum. Med Teach. 1997;19(3):185–9.
Kaufman A, Mennin S, Waterman R, Duban S, Hansbarger C, Silverblatt H, et al. The New Mexico experiment: educational innovation and institutional change. J Med Educ. 1989;64(6):285–94.
Baxter H, Singh SP, Standen P, Duggan C. The attitudes of “tomorrow’s doctors” towards mental illness and psychiatry: changes during the final undergraduate year. Med Educ. 2001;35(4):381–3.
Dolmans DHJM, Schmidt HG. What drives the student in problem-based learning? Med Educ. 1994;28(5):372–80.
Srinivasan M, Wilkes M, Stevenson F, Nguyen T, Slavin S. Comparing problem-based learning with case-based learning: effects of a major curricular shift at two institutions. Acad Med. 2007;82(1):74–82.
Woltering V, Herrler A, Spitzer K, Spreckelsen C. Blended learning positively affects students’ satisfaction and the role of the tutor in the problem-based learning process: results of a mixed-method evaluation. Adv Heal Sci Educ. 2009;14(5):725–38.
Antepohl W, Herzig S. Problem-based learning versus lecture-based learning in a course of basic pharmacology: a controlled, randomized study. Med Educ. 1999;33(2):106–13.
Maudsley G. Roles and responsibilities of the problem based learning tutor in the undergraduate medical curriculum. Br Med J. 1999;318(7184):657–61.
Silver M, Wilkerson L. Effects of tutors with subject expertise on the problem-based tutorial process. Acad Med. 1991;66(5):298–300.
Thomas RE. Problem-based learning: measurable outcomes. Med Educ. 1997;31(5):320–9.
Charlin B, Mann K, Hansen P. The many faces of problem-based learning: a framework for understanding and comparison. Med Teach. 1998;20(4):323–30.
Schmidt HG. Resolving inconsistencies in tutor expertise research: does lack of structure cause students to seek tutor guidance? Acad Med. 1994;69(8):656–62.
Schmidt HG, Loyens SMM, Van Gog T, Paas F. Problem-based learning is compatible with human cognitive architecture: commentary on Kirschner, Sweller, and Clark (2006). Educ Psychol. 2007;42(2):91–7.
van der Vleuten CPM, Verwijnen GM, Wijnen WHFW. Fifteen years of experience with progress testing in a problem-based learning curriculum. Med Teach. 1996;18(2):103–9.
Taradi SK, Taradi M, Radić K, Pokrajac N. Blending problem-based learning with Web technology positively impacts student learning outcomes in acid-base physiology. Am J Physiol - Adv Physiol Educ. 2005;29(1):35–9.
Jones A, McArdle PJ, O’Neill PA. Perceptions of how well graduates are prepared for the role of pre-registration house of(r)cer: a comparison of outcomes from a traditional and an integrated PBL curriculum. Med Educ. 2002;36(1):16–25.
Hill J, Rolfe IE, Pearson S-A, Heathcote A. Do junior doctors feel they are prepared for hospital practice? A study of graduates from traditional and non-traditional medical schools. Med Educ. 1998;32(1):19–24.
Berkson L. Problem-based learning: have the expectations been met? Acad Med. 1993;68(10):S79-88.
Schmidt HG, Machiels-Bongaerts M, Hermans H, Ten Cate TJ, Venekamp R, Boshuizen HPA. The development of diagnostic competence: comparison of a problem-based, an integrated, and a conventional medical curriculum. Acad Med. 1996;71(6):658–64.
De Grave WS, Dolmans DHJM, van der Vleuten CPM. Profiles of effective tutors in problem-based learning: scaffolding student learning. Med Educ. 1999;33(12):901–6.
Kelly PA, Haidet P, Schneider V, Searle N, Seidel CL, Richards BF. A comparison of in-class learner engagement across lecture, problem-based learning, and team learning using the STROBE classroom observation tool. Teach Learn Med. 2005;17(2):112–8.
Nandi PL, Chan JN, Chan CP, Chan P, Chan LP. Undergraduate medical education: comparison of problem-based learning and conventional teaching. Hong Kong Med J. 2000;6(3):301–6.
Prince KJAH, Van Eijs PWLJ, Boshuizen HPA, van der Vleuten CPM, Scherpbier AJJA. General competencies of problem-based learning (PBL) and non-PBL graduates. Med Educ. 2005;39(4):394–401.
Kassab SE, Abu-Hijleh MF, Al-Shboul Q, Hamdy H. Student-led tutorials in problem-based learning: educational outcomes and students’ perceptions. Med Teach. 2005;27(6):521–6.
Bernstein P, Tipping J, Bercovitz K, Skinner HA. Shifting students and faculty to a PBL curriculum: attitudes changed and lessons learned. Acad Med. 1995;70(3):245–7.
Neville AJ. The problem-based learning tutor: Teacher? Facilitator? Evaluator? Med Teach. 1999;21(4):393–401.
Dornan T, Hadfield J, Brown M, Boshuizen H, Scherpbier A. How can medical students learn in a self-directed way in the clinical environment? Design-based research Med Educ. 2005;39(4):356–64.
Balslev T, De Grave WS, Muijtjens AMM, Scherpbier AJJA. Comparison of text and video cases in a postgraduate problem-based learning format. Med Educ. 2005;39(11):1086–92.
Steadman RH, Coates WC, Yue MH, Matevosian R, Larmon BR, McCullough L, et al. Simulation-based training is superior to problem-based learning for the acquisition of critical assessment and management skills. Crit Care Med. 2006;34(1):151–7.
Shin JH, Haynes RB, Johnston ME. Effect of problem-based, self-directed undergraduate education on life-long learning. CMAJ. 1993;148(6):969–76.
Prince KJAH, Van Mameren H, Hylkema N, Drukker J, Scherpbier AJJA, van der Vleuten CPM. Does problem-based learning lead to deficiencies in basic science knowledge? An empirical case on anatomy. Med Educ. 2003;37(1):15–21.
De Grave WS, Boshuizen HPA, Schmidt HG. Problem based learning: cognitive and metacognitive processes during problem analysis. Instr Sci. 1996;24(5):321–41.
Taylor D, Miflin B. Problem-based learning: where are we now? Med Teach. 2008;30(8):742–63.
Peters AS, Greenberger-Rosovsky R, Crowder C, Block SD, Moore GT. Long-term outcomes of the new pathway program at Harvard Medical School: a randomized controlled trial. Acad Med. 2000;75(5):470–9.
Dolmans D, Schmidt H. The advantages of problem-based curricula. Postgrad Med J. 1996;72(851):535–8.
Distlehorst LH, Dawson E, Robbs RS, Barrows HS. Problem-based learning outcomes: the glass half-full. Acad Med. 2005;80(3):294–9.
Moffat KJ, McConnachie A, Ross S, Morrison JM. First year medical student stress and coping in a problem-based learning medical curriculum. Med Educ. 2004;38(5):482–91.
Newble DI, Clarke RM. The approaches to learning of students in a traditional and in an innovative problem-based medical school. Med Educ. 1986;20(4):267–73.
Barrows HS. Problem-based, self-directed learning. JAMA J Am Med Assoc. 1983;250(22):3077–80.
Blumberg P, Michael JA, Blumberg P. Development of self-directed learning behaviors in a partially teacher-directed problem-based learning curriculum. Teach Learn Med. 1992;4(1):3–8.
Neville AJ, Norman GR. PBL in the undergraduate MD Program at McMaster University: three iterations in three decades. Acad Med. 2007;82(4):370–4.
Khoo HE. Implementation of problem-based learning in Asian medical schools and students’ perceptions of their experience. Med Educ. 2003;37(5):401–9.
Papinczak T, Young L, Groves M, Haynes M. An analysis of peer, self, and tutor assessment in problem-based learning tutorials. Med Teach. 2007;29(5).
Kamin C, O’Sullivan P, Deterding R, Younger M. A comparison of critical thinking in groups of third-year medical students in text, video, and virtual PBL case modalities. Acad Med. 2003;78(2):204–11.
Hendry GD, Ryan G, Harris J. Group problems in problem-based learning. Med Teach. 2003;25(6):609–16.
Blake RL, Hosokawa MC, Riley SL. Student performances on Step 1 and Step 2 of the United States Medical Licensing Examination following implementation of a problem-based learning curriculum. Acad Med. 2000;75(1):66–70.
Schmidt HG, Vermeulen L, Van Der Molen HT. Longterm effects of problem-based learning: a comparison of competencies acquired by graduates of a problem-based and a conventional medical school. Med Educ. 2006;40(6):562–7.
McParland M, Noble LM, Livingston G. The effectiveness of problem-based learning compared to traditional teaching in undergraduate psychiatry. Med Educ. 2004;38(8):859–67.
Dolmans DHJM, Schmidt HG. What do we know about cognitive and motivational effects of small group tutorials in problem-based learning? Adv Heal Sci Educ. 2006;11(4):321–36.
Schmidt HG, van der Arend A, Moust JHC, Kokx I, Boon L. Influence of tutors’ subject-matter expertise on student effort and achievement in problem-based learning. Acad Med. 1993;68(10):784–91.
Donner RS, Bickley H. Problem-based learning in American medical education: an overview. Bull Med Libr Assoc. 1993;81(3):294–8.
Eagle CJ, Harasym PH, Mandin H. Effects of tutors with case expertise on problem-based learning issues. Acad Med. 1992;67(7):465–9.
Papinczak T, Young L, Groves M. Peer assessment in problem-based learning: a qualitative study. Adv Heal Sci Educ. 2007;12(2):169–86.
Hoffman K, Hosokawa M, Blake R Jr, Headrick L, Johnson G. Problem-based learning outcomes: ten years of experience at the University of Missouri-Columbia School of Medicine. Acad Med. 2006;81(7):617–25.
van Eck NJ, Waltman L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics. 2010;84(2):523–38.
Ioannidis JPA, Fanelli D, Dunne DD, Goodman SN. Meta-research: evaluation and improvement of research methods and practices. PLoS Biol. 2015;13(10):1–7.
Miranda R, Garcia-Carpintero E. Overcitation and overrepresentation of review papers in the most cited papers. J Informetr. 2018;12(4):1015–30.
Lee K, Whelan JS, Tannery NH, Kanter SL, Peters AS. 50 years of publication in the field of medical education. Med Teach. 2013;35(7):591–8.
Albert M, Rowland P, Friesen F, Laberge S. Interdisciplinarity in medical education research: myth and reality. Adv Heal Sci Educ. 2020;25(5):1243–53.
Association of American Medical Colleges. Use of instructional methods [Internet]. Washngton, (DC): Association of American Medical Colleges; [cited 14 Dec 2022]. Available from: https://www.aamc.org/data-reports/curriculum-reports/interactive-data/instructional-methods-used-medical-schools
Hung W, Dolmans DHJM, van Merriënboer JJG. A review to identify key perspectives in PBL meta-analyses and reviews: trends, gaps and future research directions. Adv Heal Sci Educ. 2019;24(5):943–57.
Yew EHJ, Goh K. Problem-based learning: an overview of its process and impact on learning. Heal Prof Educ. 2016;2(2):75–9.
O’Sullivan PS, Stoddard HA, Kalishman S. Collaborative research in medical education: a discussion of theory and practice. Med Educ. 2010;44(12):1175–84.
Hillen H, Scherpbier A, Wijnen W. History of problem-based learning in medical education. In: van Berkel H, Scherpbier A, Hillen H, van der Vleuten C, editors. Lessons from problem-based learning. Oxford: Oxford University Press; 2010. p. 5–11.
Singh VK, Singh P, Karmakar M, Leta J, Mayr P. The journal coverage of Web of Science, Scopus and Dimensions: a comparative analysis. Scientometrics. 2021;126(6):5113–42.
Kacem A, Flatt JW, Mayr P. Tracking self-citations in academic publishing. Scientometrics. 2020;123(2):1157–65.
Funding
No funding was received for conducting this study.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by Violet Kulo and Christina Cestone. The first draft of the manuscript was written by Violet Kulo and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics Approval
Not applicable. This is a descriptive study.
Informed Consent
No human research participants were involved in the study.
Competing Interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Kulo, V., Cestone, C. A Bibliometric Analysis of the 100 Most Cited Articles on Problem-Based Learning in Medical Education. Med.Sci.Educ. 33, 1409–1426 (2023). https://doi.org/10.1007/s40670-023-01893-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40670-023-01893-x