Abstract
For successful learning, students need to evaluate their learning status relative to their learning goals and regulate their study in response to such monitoring. The present study investigated whether making metacognitive judgments on previously studied text would enhance the learning of that studied (backward effect) and newly studied text material (forward effect). We also examined how different learning goals would orient learners to adopt different study strategies and, in turn, affect learning performance by asking learners to make different types of metacognitive judgments. In two experiments, participants studied two different passages across two sections (Sections A and B). They were asked to make either inference-based or memory-based metacognitive judgments on the studied passage of Section A before studying a new passage in Section B. The study-only control group did not make any metacognitive judgments between sections. On completion of Section B, all participants were given final retention and transfer tests on both sections. The meta-analytic results from the two experiments revealed that making inference-based metacognitive judgments was more beneficial than simply studying the material for both retention and transfer of the previously studied and newly studied text material. However, the benefit of memory-based metacognitive judgments was limited, in that it did not enhance retention performance of the previously studied material compared to the control condition. The current findings suggest that the effectiveness of metacognitive judgments varies depending on the learning goal. Highlighting a high-level learning goal seems to influence learners’ all knowledge levels, showing a cascading effect.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The data for all experiments have been made available at https://osf.io/abhvu/?view_only=b90eae66055b450cab7920c356708b26. None of the experiments were preregistered.
Notes
We thank Andrew C. Butler at Washington University in St. Louis for providing the original text material used in this study (https://doi.org/10.1037/a0019902.supp)
References
Ackerman, R., & Goldsmith, M. (2011). Metacognitive regulation of text learning: On screen versus on paper. Journal of Experimental Psychology: Applied, 17(1), 18–32. https://doi.org/10.1037/a0022086
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., & Wittrock, M. C. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives (abridged edition). Addison Wesley Longman.
Ariel, R., Karpicke, J. D., Witherby, A. E., & Tauber, S. K. (2021). Do judgments of learning directly enhance learning of educational materials? Educational Psychology Review, 33(2), 693–712. https://doi.org/10.1007/s10648-020-09556-8
Bannert, M., Hildebrand, M., & Mengelkamp, C. (2009). Effects of a metacognitive support device in learning environments. Computers in Human Behavior, 25, 829–835. https://doi.org/10.1016/j.chb.2008.07.002
Bannert, M., & Reimann, P. (2012). Supporting self-regulated hypermedia learning through prompts. Instructional Science, 40(1), 193–211. https://doi.org/10.1007/s11251-011-9167-4
Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64(1), 417–444. https://doi.org/10.1146/annurev-psych-113011-143823
Butler, A. C. (2010). Repeated testing produces superior transfer of learning relative to repeated studying. Journal of Experimental Psychology: Learning Memory and Cognition, 36(5), 1118–1133. https://doi.org/10.1037/a0019902
Butler, D. L. (1994). From learning strategies to strategic learning: Promoting self-regulated learning by post secondary students with learning disabilities. Canadian Journal of Special Education, 9(3–4), 69–101.
Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245–281. https://doi.org/10.3102/003465430650032450
Choi, H., & Lee, H. S. (2020). Knowing is not half the battle: The role of actual test experience in the forward testing effect. Educational Psychology Review, 32(3), 765–789. https://doi.org/10.1007/s10648-020-09518-0
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104
Daumiller, M., & Dresel, M. (2019). Supporting self-regulated learning with digital media using motivational regulation and metacognitive prompts. The Journal of Experimental Education, 87(1), 161–176. https://doi.org/10.1080/00220973.2018.1448744
Dobson, J. L., Linderholm, T., & Stroud, L. (2019). Retrieval practice and judgements of learning enhance transfer of physiology information. Advances in Health Sciences Education, 24(3), 525–537. https://doi.org/10.1007/s10459-019-09881-w
Double, K. S., Birney, D. P., & Walker, S. A. (2018). A meta-analysis and systematic review of reactivity to judgements of learning. Memory (Hove, England), 26(6), 741–750. https://doi.org/10.1080/09658211.2017.1404111
Double, K. S., & Birney, D. P. (2019). Reactivity to measures of metacognition. Frontiers in Psychology, 10, 1–12. https://doi.org/10.3389/fpsyg.2019.02755
Dougherty, M. R., Scheck, P., Nelson, T. O., & Narens, L. (2005). Using the past to predict the future. Memory & Cognition, 33, 1096–1115. https://doi.org/10.3758/BF03193216
Dunlosky, J., & Lipko, A. R. (2007). Metacomprehension: A brief history and how to improve its accuracy. Current Directions in Psychological Science, 16(4), 228–232. https://doi.org/10.1111/j.1467-8721.2007.00509.x
Dunlosky, J., & Nelson, T. O. (1992). Importance of the kind of cue for judgments of learning (JOL) and the delayed-JOL effect. Memory & Cognition, 20(4), 374–380. https://doi.org/10.3758/BF03210921
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4–58. https://doi.org/10.1177/1529100612453266
Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G* power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/BF03193146
Finley, J. R., & Benjamin, A. S. (2012). Adaptive and qualitative changes in encoding strategy with experience: Evidence from the test-expectancy paradigm. Journal of Experimental Psychology: Learning Memory and Cognition, 38(3), 632–652. https://doi.org/10.1037/a0026215
Goh, J. X., Hall, J. A., & Rosenthal, R. (2016). Mini meta-analysis of your own studies: Some arguments on why and a primer on how. Social and Personality Psychology Compass, 10(10), 535–549. https://doi.org/10.1111/spc3.12267
Halis, Ä. (2002). Instructional technologies and material development. Nobel Press.
Janes, J. L., Rivers, M. L., & Dunlosky, J. (2018). The influence of making judgments of learning on memory performance: Positive, negative, or both? Psychonomic Bulletin & Review, 25(6), 2356–2364. https://doi.org/10.3758/s13423-018-1463-4
Jensen, J. L., McDaniel, M. A., Woodard, S. M., & Kummer, T. A. (2014). Teaching to the test… or testing to teach: Exams requiring higher order thinking skills encourage greater conceptual understanding. Educational Psychology Review, 26(2), 307–329. https://doi.org/10.1007/s10648-013-9248-9
Jönsson, F. U., Hedner, M., & Olsson, M. J. (2012). The testing effect as a function of explicit testing instructions and judgments of learning. Experimental Psychology, 59, 251–257. https://doi.org/10.1027/1618-3169/a000150
Joughin, G. (2010). The hidden curriculum revisited: A critical review of research into the influence of summative assessment on learning. Assessment & Evaluation in Higher Education, 35(3), 335–345. https://doi.org/10.1080/02602930903221493
Kimball, D. R., & Metcalfe, J. (2003). Delaying judgments of learning affects memory, not metamemory. Memory & Cognition, 31(6), 918–929. https://doi.org/10.3758/BF03196445
Kubik, V., Koslowski, K., Schubert, T., & Aslan, A. (2022). Metacognitive judgments can potentiate new learning: The role of covert retrieval. Metacognition and Learning, 1–21. https://doi.org/10.1007/s11409-022-09307-w
Lee, H. S., & Ahn, D. (2018). Testing prepares students to learn better: The forward effect of testing in category learning. Journal of Educational Psychology, 110(2), 203–217. https://doi.org/10.1037/edu0000211
Lee, H. S., & Ha, H. (2019). Metacognitive judgments of prior material facilitate the learning of new material: The forward effect of metacognitive judgments in inductive learning. Journal of Educational Psychology, 111(7), 1189–1201. https://doi.org/10.1037/edu0000339
Lin, X., Hmelo, C., Kinzer, C. K., & Secules, T. J. (1999). Designing technology to support reflection. Educational Technology Research and Development, 47(3), 43–62. https://doi.org/10.1007/BF02299633
Lin, X., & Lehman, J. D. (1999). Supporting learning of variable control in a computer-based biology environment: Effects of prompting college students to reflect on their own thinking. Journal of Research in Science Teaching, 36(7), 837–858. https://doi.org/10.1002/(SICI)1098-2736(199909)36:7<837::AID-TEA6>3.0.CO;2-U
McDaniel, M. A., Thomas, R. C., Agarwal, P. K., McDermott, K. B., & Roediger, H. L. (2013). Quizzing in middle-school science: Successful transfer performance on classroom exams. Applied Cognitive Psychology, 27(3), 360–372. https://doi.org/10.1002/acp.2914
Metcalfe, J., & Finn, B. (2008). Evidence that judgments of learning are causally related to study choice. Psychonomic Bulletin & Review, 15(1), 174–179. https://doi.org/10.3758/PBR.15.1.174
Mevarech, Z. R., & Kramarski, B. (2003). The effects of metacognitive training versus worked-out examples on students’ mathematical reasoning. British Journal of Educational Psychology, 73(4), 449–471. https://doi.org/10.1348/000709903322591181
Middlebrooks, C. D., Murayama, K., & Castel, A. D. (2017). Test expectancy and memory for important information. Journal of Experimental Psychology: Learning Memory and Cognition, 43(6), 972–985. https://doi.org/10.1037/xlm0000360
Mitchum, A. L., Kelley, C. M., & Fox, M. C. (2016). When asking the question changes the ultimate answer: Metamemory judgments change memory. Journal of Experimental Psychology: General, 145(2), 200–219. https://doi.org/10.1037/a0039923
Myers, S. J., Rhodes, M. G., & Hausman, H. E. (2020). Judgments of learning (JOLs) selectively improve memory depending on the type of test. Memory & Cognition, 48(5), 745–758. https://doi.org/10.3758/s13421-020-01025-5
Nelson, T. O., & Dunlosky, J. (1991). When people’s judgments of learning (JOLs) are extremely accurate at predicting subsequent recall: The “delayed-JOL effect. Psychological Science, 2(4), 267–271. https://doi.org/10.1111/j.1467-9280.1991.tb00147.x
Nelson, T. O., & Narens, L. (1990). Metamemory: A theoretical framework and new findings. In G. H. Bower (Ed.), The psychology of learning and motivation (26 vol., pp. 125–141). Academic.
Nguyen, K., & McDaniel, M. A. (2016). The JOIs of text comprehension: Supplementing retrieval practice to enhance inference performance. Journal of Experimental Psychology: Applied, 22, 59–71. https://doi.org/10.1037/xap0000066a
Pastötter, B., & Bäuml, K. H. T. (2014). Retrieval practice enhances new learning: The forward effect of testing. Frontiers in Psychology, 5, 1–5. https://doi.org/10.3389/fpsyg.2014.00286
Putnam, A. L., & Roediger, H. L. (2013). Does response mode affect amount recalled or the magnitude of the testing effect? Memory & Cognition, 41(1), 36–48. https://doi.org/10.3758/s13421-012-0245-x
Rivers, M. L., & Dunlosky, J. (2021). Are test-expectancy effects better explained by changes in encoding strategies or differential test experience? Journal of Experimental Psychology: Learning Memory and Cognition, 47(2), 195–207. https://doi.org/10.1037/xlm0000949
Roediger, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181–210. https://doi.org/10.1111/j.1745-6916.2006.00012.x
Roelle, J., Roelle, D., & Berthold, K. (2019). Test-based learning: Inconsistent effects between higher-and lower-level test questions. The Journal of Experimental Education, 87(2), 299–313. https://doi.org/10.1080/00220973.2018.1434756
Rouet, J., Vidal-Abarca, E., Erboul, A. B., & Millogo, V. (2001). Effects of information search tasks on the comprehension of instructional text. Discourse Processes, 31(2), 163–186. https://doi.org/10.1207/S15326950DP3102_03
Safari, Y., & Meskini, H. (2016). The effect of metacognitive instruction on problem solving skills in iranian students of health sciences. Global Journal of Health Science, 8(1), 150–156. https://doi.org/10.5539/gjhs.v8n1p150
Schraw, G. (1998). Promoting general metacognitive awareness. Instructional Science, 26(1/2), 113–125. https://doi.org/10.1023/A:1003044231033
Soderstrom, N. C., Clark, C. T., Halamish, V., & Bjork, E. L. (2015). Judgments of learning as memory modifiers. Journal of Experimental Psychology: Learning Memory and Cognition, 41(2), 553–558. https://doi.org/10.1037/a0038388
Sommer, W., Heinz, A., Leuthold, H., Matt, J., & Schweinberger, S. R. (1995). Metamemory, distinctiveness, and event-related potentials in recognition memory for faces. Memory & Cognition, 23(1), 1–11. https://doi.org/10.3758/BF03210552
Son, L. K., & Metcalfe, J. (2005). Judgments of learning: Evidence for a two-stage process. Memory & Cognition, 33(6), 1116–1129. https://doi.org/10.3758/BF03193217
Spellman, B. A., & Bjork, R. A. (1992). When predictions create reality: Judgments of learning may alter what they are intended to assess. Psychological Science, 3(5), 315–317. https://doi.org/10.1111/j.1467-9280.1992.tb00680.x
Swanson, H. L. (1990). Influence of metacognitive knowledge and aptitude on problem solving. Journal of Educational Psychology, 82(2), 306–314. https://doi.org/10.1037/0022-0663.82.2.306
Tauber, S. K., Dunlosky, J., & Rawson, K. A. (2015). The influence of retrieval practice versus delayed judgments of learning on memory: Resolving a memory-metamemory paradox. Experimental Psychology, 62(4), 254–263. https://doi.org/10.1027/1618-3169/a000296
Tekin, E., & Roediger, H. L. (2020). Reactivity of judgments of learning in a levels-of-processing paradigm. Zeitschrift für Psychologie, 228(4), 278–290. https://doi.org/10.1027/2151-2604/a000425
Tekin, E., & Roediger, H. L. (2021). The effect of delayed judgments of learning on retention. Metacognition and Learning, 16(2), 407–429. https://doi.org/10.1007/s11409-021-09260-0
Thiede, K. W., Anderson, M. C. M., & Therriault, D. (2003). Accuracy of metacognitive monitoring affects learning of texts. Journal of Educational Psychology, 95(1), 66–73. https://doi.org/10.1037/0022-0663.95.1.66
Thiede, K. W., Wiley, J., & Griffin, T. D. (2011). Test expectancy affects metacomprehension accuracy. British Journal of Educational Psychology, 81(2), 264–273. https://doi.org/10.1348/135910710X510494
Winne, P. H., & Nesbit, J. C. (2010). The psychology of academic achievement. Annual Review of Psychology, 61, 653–678. https://doi.org/10.1146/annurev.psych.093008.100348
Witherby, A. E., & Tauber, S. K. (2017). The influence of judgments of learning on long-term learning and short-term performance. Journal of Applied Research in Memory and Cognition, 6(4), 496–503. https://doi.org/10.1016/j.jarmac.2017.08.004
Acknowledgements
This work was supported by the Yonsei University Research Grant of 2021 (R202106047). Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the Yonsei University. The authors wish to thank Yewon Kang and Yundeok Kim for their valuable assistance in conducting this research.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethics approval
This study was approved by the Institutional Review Board at the Yonsei University (#IRB-7001988-202209-HR-954-05).
Informed consent
All participants provided informed consent prior to their participation, and they understood that they could quit at any time.
Conflict of interest
The authors declare that they do not have any conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ha, H., Lee, H.S. Think higher, gain more: the effect of making inference- and memory-based metacognitive judgments on text learning. Metacognition Learning 18, 567–590 (2023). https://doi.org/10.1007/s11409-023-09341-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11409-023-09341-2