Arthur Câmara
Nirmal Roy
David Maxwell
ABSTRACT
Web search engines are today considered to be the primary tool to assist and empower learners in finding information relevant to their learning goals- be it learning something new, improving their existing skills, or just fulfilling a curiosity. While several approaches for improving search engines for the learning scenario have been proposed (e.g. a specific ranking function), instructional scaffolding (or simply scaffolding)-a traditional learning support strategy-has not been studied in the context of search as learning, despite being shown to be effective for improving learning in both digital and traditional learning contexts. When scaffolding is employed, instructors provide learners with support throughout their autonomous learning process. We hypothesize that the usage of scaffolding techniques within a search system can be an effective way to help learners achieve their learning objectives whilst searching. As such, this paper investigates the incorporation of scaffolding into a search system employing three different strategies (as well as a control condition): (i) AQe, the automatic expansion of user queries with relevant subtopics; (ii) CURATEDsc, the presenting of a manually curated static list of relevant subtopics on the search engine result page; and (iii) FEEDBACKsc, which projects real-time feedback about a user's exploration of the topic space on top of the CURATEDsc visualization. To investigate the effectiveness of these approaches with respect to human learning, we conduct a user study (N=126) where participants were tasked with searching and learning about topics such as genetically modified organisms. We find that (i) the introduction of the proposed scaffolding methods in the proposed topics does not significantly improve learning gains. However, (ii) it does significantly impact search behavior. Furthermore, (iii) immediate feedback of the participants' learning (FEEDBACKsc) leads to undesirable user behavior, with participants seemingly focusing on the feedback gauges instead of learning.
- L. Anderson-Inman and L. Zeitz. 1993. Computer-based concept mapping: Active studying for active learners. The Computer Teacher.(Aug./Sept.) (1993).Google Scholar
- B.R. Belland. 2017. Instructional scaffolding: foundations and evolving definition. In Instructional Scaffolding in STEM Education. Springer, 17--53.Google Scholar
- N. Bhattacharya and J. Gwizdka. 2018. Relating eye-tracking measures with changes in knowledge on search tasks. In Proc. 10th ACM ETRA. 1--5.Google Scholar
- N. Bhattacharya and J. Gwizdka. 2019. Measuring learning during search: differences in interactions, eye-gaze, and semantic similarity to expert knowledge. In Proc. 4textsuperscriptth ACM CHIIR. 63--71.Google Scholar
- J.P. Biddix, C.J. Chung, and H.W. Park. 2011. Convenience or credibility? A study of college student online research behaviors. The Internet & Higher Education, Vol. 14, 3 (2011), 175--182.Google ScholarCross Ref
- A.L. Brown and A.S. Palincsar. 1986. Guided, Cooperative Learning and Individual Knowledge Acquisition. Technical Report. U. Illinois Urbana-Champaign. 116 pages.Google Scholar
- K. Collins-Thompson, P. Hansen, and C. Hauff. 2017. Search as Learning. Dagstuhl Reports, Vol. 7, 2 (2017), 135--162.Google Scholar
- Kevyn Collins-Thompson, Soo Young Rieh, Carl C Haynes, and Rohail Syed. 2016. Assessing learning outcomes in web search: A comparison of tasks and query strategies. In Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval. ACM, 163--172.Google ScholarDigital Library
- H.G. Colt, M. Davoudi, S. Murgu, and N.Z. Rohani. 2011. Measuring learning gain during a one-day introductory bronchoscopy course. Surgical endoscopy, Vol. 25, 1 (2011), 207--216.Google Scholar
- J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proc. NAACL, Vol. 1. 4171--4186.Google Scholar
- L. Dietz, M. Verma, F. Radlinski, and N. Craswell. 2017. TREC Complex Answer Retrieval Overview. TREC.Google Scholar
- C. Eickhoff, J. Teevan, R. White, and S. Dumais. 2014. Lessons from the journey: a query log analysis of within-session learning. In Proc. 7th ACM WSDM. 223--232.Google Scholar
- U. Gadiraju, R. Yu, S. Dietze, and P. Holtz. 2018. Analyzing knowledge gain of users in informational search sessions on the web. In Proc. 3rd ACM CHIIR. 2--11.Google Scholar
- S.M. Glynn and F.J. Di Vesta. 1977. Outline and hierarchical organization as aids for study and retrieval. J. Educational Psychology, Vol. 69, 2 (1977), 89.Google ScholarCross Ref
- G. Golovchinsky, A. Diriye, and T. Dunnigan. 2012. The future is in the past: designing for exploratory search. In Proc. 4th IIiX. 52--61.Google Scholar
- A. Hassan Awadallah, R.W. White, P. Pantel, S.T. Dumais, and Y.-M. Wang. 2014. Supporting complex search tasks. In Proc. 23rd ACM CIKM. 829--838.Google Scholar
- J.R. Hill and M.J. Hannafin. 2001. Teaching and learning in digital environments: The resurgence of resource-based learning. Educational Tech. Research & Development, Vol. 49, 3 (9 2001), 37--52.Google Scholar
- R. Kalyani and U. Gadiraju. 2019. Understanding User Search Behavior Across Varying Cognitive Levels. In Proc. 30th ACM HT. 123--132.Google Scholar
- D.R. Krathwohl and L.W. Anderson. 2009. A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. Longman.Google Scholar
- C. Liu and X. Song. 2018. How do Information Source Selection Strategies Influence Users' Learning Outcomes'. In Proc. 3rd ACM CHIIR. 257--260.Google Scholar
- H. Liu, C. Liu, and N.J. Belkin. 2019. Investigation of users' knowledge change process in learning-related search tasks. Proc. ASIS&T, Vol. 56, 1 (2019), 166--175.Google Scholar
- Y. Lu and I-H. Hsiao. 2017. Personalized Information Seeking Assistant (PiSA): from programming information seeking to learning. IRJ, Vol. 20, 5 (2017), 433--455.Google ScholarDigital Library
- B. Luyt. 2012. The inclusivity of Wikipedia and the drawing of expert boundaries: An examination of talk pages and reference lists. JASIST, Vol. 63, 9 (2012), 1868--1878.Google ScholarDigital Library
- G. Marchionini. 2006. Exploratory search: from finding to understanding. Comm. ACM, Vol. 49, 4 (2006), 41--46.Google ScholarDigital Library
- R.E. Mayer, E. Griffith, I.TN Jurkowitz, and D. Rothman. 2008. Increased interestingness of extraneous details in a multimedia science presentation leads to decreased learning. J. Experimental Psychology: Applied, Vol. 14, 4 (2008), 329.Google ScholarCross Ref
- D.C. Merrill, B.J. Reiser, M. Ranney, and J.Gregory. Trafton. 1992. Effective tutoring techniques: A comparison of human tutors and intelligent tutoring systems. J. Learning Sciences, Vol. 2, 3 (1992), 277--305.Google ScholarCross Ref
- F. Moraes, S.R. Putra, and C. Hauff. 2018. Contrasting Search as a Learning Activity with Instructor-designed Learning. In Proc. 27th ACM CIKM. 167--176.Google Scholar
- D. Nicholas, I. Rowlands, D. Clark, and P. Williams. 2011. Google Generation II: web behaviour experiments with the BBC. In Aslib proceedings, Vol. 63. 28--45.Google Scholar
- H.L. O'Brien, A. Kampen, A.W. Cole, and K. Brennan. 2020. The Role of Domain Knowledge in Search as Learning. In Proc. 5th ACM CHIIR. 313--317.Google Scholar
- G. Pardi, J. von Hoyer, P. Holtz, and Y. Kammerer. 2020. The Role of Cognitive Abilities and Time Spent on Texts and Videos in a Multimodal Searching as Learning Task. In Proc. 5th ACM CHIIR. 378--382.Google Scholar
- S.R. Putra, F. Moraes, and C. Hauff. 2018. SearchX: Empowering Collaborative Search Research. In Proc. 41st ACM SIGIR. 1265--1268.Google Scholar
- B. Rogoff. 1986. Adult assistance of children's learning. The contexts of school-based literacy (1986), 27--40.Google Scholar
- I. Rowlands, D. Nicholas, P. Williams, P. Huntington, M. Fieldhouse, B. Gunter, R. Withey, H.R. Jamali, T. Dobrowolski, and C. Tenopir. 2008. The Google generation: the information behaviour of the researcher of the future. In Aslib proceedings, Vol. 60. 290--310.Google Scholar
- N. Roy, F. Moraes, and C. Hauff. 2020. Exploring Users? Learning Gains within Search Sessions. In Proc. 5th ACM CHIIR. 432--436.Google Scholar
- N. Selwyn. 2008. An investigation of differences in undergraduates' academic use of the internet. Active learning in higher education, Vol. 9, 1 (2008), 11--22.Google Scholar
- P. Sharma and M. J. Hannafin. 2007. Scaffolding in technology-enhanced learning environments. Interactive learning environments, Vol. 15, 1 (2007), 27--46.Google Scholar
- J.L. Shefelbine. 1990. Student factors related to variability in learning word meanings from context. J. Reading Behavior, Vol. 22, 1 (1990), 71--97.Google ScholarCross Ref
- C.L. Smith and S.Y. Rieh. 2019. Knowledge-context in search systems: Toward information-literate actions. In Proc. 4th ACM CHIIR. 55--62.Google Scholar
- S.P. Swinnen, R.A. Schmidt, D.E. Nicholson, and D.C. Shapiro. 1990. Information feedback for skill acquisition: Instantaneous knowledge of results degrades learning. J. Experimental Psychology: Learning, Memory, & Cognition, Vol. 16, 4 (1990), 706.Google ScholarCross Ref
- R. Syed and K. Collins-Thompson. 2017a. Optimizing search results for human learning goals. IRJ, Vol. 20, 5 (2017), 506--523.Google ScholarDigital Library
- R. Syed and K. Collins-Thompson. 2017b. Retrieval algorithms optimized for human learning. In Proc 40th ACM SIGIR. 555--564.Google Scholar
- R. Syed and K. Collins-Thompson. 2018. Exploring Document Retrieval Features Associated with Improved Short-and Long-term Vocabulary Learning Outcomes. In Proc. 3rd ACM CHIIR. 191--200.Google Scholar
- R. Syed, K. Collins-Thompson, P.N. Bennett, M. Teng, S. Williams, W.W. Tay, and S. Iqbal. 2020. Improving Learning Outcomes with Gaze Tracking and Automatic Question Generation. In Proc. 29textsuperscriptth WWW. 1693--1703.Google Scholar
- K. Umemoto, T. Yamamoto, and K. Tanaka. 2016. ScentBar: A Query Suggestion Interface Visualizing the Amount of Missed Relevant Information for Intrinsically Diverse Search. In Proc. 39th ACM SIGIR. 405--414.Google Scholar
- K. Urgo, J. Arguello, and R. Capra. 2019. Anderson and Krathwohl's Two-Dimensional Taxonomy Applied to Task Creation and Learning Assessment. In Proc. 5th ACM ICTIR. 117--124.Google Scholar
- J. von Hoyer, G. Pardi, Y. Kammerer, and P. Holtz. 2019. Metacognitive Judgments in Searching as Learning (SAL) Tasks: Insights on (Mis-) Calibration, Multimedia Usage, and Confidence. In Proc. 1textsuperscriptst ACM SALMM. 3--10.Google Scholar
- L.S. Vygotsky. 1980. Mind in society: The development of higher psychological processes. Harvard Univ. Press.Google Scholar
- M. Wesche and T.S. Paribakht. 1996. Assessing second language vocabulary knowledge: Depth versus breadth. Canadian Modern Language Review, Vol. 53, 1 (1996), 13--40.Google ScholarCross Ref
- B.M. Wildemuth. 2004. The effects of domain knowledge on search tactic formulation. JASIST, Vol. 55, 3 (2004), 246--258.Google ScholarDigital Library
- M.J. Wilson and M.L. Wilson. 2013. A comparison of techniques for measuring sensemaking and learning within participant-generated summaries. JASIST, Vol. 64, 2 (2013), 291--306.Google ScholarDigital Library
- R. Yu, U. Gadiraju, P. Holtz, M. Rokicki, P. Kemkes, and S. Dietze. 2018. Predicting user knowledge gain in informational search sessions. In Proc. 41st ACM SIGIR. 75--84.Google Scholar
- R. Zhang, J. Guo, Y. Fan, Y. Lan, and X. Cheng. 2019. Outline Generation: Understanding the Inherent Content Structure of Documents. In Proc 42th ACM SIGIR. 745--754.Google Scholar
Index Terms
- Searching to Learn with Instructional Scaffolding
Recommendations
Effects of interactivity and instructional scaffolding on learning: Self-regulation in online video-based environments
Online learning often requires learners to be self-directed and engaged. The present study examined students' self-regulatory behaviors in online video-based learning environments. Using an experimental design, this study investigated the effects of a ...
Instructional Scaffolding: The Computerized Classroom
This article addresses the use of embedded scaffolding to enhance web-based learning in an online course which covers an introduction to multiple programming languages, utilizing selected resources and strategies as scaffolding techniques. This ...
Instructional Scaffolding in Online Learning Environment: A Meta-analysis
LATICE '14: Proceedings of the 2014 International Conference on Teaching and Learning in Computing and EngineeringInstructional scaffolding or simply known as scaffolding in education is defined as a guidance or support from teachers, instructors or other knowledgeable persons that facilitate students to achieve their goals in learning. Conceptually, scaffolding ...
Comments