Heather L. O'Brien
ABSTRACT
The purpose of this paper is to explore the differences in learning outcomes between domain experts and non-experts interacting with a digital library. This paper provides a preliminary analysis of a subset of data from a larger study and compares the learning outcomes of 10 domain experts and 10 non-experts. Participants completed three search tasks designed to elicit different cognitive processes and behaviors; learning outcomes were explored with pre- and post-session written summaries and pre- and post-task questions. We used existing metrics to examine the learning breadth and depth of the written summaries. General searchers wrote longer summaries than domain experts, but there were no significant differences in the learning outcomes between the two groups.
- James Allan, Bruce Croft, Alistair Moffat, and Mark Sanderson. 2012. Frontiers, challenges, and opportunities for information retrieval: Report from SWIRL 2012 the second strategic workshop on information retrieval in Lorne. ACM SIGIR Forum 46, 1 (May 2012), 2. DOI:https://doi.org/10.1145/2215676.2215678Google Scholar
Digital Library
- Earl Bailey. 2017. Measuring Online Search Expertise. (Unpublished Dissertation). Dissertation. The University of North Carolina, Chapel Hill.Google Scholar
- Christine L Borgman. 1999. What are digital libraries? Competing visions. Information Processing and Management (1999), 17.Google Scholar
- Pia Borlund. 2016. Framing of different types of information needs within simulated work task situations: An empirical study in the school context. Journal of Information Science 42, 3 (June 2016), 313--323. DOI:https://doi.org/10.1177/0165551515625028Google ScholarDigital Library
- Kevyn Collins-Thompson, Soo Young Rieh, Carl C. Haynes, and Rohail Syed. 2016. Assessing learning outcomes in web search: A comparison of tasks and query strategies. In 2016 ACM on Conference on Human Information Interaction and Retrieval, ACM Press, 163--172. DOI:https://doi.org/10.1145/2854946.2854972Google ScholarDigital Library
- Luanne Freund, Rick Kopak, and Heather O'Brien. 2016. The effects of textual environment on reading comprehension: Implications for searching as learning. Journal of Information Science 42, 1 (February 2016), 79--93. DOI:https://doi.org/10.1177/0165551515614472Google ScholarDigital Library
- Ujwal Gadiraju, Ran Yu, Stefan Dietze, and Peter Holtz. 2018. Analyzing Knowledge Gain of Users in Informational Search Sessions on the Web. In Proceedings of the 2018 Conference on Human Information Interaction and Retrieval - CHIIR '18, ACM Press, New Brunswick, NJ, USA, 2--11. DOI:https://doi.org/10.1145/3176349.3176381Google ScholarDigital Library
- Yvonne Kammerer, Rowan Nairn, Peter Pirolli, and Ed H. Chi. 2009. Signpost from the masses: learning effects in an exploratory social tag search browser. In Proceedings of the 27th international conference on Human factors in computing systems - CHI 09, ACM Press, Boston, MA, USA, 625. DOI:https://doi.org/10.1145/1518701.1518797Google Scholar
- Saraschandra Karanam, Guillermo Jorge-Botana, Ricardo Olmos, and Herre van Oostendorp. 2017. The role of domain knowledge in cognitive modeling of information search. Information Retrieval Journal 20, 5 (October 2017), 456--479. DOI:https://doi.org/10.1007/s10791-017--9308--8Google ScholarDigital Library
- Diane Kelly. 2007. Methods for Evaluating Interactive Information Retrieval Systems with Users. Foundations and Trends in Information Retrieval 3, 1-2 (2007), 1--224. DOI:https://doi.org/10.1561/1500000012Google ScholarDigital Library
- Diane Kelly, Jaime Arguello, Ashlee Edwards, and Wan-ching Wu. 2015. Development and evaluation of search tasks for IIR experiments using a cognitive complexity framework. In Proceedings of the 2015 International Conference on Theory of Information Retrieval - ICTIR '15, 101--110. ACM Press, 101--110. DOI:https://doi.org/10.1145/2808194.2809465Google ScholarDigital Library
- Jeonghyun Kim. 2006. Task difficulty as a predictor and indicator of web searching interaction. In CHI '06 extended abstracts on Human factors in computing systems - CHI EA '06, ACM Press, Montreal, QC Canada, 959. DOI:https://doi.org/10.1145/1125451.1125636Google Scholar
- Walter Kintsch. 1998. Comprehension: a paradigm for cognition. Cambridge University Press, Cambridge; New York, NY, USA.Google Scholar
- Walter Kintsch and Eileen Kintsch. 2005. Comprehension. In Children's Reading Comprehension and Assessment, Scott G. Paris, Steven A. Stahl and Steven A. Stahl (eds.). Routledge. DOI:https://doi.org/10.4324/9781410612762Google Scholar
- David R. Krathwohl. 2002. A revision of bloom's taxonomy: An overview. Theory into Practice (2002), 212--264.Google Scholar
- Yuelin Li and Nicholas Belkin. 2008. A faceted approach to conceptualizing tasks in information seeking. Information Processing & Management 44, 6 (November 2008), 1822--1837.Google ScholarDigital Library
- Jingjing Liu, Nicholas J. Belkin, Xiangmin Zhang, and Xiaojun Yuan. 2013. Examining users' knowledge change in the task completion process. Information Processing & Management 49, 5 (September 2013), 1058--1074. DOI:https://doi.org/10.1016/j.ipm.2012.08.006Google ScholarDigital Library
- Gary Marchionini. 2006. Exploratory search: from finding to understanding. Communications of the ACM 49, 4 (April 2006), 41. DOI:https://doi.org/10.1145/1121949.1121979Google ScholarDigital Library
- Heather L. O'Brien, Paul Cairns, and Mark Hall. 2018. A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. International Journal of Human-Computer Studies 112, (April 2018), 28--39. DOI:https://doi.org/10.1016/j.ijhcs.2018.01.004Google ScholarCross Ref
- Christopher Power, Paul Cairns, Alena Denisova, Themis Papaioannou, and Ruth Gultrom. 2018. Lost at the Edge of Uncertainty: Measuring Player Uncertainty in Digital Games. International Journal of Human--Computer Interaction (September 2018), 1--13. DOI:https://doi.org/10.1080/10447318.2018.1507161Google Scholar
- Soo Young Rieh, Kevyn Collins-Thompson, Preben Hansen, and Hye-Jung Lee. 2016. Towards searching as a learning process: A review of current perspectives and future directions. Journal of Information Science 42, 1 (2016), 19--34. DOI:https://doi.org/10.1177/0165551515615841Google ScholarDigital Library
- Soo Young Rieh, Jacek Gwizdka, Luanne Freund, and Kevyn Collins?Thompson. 2014. Searching as learning: Novel measures for information interaction research. In Proceedings of the American Society for Information Science and Technology 51, 1 (2014), 1--4. DOI:https://doi.org/10.1002/meet.2014.14505101021Google Scholar
- Ryen W. White, Susan T. Dumais, and Jaime Teevan. 2009. Characterizing the influence of domain expertise on web search behavior. In Proceedings of the Second ACM International Conference on Web Search and Data Mining - WSDM '09, ACM Press, Barcelona, Spain, 132. DOI:https://doi.org/10.1145/1498759.1498819Google ScholarDigital Library
- Ryen W. White and Resa A. Roth. 2009. Exploratory search: Beyond the query-response paradigm. Synthesis Lectures on Information Concepts, Retrieval, and Services 1, 1 (January 2009), 1--98. DOI:https://doi.org/10.2200/S00174ED1V01Y200901ICR003Google ScholarCross Ref
- Barbara M. Wildemuth. 2004. The effects of domain knowledge on search tactic formulation. Journal of the American Society for Information Science and Technology 55, 3 (2004), 246--258. DOI:https://doi.org/10.1002/asi.10367Google ScholarDigital Library
- Mathew J. Wilson and Max L. Wilson. 2013. A comparison of techniques for measuring sensemaking and learning within participant-generated summaries. Journal of the American Society for Information Science and Technology 64, 2 (February 2013), 291--306. DOI:https://doi.org/10.1002/asi.22758Google ScholarDigital Library
Index Terms
- The Role of Domain Knowledge in Search as Learning
Recommendations
Searching as Learning: Exploring Search Behavior and Learning Outcomes in Learning-related Tasks
CHIIR '18: Proceedings of the 2018 Conference on Human Information Interaction & RetrievalIn this paper, we investigate the relationship between searching and learning, by conceptualizing information seeking as a learning process, and learning as an outcome of the information seeking process. We present the participants with four search ...
The Effects of Goal-setting on Learning Outcomes and Self-Regulated Learning Processes
CHIIR '24: Proceedings of the 2024 Conference on Human Information Interaction and RetrievalWe present a user study (N = 40) that investigated the role of goal-setting on learning during search. To this end, we developed a tool called the Subgoal Manager (SM). The SM was designed to help searchers break apart a learning-oriented search task ...
Contrasting Search as a Learning Activity with Instructor-designed Learning
CIKM '18: Proceedings of the 27th ACM International Conference on Information and Knowledge ManagementThe field of Search as Learning addresses questions surrounding human learning during the search process. Existing research has largely focused on observing how users with learning-oriented information needs behave and interact with search engines. What ...
Comments