skip to main content
10.1145/2362724.2362768acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiiixConference Proceedingsconference-collections
research-article

Grannies, tanning beds, tattoos and NASCAR: evaluation of search tasks with varying levels of cognitive complexity

Authors Info & Claims
Published:21 August 2012Publication History

ABSTRACT

One of the most challenging aspects of designing an interactive information retrieval (IIR) study is the development of search tasks. In this paper, we present preliminary results of a study designed to evaluate a set of search tasks that were developed for use in IIR studies. We created 20 search tasks using five levels of cognitive complexity and four domains, and conducted a laboratory evaluation of these tasks with 48 undergraduate subjects. We describe preliminary results from an analysis of data from 24 subjects for 10 search tasks. Initial results show that, in general, as cognitive complexity increased, subjects issued more queries, clicked on more search results, viewed more URLs and took more time to complete the task. Subjects' expected and experienced difficulty ratings of tasks generally increased as cognitive complexity increased with some exceptions. When subjects were asked to rank tasks according to difficulty and engagement, tasks with higher cognitive complexity were rated as more difficult than tasks with lower cognitive complexity, but not necessarily as more engaging. These preliminary results suggest that behaviors and ratings are fairly consistent with the differences one might expect among the search tasks and provide initial evidence of the usefulness of these tasks in IIR studies.

References

  1. Anderson, L. W. & Krathwohl, D. A. (2001). A taxonomy for learning, teaching and assessing: A revision of Bloom's taxonomy of educational objectives. New York: Longman.Google ScholarGoogle Scholar
  2. Arguello, J., Wu, W. C., Kelly, D., & Edwards, A. (2012). Task complexity, vertical display and user interaction in aggregated search. Proceedings of SIGIR '12, Portland, OR. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Borlund, P. (2003). The IIR evaluation model: A framework for the evaluation of interactive information retrieval systems. Information Research, 8(3), paper 152.Google ScholarGoogle Scholar
  4. Byström, K. & Jäärvelin, K. (1995). Task complexity affects information seeking and use. IP&M, 31(2), 191--213. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Jansen, B. J., Booth, D., & Smith, D. (2009). Using the taxonomy of cognitive learning to model online searching. IP&M, 45, 643--663. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Li, Y. & Belkin, N. J. (2008). A faceted approach to conceptualizing tasks in information seeking. IP&M, 44, 1822--1837. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Toms, E. G. (2011). Task-based information searching and retrieval. In I. Ruthven & D. Kelly (Eds.) Interactive Information Seeking, Behaviour and Retrieval. Facet Publishing, 43--59.Google ScholarGoogle Scholar
  8. Vakkari, P. (2003). Task-based information searching. ARIST, 37, 413--464.Google ScholarGoogle Scholar
  9. Whittaker, S., Terveen, L., & Nardi, B. (2000). Let's stop pushing the envelope and start addressing it: a reference task agenda for HCI. Human Computer Interaction, 15, 75--106. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Wildemuth, B. M. & Freund, L. (2009) Search tasks and their role in studies of search behaviors. Paper presented at the Third Annual Workshop on HCIR, Washington D.C.Google ScholarGoogle Scholar

Index Terms

  1. Grannies, tanning beds, tattoos and NASCAR: evaluation of search tasks with varying levels of cognitive complexity

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader