Abstract
In this article we propose the concept of relevance criteria profiles, which provide a global view of user behaviour in judging the relevance of retrieved information. We further propose a plotting technique which provides a session based overview of the relevance judgement processes interlaced with interactions that allow the researcher to visualise and quickly detect emerging patterns in both interactions and relevance criteria usage. We discuss by example, using data from a user study conducted between the months of January and August of 2008, how these tools support the better understanding of task based user valuation of documents that is likely to lead to recommendations for improving end-user services in digital libraries.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Barry, C.L.: User-defined relevance criteria: an exploratory study. Journal of the American Society for Information Science 45(3), 149–159 (1994)
Barry, C.L., Schamber, L.: Users’criteria for relevance evaluation: A cross-situational comparison. Information Processing and Management 34(2-3), 219–236 (1998)
Borlund, P.: The concept of relevance in IR. Journal of the American Society for Information Science and Technology 54(10), 913–925 (2003)
Borlund, P.: The IIR evaluation model: a framework for evaluation of interactive information retrieval systems. Information Research 8(3), 8–13 (2003)
Cleverdon, C.W., Mills, J., Keen, E.M.: Factors Determining the Performance of Indexing Systems. Design, vol. 1., Test Results, vol. 2. Aslib Cranfield Research Project, Cranfield, Lagland (1966)
Ericsson, K.A., Simon, H.A.: Protocol analysis: verbal reports as data. MIT Press, Cambridge (1993)
Hurvich, L.M., Jameson, D.: An opponent-process theory of color vision. Psychological Review 64, 384–404 (1957)
Kullback, S., Leibler, R.A.: On information and sufficiency. Annals of Mathematical Statistics 22, 79–86 (1951)
Lin, J.: Divergence measures based on the shannon entropy. IEEE Transactions on Information theory 37, 145–151 (1991)
Robertson, S.E., Hancock-Beaulieu, M.M.: On the evaluation of ir systems. Information Processing Management 28(4), 457–466 (1992)
Schamber, L.: Users’criteria for evaluation in a multimedia environment. Proceedings of the 54 Annual Meeting of the American Society for Information Science 28, 126–133 (1991)
Wang, P., White, M.D.: A cognitive model of document use during a research project. study ii. decisions at the reading and citing stages. Journal of the American Society of Information Sciences 50(2), 98–114 (1999)
Ware, C.: Color sequences for univariate maps: Theory, experiments and principles. IEEE Computer Graphics and Applications 8(5), 41–49 (1988)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cerviño Beresi, U., Kim, Y., Song, D., Ruthven, I., Baillie, M. (2010). Relevance in Technicolor. In: Lalmas, M., Jose, J., Rauber, A., Sebastiani, F., Frommholz, I. (eds) Research and Advanced Technology for Digital Libraries. ECDL 2010. Lecture Notes in Computer Science, vol 6273. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15464-5_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-15464-5_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15463-8
Online ISBN: 978-3-642-15464-5
eBook Packages: Computer ScienceComputer Science (R0)