Abstract
User clicks—also known as clickthrough data—have been cited as an implicit form of relevance feedback. Previous work suggests that relative preferences between documents can be accurately derived from user clicks. In this paper, we analyze the impact of document reordering—based on clickthrough—on search effectiveness, measured using both TREC and user relevance judgments. We also propose new strategies for document reordering that can outperform current techniques. Preliminary results show that current reordering methods do not lead to consistent improvements of search quality, but may even lead to poorer results if not used with care.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Agichtein, E., Brill, E., Dumais, S., Ragno, R.: Learning user interaction models for predicting web search result preferences. In: Efthimiadis, et al. (eds.) [6], pp. 3–10
Bailey, P., Craswell, N., Hawking, D.: Engineering a multi-purpose test collection for web retrieval experiments. IPM 39(6), 853–871 (2003)
Borlund, P.: The concept of relevance in IR. JASIS 54(10), 913–925 (2003)
Bruce, H.: A cognitive view of the situational dynamism of user-centered relevance estimation. JASIS 45(3), 142–148 (1994)
Craswell, N., Szummer, M.: Random walks on the click graph. In: Proc. ACM SIGIR, Amsterdam, The Netherlands, pp. 239–246 (2007)
Efthimiadis, E., Dumais, S., Hawking, D., Järvelin, K. (eds.): Proc. ACM SIGIR, Seattle, WA (2006)
Fox, S., Karnawat, K., Mydland, M., Dumais, S., White, T.: Evaluating implicit measures to improve web search. ACM TOIS 23(2), 147–168 (2005)
Joachims, T.: Optimizing search engines using clickthrough data. In: Proc. ACM SIGKDD, Edmonton, Alberta, Canada, pp. 133–142. ACM Press, New York (2002)
Joachims, T., Granka, L., Pan, B., Hembrooke, H., Radlinski, F., Gay, G.: Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search. ACM TOIS 25(2), 7 (2007)
Joachims, T., Radlinski, F.: Search engines that learn from implicit feedback. IEEE Computer 40(8), 34–40 (2007)
Kemp, C., Ramamohanarao, K.: Long-term learning for web search engines. In: Proceedings of the 6th European Conference on Principles of Data Mining and Knowledge Discovery, pp. 263–274. Springer, London (2002)
Radlinski, F., Joachims, T.: Query chains: learning to rank from implicit feedback. In: Proc. ACM SIGKDD, Chicago, Illinois, pp. 239–248 (2005)
Radlinski, F., Joachims, T.: Active exploration for learning rankings from clickthrough data. In: Proc. ACM SIGKDD, San Jose, California, pp. 570–579 (2007)
Turpin, A., Scholer, F.: User performance versus precision measures for simple search tasks. In: Efthimiadis, et al. (eds.) [6], pp. 11–18
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Shokouhi, M., Scholer, F., Turpin, A. (2008). Investigating the Effectiveness of Clickthrough Data for Document Reordering. In: Macdonald, C., Ounis, I., Plachouras, V., Ruthven, I., White, R.W. (eds) Advances in Information Retrieval. ECIR 2008. Lecture Notes in Computer Science, vol 4956. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78646-7_61
Download citation
DOI: https://doi.org/10.1007/978-3-540-78646-7_61
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-78645-0
Online ISBN: 978-3-540-78646-7
eBook Packages: Computer ScienceComputer Science (R0)