Abstract
Personalised Information Retrieval (PIR) has gained considerable attention in recent literature. In PIR different stages of the retrieval process are adapted to the user, such as adapting the user’s query or the results. Personalised recommender frameworks are endowed with intelligent mechanisms to search for products, goods and services that users are interested in. The objective of such tools is to evaluate and filter the huge amount of information available within a specific scope to assist users in their information access processes. This paper presents a web-based adaptive framework for evaluating personalised information retrieval systems. The framework uses implicit recommendation to guide users in deciding which evaluation techniques, metrics and criteria to use. A task-based experiment was conducted to test the functionality and performance of the framework. A Review of evaluation techniques for personalised IR systems was conducted and the results of the analysed survey are presented.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Agichtein, E., Brill, E., Dumais, S.: Improving Web Search Ranking by Incorporating User Behavior Information. In: 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2006), ACM, Seattle (2006)
Barry, C.L., Schamber, L.: Users’ criteria for relevance evaluation: a cross-situational comparison. Information processing & management 34, 219–236 (1998)
Chirita, P.-A., Firan, C., Nejdl, W.: Personalised Query Expansion for the Web. In: 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2007). ACM, Amsterdam (2007)
Cleverdon, C.W., Mills, J., Keen, E.M.: An inquiry in testing of information retrieval systems (vols. 2) (Cranfield, UK: Aslib Cranfield Research Project, College of Aeronautics) (1966)
Gao, W., Niu, C., Nie, J.-Y., Zhou, D., Hu, J., Wong, K.-F., Hon, H.-W.: Cross-Lingual Query Suggestion Using Query Logs of Different Languages. In: 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2007). ACM, Amsterdam (2007)
Ghorab, M.R., Zhou, D., O’Connor, A., And Wade, V.: Users’ Search History Approach for Personalised Information Retrieval: survey and Classification. Submitted to the Journal of User Modelling and User Adapted Interaction (2011) (Under Review)
Lawless, S., Mulwa, C., O’Connor, A.: A Proposal for the Evaluation of Adaptive Personalised Information Retrieval. In: Proceedings of the 2nd International Workshop on Contextual Information Access, Seeking and Retrieval Evaluation, Milton Keynes, UK. CEUR-WS.org 4, March 28 (2010)
Koutrika, G., Ioannidis, Y.: Rule-based Query Personalised in Digital Libraries. International Journal on Digital Libraries 4, 60–63 (2004)
Micarelli, A., Sciarrone, F.: Anatomy and Empirical Evaluation of an Adaptive Web-Based Information Filtering System. User Modeling and User-Adapted Interaction 14, 159–200 (2004)
Pretschner, A., Gauch, S.: Ontology Based Personalised Search. In: 11th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 1999). IEEE, Chicago (1999)
Pitkow, J., Schutze, H., Cass, T., Cooley, R., Turnbull, D., Edmonds, A., Adar, E., Breuel, T.: Personalised Search. Communications of the ACM 45, 50–55 (2002)
Smyth, B., Balfe, E.: Anonymous Personalised in Collaborative Web Search. Information Retrieval 9, 165–190 (2006)
Speretta, M., Gauch, S.: Misearch. In: IEEE/WIC/ACM International Conference on Web Intelligence (WI 2005), Compiegne University of Technology. IEEE Computer Society, France (2005a)
Stamou, S., Ntoulas, A.: Search Personalised Through Query and Page Topical Analysis. User Modeling and User-Adapted Interaction 19, 5–33 (2009)
Stefani, A., Strapparava, C.: Exploiting NLP Techniques to Build User Model for Web Sites: the Use of WordNet in SiteIF Project. In: 2nd Workshop on Adaptive Systems and User Modeling on the World Wide Web, Toronto, Canada (1999)
Sugiyama, K., Hatano, K., Yoshikawa, M.: Adaptive Web Search Based on User Profile Constructed without Any Effort from Users. In: 13th International Conference on World Wide Web. ACM, New York (2004)
Teevan, J., Dumais, S.T., Horvitz, E.: Personalizing Search via Automated Analysis of Interests and Activities. In: 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2005). ACM, Salvador (2005)
Tobar, C.M.: Yet another evaluation framework. In: Second Workshop on Empirical Evaluation of Adaptive Systems is part of the 9th International Conference on User Modeling (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mulwa, C., Lawless, S., Ghorab, M.R., O’Donnell, E., Sharp, M., Wade, V. (2011). A Framework for the Evaluation of Adaptive IR Systems through Implicit Recommendation. In: Andrews, S., Polovina, S., Hill, R., Akhgar, B. (eds) Conceptual Structures for Discovering Knowledge. ICCS 2011. Lecture Notes in Computer Science(), vol 6828. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22688-5_32
Download citation
DOI: https://doi.org/10.1007/978-3-642-22688-5_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-22687-8
Online ISBN: 978-3-642-22688-5
eBook Packages: Computer ScienceComputer Science (R0)