skip to main content
10.1145/3511808.3557536acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
short-paper
Open Access

A Multi-Domain Benchmark for Personalized Search Evaluation

Authors Info & Claims
Published:17 October 2022Publication History

ABSTRACT

Personalization in Information Retrieval has been a hot topic in both academia and industry for the past two decades. However, there is still a lack of high-quality standard benchmark datasets for conducting offline comparative evaluations in this context. To mitigate this problem, in the past few years, approaches to derive synthetic datasets suited for evaluating Personalized Search models have been proposed. In this paper, we put forward a novel evaluation benchmark for Personalized Search with more than 18 million documents and 1.9 million queries across four domains. We present a detailed description of the benchmark construction procedure, highlighting its characteristics and challenges. We provide baseline performance including pre-trained neural models, opening room for the evaluation of personalized approaches, as well as domain adaptation and transfer learning scenarios. We make both datasets and models available for future research.

Skip Supplemental Material Section

Supplemental Material

CIKM-sp0074.mp4

mp4

133.3 MB

References

  1. Eytan Adar. 2007. User 4xxxxx9: Anonymizing query logs. In Proc of Query Log Analysis Workshop, International Conference on World Wide Web.Google ScholarGoogle Scholar
  2. Qingyao Ai, Daniel N. Hill, S. V. N. Vishwanathan, and W. Bruce Croft. 2019. A Zero Attention Model for Personalized Product Search. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3--7, 2019. ACM, 379--388.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Qingyao Ai, Yongfeng Zhang, Keping Bi, Xu Chen, and W. Bruce Croft. 2017. Learning a Hierarchical Embedding Model for Personalized Product Search. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Shinjuku, Tokyo, Japan, August 7--11, 2017. ACM, 645--654.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4--8, 2019. ACM, 2623--2631.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. James Allan. 2003. HARD Track Overviewin TREC 2003: High Accuracy Retrieval from Documents. In Proceedings of The Twelfth Text REtrieval Conference, TREC 2003, Gaithersburg, Maryland, USA, November 18--21, 2003 (NIST Special Publication, Vol. 500--255). National Institute of Standards and Technology (NIST), 24--37.Google ScholarGoogle Scholar
  6. James Allan. 2004. HARD Track Overview in TREC 2004 - High Accuracy Retrieval from Documents. In Proceedings of the Thirteenth Text REtrieval Conference, TREC 2004, Gaithersburg, Maryland, USA, November 16--19, 2004 (NIST Special Publication, Vol. 500--261). National Institute of Standards and Technology (NIST).Google ScholarGoogle Scholar
  7. James Allan. 2005. HARD Track Overview in TREC 2005 High Accuracy Retrieval from Documents. In Proceedings of the Fourteenth Text REtrieval Conference, TREC 2005, Gaithersburg, Maryland, USA, November 15--18, 2005 (NIST Special Publication, Vol. 500--266). National Institute of Standards and Technology (NIST).Google ScholarGoogle Scholar
  8. Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, Rodney Kinney, Sebastian Kohlmeier, Kyle Lo, Tyler Murray, Hsu-Han Ooi, Matthew E. Peters, Joanna Power, Sam Skjonsberg, Lucy LuWang, Chris Wilhelm, Zheng Yuan, Madeleine van Zuylen, and Oren Etzioni. 2018. Construction of the Literature Graph in Semantic Scholar. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1--6, 2018, Volume 3 (Industry Papers). Association for Computational Linguistics, 84--91.Google ScholarGoogle ScholarCross RefCross Ref
  9. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7--9, 2015, Conference Track Proceedings.Google ScholarGoogle Scholar
  10. Michael Barbaro, Tom Zeller, and Saul Hansell. 2006. A face is exposed for AOL searcher no. 4417749. New York Times 9, 2008 (2006), 8.Google ScholarGoogle Scholar
  11. Elias Bassani. 2022. ranx: A Blazing-Fast Python Library for Ranking Evaluation and Comparison. In Advances in Information Retrieval - 44th European Conference on IR Research, ECIR 2022, Stavanger, Norway, April 10--14, 2022, Proceedings, Part II (Lecture Notes in Computer Science, Vol. 13186). Springer, 259--264.Google ScholarGoogle Scholar
  12. Elias Bassani and Gabriella Pasi. 2022. A multi-representation re-ranking model for Personalized Product Search. Inf. Fusion 81 (2022), 240--249.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Elias Bassani and Luca Romelli. 2022. ranx.fuse: A Python Library for Metasearch. In Proceedings of the 31st ACM International Conference on Information and Knowledge Management, CIKM 2022, Atlanta, Georgia, USA, October 17--21, 2022. ACM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Keping Bi, Qingyao Ai, and W. Bruce Croft. 2020. A Transformer-based Embedding Model for Personalized Product Search. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25--30, 2020. ACM, 1521--1524.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Keping Bi, Qingyao Ai, and W. Bruce Croft. 2021. Learning a Fine-Grained Review-based Transformer Model for Personalized Product Search. In SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11--15, 2021. ACM, 123--132.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural Language Processing with Python. O'Reilly.Google ScholarGoogle Scholar
  17. Mohamed Reda Bouadjenek, Amyn Bennamane, Hakim Hacid, and Mokrane Bouzeghoub. 2013. Evaluation of Personalized Social Ranking Functions of Information Retrieval. In Web Engineering - 13th International Conference, ICWE 2013, Aalborg, Denmark, July 8--12, 2013. Proceedings (Lecture Notes in Computer Science, Vol. 7977). Springer, 283--290.Google ScholarGoogle Scholar
  18. Mohamed Reda Bouadjenek, Hakim Hacid, and Mokrane Bouzeghoub. 2013. Sopra: a new social personalized ranking function for improving web search. In The 36th International ACM SIGIR conference on research and development in Information Retrieval, SIGIR '13, Dublin, Ireland - July 28 - August 01, 2013. ACM, 861--864.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Mohamed Reda Bouadjenek, Hakim Hacid, Mokrane Bouzeghoub, and Athena Vakali. 2013. Using social annotations to enhance document representation for personalized search. In The 36th International ACM SIGIR conference on research and development in Information Retrieval, SIGIR '13, Dublin, Ireland - July 28 - August 01, 2013. ACM, 1049--1052.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Mohamed Reda Bouadjenek, Hakim Hacid, Mokrane Bouzeghoub, and Athena Vakali. 2016. PerSaDoR: Personalized social document representation for improving web search. Inf. Sci. 369 (2016), 614--633.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Ben Carterette, Ashraf Bah, Evangelos Kanoulas, Mark M. Hall, and Paul D. Clough. 2013. Overview of the TREC 2013 Session Track. In Proceedings of The Twenty-Second Text REtrieval Conference, TREC 2013, Gaithersburg, Maryland, USA, November 19--22, 2013 (NIST Special Publication, Vol. 500--302). National Institute of Standards and Technology (NIST).Google ScholarGoogle Scholar
  22. Ben Carterette, Evangelos Kanoulas, Mark M. Hall, and Paul D. Clough. 2014. Overview of the TREC 2014 Session Track. In Proceedings of The Twenty-Third Text REtrieval Conference, TREC 2014, Gaithersburg, Maryland, USA, November 19--21, 2014 (NIST Special Publication, Vol. 500--308). National Institute of Standards and Technology (NIST).Google ScholarGoogle Scholar
  23. Cyril Cleverdon. 1967. The Cranfield tests on index language devices. In Aslib proceedings. MCB UP Ltd.Google ScholarGoogle Scholar
  24. James R. Clough, Jamie Gollings, Tamar V. Loach, and Tim S. Evans. 2014. Transitive reduction of citation networks. Journal of Complex Networks 3, 2 (09 2014), 189--203. https://doi.org/10.1093/comnet/cnu039 arXiv:https://academic.oup.com/comnet/articlepdf/3/2/189/1071092/cnu039.pdfGoogle ScholarGoogle Scholar
  25. Adriel Dean-Hall, Charles L. A. Clarke, Jaap Kamps, Julia Kiseleva, and Ellen M. Voorhees. 2015. Overview of the TREC 2015 Contextual Suggestion Track. In Proceedings of The Twenty-Fourth Text REtrieval Conference, TREC 2015, Gaithersburg, Maryland, USA, November 17--20, 2015 (NIST Special Publication, Vol. 500--319). National Institute of Standards and Technology (NIST).Google ScholarGoogle Scholar
  26. Adriel Dean-Hall, Charles L. A. Clarke, Jaap Kamps, Paul Thomas, and Ellen M. Voorhees. 2012. Overview of the TREC 2012 Contextual Suggestion Track. In Proceedings of The Twenty-First Text REtrieval Conference, TREC 2012, Gaithersburg, Maryland, USA, November 6--9, 2012 (NIST Special Publication, Vol. 500--298). National Institute of Standards and Technology (NIST).Google ScholarGoogle Scholar
  27. Adriel Dean-Hall, Charles L. A. Clarke, Jaap Kamps, Paul Thomas, and Ellen M. Voorhees. 2014. Overview of the TREC 2014 Contextual Suggestion Track. In Proceedings of The Twenty-Third Text REtrieval Conference, TREC 2014, Gaithersburg, Maryland, USA, November 19--21, 2014 (NIST Special Publication, Vol. 500--308). National Institute of Standards and Technology (NIST).Google ScholarGoogle Scholar
  28. Adriel Dean-Hall, Charles L. A. Clarke, Nicole Simone, Jaap Kamps, Paul Thomas, and Ellen M. Voorhees. 2013. Overview of the TREC 2013 Contextual Suggestion Track. In Proceedings of The Twenty-Second Text REtrieval Conference, TREC 2013, Gaithersburg, Maryland, USA, November 19--22, 2013 (NIST Special Publication, Vol. 500--302). National Institute of Standards and Technology (NIST).Google ScholarGoogle Scholar
  29. Zhicheng Dou, Ruihua Song, and Ji-RongWen. 2007. A large-scale evaluation and analysis of personalized search strategies. In Proceedings of the 16th International Conference on World Wide Web, WWW 2007, Banff, Alberta, Canada, May 8--12, 2007. ACM, 581--590.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Susan T. Dumais, Edward Cutrell, Jonathan J. Cadiz, Gavin Jancke, Raman Sarin, and Daniel C. Robbins. 2003. Stuff I've seen: a system for personal information retrieval and re-use. In SIGIR 2003: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 28 - August 1, 2003, Toronto, Canada. ACM, 72--79.Google ScholarGoogle Scholar
  31. Lu Fan, Qimai Li, Bo Liu, Xiao-MingWu, Xiaotong Zhang, Fuyu Lv, Guli Lin, Sen Li, Taiwei Jin, and Keping Yang. 2022. Modeling User Behavior with Graph Convolution for Personalized Product Search. InWWW'22: The ACMWeb Conference 2022, Virtual Event, Lyon, France, April 25 - 29, 2022. ACM, 203--212.Google ScholarGoogle Scholar
  32. Michael Färber. 2019. The Microsoft Academic Knowledge Graph: A Linked Data Source with 8 Billion Triples of Scholarly Data. In The Semantic Web - ISWC 2019 - 18th International Semantic Web Conference, Auckland, New Zealand, October 26--30, 2019, Proceedings, Part II (Lecture Notes in Computer Science, Vol. 11779). Springer, 113--129.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Debasis Ganguly, Johannes Leveling, and Gareth J. F. Jones. 2011. Overview of the Personalized and Collaborative Information Retrieval (PIR) Track at FIRE-2011. In Multilingual Information Access in South Asian Languages - Second International Workshop, FIRE 2010, Gandhinagar, India, February 19--21, 2010 and Third International Workshop, FIRE 2011, Bombay, India, December 2--4, 2011, Revised Selected Papers (Lecture Notes in Computer Science, Vol. 7536). Springer, 227--240.Google ScholarGoogle Scholar
  34. Luyu Gao, Zhuyun Dai, Tongfei Chen, Zhen Fan, Benjamin Van Durme, and Jamie Callan. 2021. Complement Lexical Retrieval Model with Semantic Residual Embeddings. In Advances in Information Retrieval - 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28 - April 1, 2021, Proceedings, Part I (Lecture Notes in Computer Science, Vol. 12656). Springer, 146--160.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Songwei Ge, Zhicheng Dou, Zhengbao Jiang, Jian-Yun Nie, and Ji-Rong Wen. 2018. Personalizing Search Results Using Hierarchical RNN with Query-aware Attention. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM 2018, Torino, Italy, October 22--26, 2018. ACM, 347--356.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Seyyed Hadi Hashemi, Jaap Kamps, Julia Kiseleva, Charles L. A. Clarke, and Ellen M. Voorhees. 2016. Overview of the TREC 2016 Contextual Suggestion Track. In Proceedings of The Twenty-Fifth Text REtrieval Conference, TREC 2016, Gaithersburg, Maryland, USA, November 15--18, 2016 (NIST Special Publication, Vol. 500--321). National Institute of Standards and Technology (NIST).Google ScholarGoogle Scholar
  37. Jyun-Yu Jiang, TaoWu, Georgios Roumpos, Heng-Tze Cheng, Xinyang Yi, Ed Chi, Harish Ganapathy, Nitin Jindal, Pei Cao, and Wei Wang. 2020. End-to-End Deep Attentive Personalized Item Retrieval for Online Content-sharing Platforms. In WWW '20: The Web Conference 2020, Taipei, Taiwan, April 20--24, 2020. ACM / IW3C2, 2870--2877.Google ScholarGoogle Scholar
  38. Evangelos Kanoulas, Ben Carterette, Mark M. Hall, Paul D. Clough, and Mark Sanderson. 2012. Overview of the TREC 2012 Session Track. In Proceedings of The Twenty-First Text REtrieval Conference, TREC 2012, Gaithersburg, Maryland, USA, November 6--9, 2012 (NIST Special Publication, Vol. 500--298). National Institute of Standards and Technology (NIST).Google ScholarGoogle Scholar
  39. Evangelos Kanoulas, Paul D. Clough, Ben Carterette, and Mark Sanderson. 2010. Overview of the TREC 2010 Session Track. In Proceedings of The Nineteenth Text REtrieval Conference, TREC 2010, Gaithersburg, Maryland, USA, November 16--19, 2010 (NIST Special Publication, Vol. 500--294). National Institute of Standards and Technology (NIST).Google ScholarGoogle Scholar
  40. Evangelos Kanoulas, Mark M. Hall, Paul D. Clough, Ben Carterette, and Mark Sanderson. 2011. Overview of the TREC 2011 Session Track. In Proceedings of The Twentieth Text REtrieval Conference, TREC 2011, Gaithersburg, Maryland, USA, November 15--18, 2011 (NIST Special Publication, Vol. 500--296). National Institute of Standards and Technology (NIST).Google ScholarGoogle Scholar
  41. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick S. H. Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16--20, 2020. Association for Computational Linguistics, 6769--6781.Google ScholarGoogle ScholarCross RefCross Ref
  42. Robert Krovetz. 1993. Viewing Morphology as an Inference Process. In Proceedings of the 16th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval. Pittsburgh, PA, USA, June 27 - July 1, 1993. ACM, 191--202.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019. Latent Retrieval for Weakly Supervised Open Domain Question Answering. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers. Association for Computational Linguistics, 6086--6096.Google ScholarGoogle ScholarCross RefCross Ref
  44. Julian J. McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. 2015. Image-Based Recommendations on Styles and Substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, Santiago, Chile, August 9--13, 2015. ACM, 43--52.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Alessandro Micarelli, Fabio Gasparetti, Filippo Sciarrone, and Susan Gauch. 2007. Personalized Search on the World Wide Web. In The Adaptive Web, Methods and Strategies of Web Personalization (Lecture Notes in Computer Science, Vol. 4321). Springer, 195--230.Google ScholarGoogle Scholar
  46. Paul Over. 2001. The TREC interactive track: an annotated bibliography. Inf. Process. Manag. 37, 3 (2001), 369--381.Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Yaoxin Pan, Shangsong Liang, Jiaxin Ren, Zaiqiao Meng, and Qiang Zhang. 2022. Personalized, Sequential, Attentive, Metric-Aware Product Search. ACM Trans. Inf. Syst. 40, 2 (2022), 36:1--36:29.Google ScholarGoogle Scholar
  48. Gabriella Pasi, Gareth J. F. Jones, Keith Curtis, Stefania Marrara, Camilla Sanvitto, Debasis Ganguly, and Procheta Sen. 2018. Overview of the CLEF 2018 Personalised Information Retrieval Lab (PIR-CLEF 2018). In Working Notes of CLEF 2018 - Conference and Labs of the Evaluation Forum, Avignon, France, September 10--14, 2018 (CEUR Workshop Proceedings, Vol. 2125). CEUR-WS.org.Google ScholarGoogle Scholar
  49. Gabriella Pasi, Gareth J. F. Jones, Lorraine Goeuriot, Liadh Kelly, Stefania Marrara, and Camilla Sanvitto. 2019. Overview of the CLEF 2019 Personalised Information Retrieval Lab (PIR-CLEF 2019). In Experimental IR Meets Multilinguality, Multimodality, and Interaction - 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9--12, 2019, Proceedings (Lecture Notes in Computer Science, Vol. 11696). Springer, 417--424.Google ScholarGoogle Scholar
  50. Gabriella Pasi, Gareth J. F. Jones, Stefania Marrara, Camilla Sanvitto, Debasis Ganguly, and Procheta Sen. 2017. Overview of the CLEF 2017 Personalised Information Retrieval Pilot Lab (PIR-CLEF 2017). In Experimental IR Meets Multilinguality, Multimodality, and Interaction - 8th International Conference of the CLEF Association, CLEF 2017, Dublin, Ireland, September 11--14, 2017, Proceedings (Lecture Notes in Computer Science, Vol. 10456). Springer, 338--345.Google ScholarGoogle Scholar
  51. Greg Pass, Abdur Chowdhury, and Cayley Torgeson. 2006. A picture of search. In Proceedings of the 1st International Conference on Scalable Information Systems, Infoscale 2006, Hong Kong, May 30-June 1, 2006 (ACM International Conference Proceeding Series, Vol. 152). ACM, 1.Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Alexander Pretschner and Susan Gauch. 1999. Ontology Based Personalized Search. In 11th IEEE International Conference on Tools with Artificial Intelligence, ICTAI '99, Chicago, Illinois, USA, November 8--10, 1999. IEEE Computer Society, 391--398.Google ScholarGoogle Scholar
  53. Filip Radlinski and Susan T. Dumais. 2006. Improving personalized web search using result diversification. In SIGIR 2006: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Seattle, Washington, USA, August 6--11, 2006. ACM, 691--692.Google ScholarGoogle Scholar
  54. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3--7, 2019. Association for Computational Linguistics, 3980--3990.Google ScholarGoogle ScholarCross RefCross Ref
  55. Stephen E. Robertson and Steve Walker. 1994. Some Simple Effective Approximations to the 2-Poisson Model for Probabilistic Weighted Retrieval. In Proceedings of the 17th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval. Dublin, Ireland, 3--6 July 1994 (Special Issue of the SIGIR Forum). ACM/Springer, 232--241.Google ScholarGoogle Scholar
  56. Xuehua Shen, Bin Tan, and ChengXiang Zhai. 2005. Implicit user modeling for personalized search. In Proceedings of the 2005 ACM CIKM International Conference on Information and Knowledge Management, Bremen, Germany, October 31 - November 5, 2005. ACM, 824--831.Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Arnab Sinha, Zhihong Shen, Yang Song, Hao Ma, Darrin Eide, Bo-June (Paul) Hsu, and Kuansan Wang. 2015. An Overview of Microsoft Academic Service (MAS) and Applications. In Proceedings of the 24th International Conference on World Wide Web (Florence, Italy) (WWW '15 Companion). Association for Computing Machinery, New York, NY, USA, 243--246.Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Mirco Speretta and Susan Gauch. 2005. Personalized Search Based on User Search Histories. In 2005 IEEE / WIC / ACM International Conference on Web Intelligence (WI 2005), 19--22 September 2005, Compiegne, France. IEEE Computer Society, 622--628.Google ScholarGoogle Scholar
  59. Shayan A. Tabrizi, Azadeh Shakery, Hamed Zamani, and Mohammad Ali Tavallaei. 2018. PERSON: Personalized information retrieval evaluation based on citation networks. Inf. Process. Manag. 54, 4 (2018), 630--656.Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. 2008. Arnet-Miner: extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, Nevada, USA, August 24--27, 2008. ACM, 990--998.Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Jaime Teevan, Susan T. Dumais, and Eric Horvitz. 2005. Personalizing search via automated analysis of interests and activities. In SIGIR 2005: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Salvador, Brazil, August 15--19, 2005. ACM, 449--456.Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Jaime Teevan, Susan T. Dumais, and Daniel J. Liebling. 2008. To personalize or not to personalize: modeling queries with variation in user intent. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2008, Singapore, July 20--24, 2008. ACM, 163--170.Google ScholarGoogle Scholar
  63. QihuaWang and Hongxia Jin. 2010. Exploring online social activities for adaptive search personalization. In Proceedings of the 19th ACM Conference on Information and Knowledge Management, CIKM 2010, Toronto, Ontario, Canada, October 26--30, 2010. ACM, 999--1008.Google ScholarGoogle Scholar
  64. Shengliang Xu, Shenghua Bao, Ben Fei, Zhong Su, and Yong Yu. 2008. Exploring folksonomy for personalized search. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2008, Singapore, July 20--24, 2008. ACM, 155--162.Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Hamed Zamani and W. Bruce Croft. 2020. Learning a Joint Search and Recommendation Model from User-Item Interactions. In WSDM '20: The Thirteenth ACM International Conference on Web Search and Data Mining, Houston, TX, USA, February 3--7, 2020. ACM, 717--725.Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Dong Zhou, Séamus Lawless, and Vincent Wade. 2012. Improving search via personalized query expansion using social media. Inf. Retr. 15, 3--4 (2012), 218--242.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A Multi-Domain Benchmark for Personalized Search Evaluation

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Conferences
        CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management
        October 2022
        5274 pages
        ISBN:9781450392365
        DOI:10.1145/3511808
        • General Chairs:
        • Mohammad Al Hasan,
        • Li Xiong

        Copyright © 2022 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 17 October 2022

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • short-paper

        Acceptance Rates

        CIKM '22 Paper Acceptance Rate621of2,257submissions,28%Overall Acceptance Rate1,861of8,427submissions,22%

        Upcoming Conference

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader