Kun Lin
Nasim Sonboli
Robin Burke
ABSTRACT
Recommender systems learn from past user preferences in order to predict future user interests and provide users with personalized suggestions. Previous research has demonstrated that biases in user profiles in the aggregate can influence the recommendations to users who do not share the majority preference. One consequence of this bias propagation effect is miscalibration, a mismatch between the types or categories of items that a user prefers and the items provided in recommendations. In this paper, we conduct a systematic analysis aimed at identifying key characteristics in user profiles that might lead to miscalibrated recommendations. We consider several categories of profile characteristics, including similarity to the average user, propensity towards popularity, profile diversity, and preference intensity. We develop predictive models of miscalibration and use these models to identify the most important features correlated with miscalibration, given different algorithms and dataset characteristics. Our analysis is intended to help system designers predict miscalibration effects and to develop recommendation algorithms with improved calibration properties.
- Himan Abdollahpouri, Robin Burke, and Bamshad Mobasher. 2017. Controlling popularity bias in learning-to-rank recommendation. In Proceedings of the Eleventh ACM Conference on Recommender Systems. ACM, 42--46.Google Scholar
Digital Library
- Himan Abdollahpouri, Masoud Mansoury, Robin Burke, and Bamshad Mobasher. 2019. The impact of popularity bias on fairness and calibration in recommendation. arXiv preprint arXiv:1910.05755 (2019).Google Scholar
- Alex Beutel, Jilin Chen, Tulsee Doshi, Hai Qian, Li Wei, Yi Wu, Lukasz Heldt, Zhe Zhao, Lichan Hong, Ed H Chi, et almbox. 2019. Fairness in recommendation ranking through pairwise comparisons. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 2212--2220.Google ScholarDigital Library
- Erik Brynjolfsson, Yu Jeffrey Hu, and Michael D Smith. 2006. From niches to riches: Anatomy of the long tail. Sloan Management Review, Vol. 47, 4 (2006), 67--71.Google Scholar
- Robin Douglas Burke, Masoud Mansoury, and Nasim Sonboli. 2020. Experimentation with Fairness-Aware Recommendation Using Librec-Auto: Hands-on Tutorial. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* '20). Association for Computing Machinery, New York, NY, USA, 700. https://doi.org/10.1145/3351095.3375670Google ScholarDigital Library
- Toon Calders and Sicco Verwer. 2010. Three naive Bayes approaches for discrimination-free classification. Data Mining and Knowledge Discovery, Vol. 21, 2 (2010), 277--292.Google ScholarDigital Library
- Òscar Celma and Pedro Cano. 2008. From hits to niches?: or how popular artists can bias music recommendation and discovery. In Proceedings of the 2nd KDD Workshop on Large-Scale Recommender Systems and the Netflix Prize Competition. ACM, 5.Google ScholarDigital Library
- Cynthia S Crowson, Elizabeth J Atkinson, and Terry M Therneau. 2016. Assessing calibration of prognostic risk scores. Statistical methods in medical research, Vol. 25, 4 (2016), 1692--1706.Google Scholar
- Michael D Ekstrand, Mucun Tian, Ion Madrazo Azpiazu, Jennifer D Ekstrand, Oghenemaro Anuyah, David McNeill, and Maria Soledad Pera. 2018a. All the cool kids, how do they fit in?: Popularity and demographic biases in recommender evaluation and effectiveness. In Conference on Fairness, Accountability and Transparency. 172--186.Google Scholar
- Michael D Ekstrand, Mucun Tian, Mohammed R Imran Kazi, Hoda Mehrpouyan, and Daniel Kluver. 2018b. Exploring author gender in book rating and recommendation. In Proceedings of the 12th ACM Conference on Recommender Systems. ACM, 242--250.Google ScholarDigital Library
- Farzad Eskandanian, Bamshad Mobasher, and Robin Burke. 2017. A clustering approach for personalizing diversity in collaborative recommender systems. In Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization. ACM, 280--284.Google ScholarDigital Library
- Farzad Eskandanian, Bamshad Mobasher, and Robin D Burke. 2016. User Segmentation for Controlling Recommendation Diversity.. In RecSys Posters.Google Scholar
- Farzad Eskandanian, Nasim Sonboli, and Bamshad Mobasher. 2019. Power of the Few: Analyzing the Impact of Influential Users in Collaborative Recommender Systems. In Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization. 225--233.Google ScholarDigital Library
- Jerome Friedman, Trevor Hastie, and Robert Tibshirani. 2001. The elements of statistical learning. Vol. 1. Springer series in statistics New York.Google Scholar
- Guibing Guo, Jie Zhang, Zhu Sun, and Neil Yorke-Smith. 2015. LibRec: A Java Library for Recommender Systems.. In UMAP Workshops, Vol. 4.Google Scholar
- Moritz Hardt, Eric Price, Nati Srebro, et almbox. 2016. Equality of opportunity in supervised learning. In Advances in neural information processing systems. 3315--3323.Google ScholarDigital Library
- Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative Filtering for Implicit Feedback Datasets.. In ICDM, Vol. 8. Citeseer, 263--272.Google ScholarDigital Library
- Dietmar Jannach, Lukas Lerche, Fatih Gedikli, and Geoffray Bonnin. 2013. What recommenders recommend--an analysis of accuracy, popularity, and sales diversity effects. In International conference on user modeling, adaptation, and personalization. Springer, 25--37.Google ScholarCross Ref
- Dietmar Jannach, Lukas Lerche, Iman Kamehkhosh, and Michael Jugovac. 2015. What recommenders recommend: an analysis of recommendation biases and possible countermeasures. User Modeling and User-Adapted Interaction, Vol. 25, 5 (2015), 427--491.Google ScholarDigital Library
- Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. 2014. Correcting Popularity Bias by Enhancing Recommendation Neutrality.. In RecSys Posters.Google Scholar
- Caitlin Kuhlman, MaryAnn VanValkenburg, and Elke Rundensteiner. 2019. FARE: Diagnostics for Fair Ranking using Pairwise Error Metrics. In The World Wide Web Conference. 2936--2942.Google ScholarDigital Library
- Solomon Kullback. 1997. Information theory and statistics. Courier Corporation.Google ScholarDigital Library
- Matevvz Kunaver and Tomavz Povz rl. 2017. Diversity in recommender systems--A survey. Knowledge-Based Systems, Vol. 123 (2017), 154--162.Google ScholarDigital Library
- Kun Lin, Nasim Sonboli, Bamshad Mobasher, and Robin Burke. 2019. Crank up the volume: preference bias amplification in collaborative recommendation. arXiv preprint arXiv:1909.06362 (2019).Google Scholar
- Masoud Mansoury, Himan Abdollahpouri, Jessie Smith, Arman Dehpanah, Mykola Pechenizkiy, and Bamshad Mobasher. 2020. Investigating Potential Factors Associated with Gender Discrimination in Collaborative Recommender Systems. arXiv preprint arXiv:2002.07786 (2020).Google Scholar
- Masoud Mansoury, Bamshad Mobasher, Robin Burke, and Mykola Pechenizkiy. 2019. Bias disparity in collaborative recommendation: Algorithmic evaluation and comparison. arXiv preprint arXiv:1908.00831 (2019).Google Scholar
- Tien T Nguyen, Pik-Mai Hui, F Maxwell Harper, Loren Terveen, and Joseph A Konstan. 2014. Exploring the filter bubble: the effect of using recommender systems on content diversity. In Proceedings of the 23rd international conference on World wide web. 677--686.Google ScholarDigital Library
- Eli Pariser. 2011. The filter bubble: How the new personalized web is changing what we read and how we think. Penguin.Google ScholarDigital Library
- Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. 2017. On fairness and calibration. In Advances in Neural Information Processing Systems. 5680--5689.Google Scholar
- Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence. AUAI Press, 452--461.Google ScholarDigital Library
- Claude E Shannon. 1951. Prediction and entropy of printed English. Bell system technical journal, Vol. 30, 1 (1951), 50--64.Google Scholar
- Harald Steck. 2018. Calibrated recommendations. In Proceedings of the 12th ACM conference on recommender systems. ACM, 154--162.Google ScholarDigital Library
- Virginia Tsintzou, Evaggelia Pitoura, and Panayiotis Tsaparas. 2018. Bias Disparity in Recommendation Systems. arXiv preprint arXiv:1811.01461 (2018).Google Scholar
- Sirui Yao and Bert Huang. 2017. Beyond parity: Fairness objectives for collaborative filtering. In Advances in Neural Information Processing Systems. 2921--2930.Google Scholar
- Hongzhi Yin, Bin Cui, Jing Li, Junjie Yao, and Chen Chen. 2012. Challenging the long tail recommendation. arXiv preprint arXiv:1205.6700 (2012).Google Scholar
- Cai-Nicolas Ziegler, Sean M McNee, Joseph A Konstan, and Georg Lausen. 2005. Improving recommendation lists through topic diversification. In Proceedings of the 14th international conference on World Wide Web. 22--32.Google ScholarDigital Library
Index Terms
- Calibration in Collaborative Filtering Recommender Systems: a User-Centered Analysis
Recommendations
The Connection Between Popularity Bias, Calibration, and Fairness in Recommendation
RecSys '20: Proceedings of the 14th ACM Conference on Recommender SystemsRecently there has been a growing interest in fairness-aware recommender systems including fairness in providing consistent performance across different users or groups of users. A recommender system could be considered unfair if the recommendations do ...
User-centered Evaluation of Popularity Bias in Recommender Systems
UMAP '21: Proceedings of the 29th ACM Conference on User Modeling, Adaptation and PersonalizationRecommendation and ranking systems are known to suffer from popularity bias; the tendency of the algorithm to favor a few popular items while under-representing the majority of other items. Prior research has examined various approaches for mitigating ...
Bias disparity in graph-based collaborative filtering recommenders
SAC '22: Proceedings of the 37th ACM/SIGAPP Symposium on Applied ComputingIn collaborative filtering (CF) recommendations, user preferences may be reinforced resulting in the under-representation of specific categories of items in the recommendation lists received by users belonging to specific groups. We call this phenomenon ...
Comments