Abstract
We study a specific case of personalized advertisement recommendation (PAR) problem, which consist of a user visiting a system (website) and the system displaying one of K ads to the user. The system uses an internal ad recommendation policy to map the user’s profile (context) to one of the ads. The user either clicks or ignores the ad and correspondingly, the system updates its recommendation policy. The policy space in large scale PAR systems are generally based on classifiers. A practical problem in PAR is extreme click sparsity, due to very few users actually clicking on ads. We systematically study the drawback of using classifier-based policies, in face of extreme click sparsity. We then suggest an alternate policy, based on rankers, learnt by optimizing the Area Under the Curve (AUC) ranking loss, which can significantly alleviate the problem of click sparsity. We create deterministic and stochastic policy spaces and conduct extensive experiments on public and proprietary datasets to illustrate the improvement in click-through-rate (CTR) obtained by using the ranker-based policy over classifier-based policy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Agarwal, A., Hsu, D., Kale, S., Langford, J., Li, L., Schapire, R.: Taming the monster: a fast and simple algorithm for contextual bandits. In: Proceedings of the 31st International Conference on Machine Learning, pp. 1638–1646 (2014)
Agarwal, D., Gabrilovich, E., Hall, R., Josifovski, V., Khanna, R.: Translating relevance scores to probabilities for contextual advertising. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management, pp. 1899–1902. ACM (2009)
Beygelzimer, A., Langford, J., Ravikumar, P.: Multiclass classification with filter trees. Preprint, 2 June 2007
Calders, T., Jaroszewicz, S.: Efficient AUC optimization for classification. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) PKDD 2007. LNCS (LNAI), vol. 4702, pp. 42–53. Springer, Heidelberg (2007). doi:10.1007/978-3-540-74976-9_8
Cao, P., Zhao, D., Zaiane, O.: An optimized cost-sensitive SVM for imbalanced data learning. In: Pei, J., Tseng, V.S., Cao, L., Motoda, H., Xu, G. (eds.) PAKDD 2013. LNCS, vol. 7819, pp. 280–292. Springer, Heidelberg (2013). doi:10.1007/978-3-642-37456-2_24
Chapelle, O., Li, L.: An empirical evaluation of thompson sampling. In: Advances in Neural Information Processing Systems, pp. 2249–2257 (2011)
Chawla, N., Japkowicz, N., Kotcz, A.: Editorial: special issue on learning from imbalanced data sets. ACM SIGKDD Explor. Newsl. 6(1), 1–6 (2004)
Cheng, T., Wang, Y., Bryant, S.: FSelector: a ruby gem for feature selection. Bioinformatics 28(21), 2851–2852 (2012)
Cortes, C., Mohri, M.: AUC optimization vs. error rate minimization. Adv. Neural Inf. Proces. Syst. 16(16), 313–320 (2004)
Dudik, M., Hsu, D., Kale, S., Karampatziakis, N., Langford, J., Reyzin, L., Zhang, T.: Efficient optimal learning for contextual bandits. In: Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence (2011)
He, X., et al.: Practical lessons from predicting clicks on ads at facebook. In: Proceedings of 20th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1–9. ACM (2014)
Japkowicz, N., Stephen, S.: The class imbalance problem: a systematic study. Intell. Data Anal. 6(5), 429–449 (2002)
Koh, E., Gupta, N.: An empirical evaluation of ensemble decision trees to improve personalization on advertisement. In: Proceedings of KDD 14 Second Workshop on User Engagement Optimization (2014)
Langford, J., Li, L., Dudik, M.: Doubly robust policy evaluation and learning. In: Proceedings of the 28th International Conference on Machine Learning, pp. 1097–1104 (2011)
Langford, J., Zhang, T.: The epoch-greedy algorithm for multi-armed bandits with side information. In: Advances in Neural Information Processing Systems, pp. 817–824 (2008)
Li, L., Chu, W., Langford, J., Wang, X.: Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms. In: Proceedings of the fourth ACM International Conference on Web Search and Data Mining, pp. 297–306. ACM (2011)
McMahan, H., et al.: Ad click prediction: a view from the trenches. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1222–1230. ACM (2013)
Richardson, M., Dominowska, E., Ragno, R.: Predicting clicks: estimating the click-through rate for new ads. In: Proceedings of the 16th International Conference on World Wide Web, pp. 521–530. ACM (2007)
Rifkin, R., Klautau, A.: In defense of one-vs-all classification. J. Mach. Learn. Res. 5, 101–141 (2004)
Shamir, O., Zhang, T.: Stochastic gradient descent for non-smooth optimization: convergence results and optimal averaging schemes. In: Proceedings of the 30th International Conference on Machine Learning, pp. 71–79 (2013)
Theocharous, G., Thomas, P., Ghavamzadeh, M.: Ad recommendation systems for life-time value optimization. In: Proceedings of the 24th International Conference on World Wide Web Companion, pp. 1305–1310 (2015)
Zhao, P., Jin, R., Yang, T., Hoi, S.C.: Online AUC maximization. In: Proceedings of the 28th International Conference on Machine Learning (ICML-11), pp. 233–240 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Chaudhuri, S., Theocharous, G., Ghavamzadeh, M. (2017). Importance of Recommendation Policy Space in Addressing Click Sparsity in Personalized Advertisement Display. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2017. Lecture Notes in Computer Science(), vol 10358. Springer, Cham. https://doi.org/10.1007/978-3-319-62416-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-62416-7_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-62415-0
Online ISBN: 978-3-319-62416-7
eBook Packages: Computer ScienceComputer Science (R0)