Weilin Lin
ABSTRACT
Feature selection plays an impactful role in deep recommender systems, which selects a subset of the most predictive features, so as to boost the recommendation performance and accelerate model optimization. The majority of existing feature selection methods, however, aim to select only a fixed subset of features. This setting cannot fit the dynamic and complex environments of practical recommender systems, where the contribution of a specific feature varies significantly across user-item interactions. In this paper, we propose an adaptive feature selection framework, AdaFS, for deep recommender systems. To be specific, we develop a novel controller network to automatically select the most relevant features from the whole feature space, which fits the dynamic recommendation environment better. Besides, different from classic feature selection approaches, the proposed controller can adaptively score each example of user-item interactions, and identify the most informative features correspondingly for subsequent recommendation tasks. We conduct extensive experiments based on two public benchmark datasets from a real-world recommender system. Experimental results demonstrate the effectiveness of AdaFS, and its excellent transferability to the most popular deep recommendation models.
Supplemental Material
- Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. IEEE TPAMI (2013).Google Scholar
Digital Library
- Leo Breiman. 1997. Arcing the edge. Technical Report.Google Scholar
- Bo Chen, Xiangyu Zhao, Yejing Wang, Wenqi Fan, Huifeng Guo, and Ruiming Tang. 2022. Automated Machine Learning for Deep Recommender Systems: A Survey. arXiv preprint arXiv:2204.01390 (2022).Google Scholar
- Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & deep learning for recommender systems. In Proc. of DLRS.Google ScholarDigital Library
- Wenqi Fan, Tyler Derr, Xiangyu Zhao, Yao Ma, Hui Liu, Jianping Wang, Jiliang Tang, and Qing Li. 2021. Attacking Black-box Recommendations via Copying Cross-domain User Profiles. In Proc. of ICDE.Google ScholarCross Ref
- Wei Fan, Kunpeng Liu, Hao Liu, Ahmad Hariri, Dejing Dou, and Yanjie Fu. 2021. AutoGFS: Automated Group-based Feature Selection via Interactive Reinforcement Learning. In Proc. of SDM.Google ScholarCross Ref
- Wei Fan, Kunpeng Liu, Hao Liu, Pengyang Wang, Yong Ge, and Yanjie Fu. 2020. AutoFS: Automated Feature selection via diversity-aware interactive reinforcement learning. In Proc. of ICDM.Google ScholarCross Ref
- Seyed Mehdi Hazrati Fard, Ali Hamzeh, and Sattar Hashemi. 2013. Using reinforcement learning to find an optimal set of features. Comput. Math. with Appl. (2013).Google Scholar
- Jerome H Friedman. 2017. The elements of statistical learning: Data mining, inference, and prediction. springer open.Google Scholar
- Yingqiang Ge, Shuchang Liu, Ruoyuan Gao, Yikun Xian, Yunqi Li, Xiangyu Zhao, Changhua Pei, Fei Sun, Junfeng Ge, Wenwu Ou, et al. 2021. Towards Long-term Fairness in Recommendation. In Proc. of WSDM.Google ScholarDigital Library
- Pablo M Granitto, Cesare Furlanello, Franco Biasioli, and Flavia Gasperi. 2006. Recursive feature elimination with random forest for PTR-MS analysis of agroindustrial products. Chemometr Intell Lab Syst (2006).Google Scholar
- Huifeng Guo, Bo Chen, Ruiming Tang,Weinan Zhang, Zhenguo Li, and Xiuqiang He. 2021. An Embedding Learning Framework for Numerical Features in CTR Prediction. In Proc. of KDD.Google ScholarDigital Library
- Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. In Proc. of IJCAI.Google ScholarCross Ref
- Isabelle Guyon and André Elisseeff. 2003. An introduction to variable and feature selection. Journal of machine learning research (2003).Google Scholar
- Mark Andrew Hall et al. 1999. Correlation-based feature selection for machine learning. (1999).Google Scholar
- Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In Proc. of ICML.Google ScholarDigital Library
- Yufan Jiang, Chi Hu, Tong Xiao, Chunliang Zhang, and Jingbo Zhu. 2019. Improved Differentiable Architecture Search for Language Modeling and Named Entity Recognition. In Proc. of EMNLP.Google ScholarCross Ref
- Wei Jin, Xiaorui Liu, Xiangyu Zhao, Yao Ma, Neil Shah, and Jiliang Tang. 2022. Automated Self-Supervised Learning for Graphs. In Proc. of ICLR.Google Scholar
- Manas R Joglekar, Cong Li, Mei Chen, Taibai Xu, Xiaoming Wang, Jay K Adams, Pranav Khaitan, Jiahui Liu, and Quoc V Le. 2020. Neural input search for large scale recommendation models. In Proc. of KDD.Google ScholarDigital Library
- Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. 1996. Reinforcement learning: A survey. Journal of artificial intelligence research (1996).Google Scholar
- Farhan Khawar, Xu Hang, Ruiming Tang, Bin Liu, Zhenguo Li, and Xiuqiang He. 2020. AutoFeature: Searching for Feature Interactions and Their Architectures for Click-through Rate Prediction. In Proc. of CIKM.Google ScholarDigital Library
- Ron Kohavi and George H John. 1997. Wrappers for feature subset selection. Artificial intelligence (1997).Google Scholar
- Mark Kroon and Shimon Whiteson. 2009. Automatic feature selection for modelbased reinforcement learning in factored MDPs. In Proc. of ICML.Google Scholar
- Bin Liu, Chenxu Zhu, Guilin Li, Weinan Zhang, Jincai Lai, Ruiming Tang, Xiuqiang He, Zhenguo Li, and Yong Yu. 2020. Autofis: Automatic feature interaction selection in factorization models for click-through rate prediction. In Proc. of KDD.Google ScholarDigital Library
- Huan Liu and Rudy Setiono. 1995. Chi2: Feature selection and discretization of numeric attributes. In Proc. of ICTAI.Google Scholar
- Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2019. DARTS: Differentiable Architecture Search. In Proc. of ICLR.Google Scholar
- Haochen Liu, Xiangyu Zhao, Chong Wang, Xiaobing Liu, and Jiliang Tang. 2020. Automated Embedding Size Search in Deep Recommender Systems. In Proc. of SIGIR.Google ScholarDigital Library
- Kunpeng Liu, Yanjie Fu, Pengfei Wang, Le Wu, Rui Bo, and Xiaolin Li. 2019. Automating feature subspace exploration via multi-agent reinforcement learning. In Proc. of KDD.Google ScholarDigital Library
- Kunpeng Liu, Yanjie Fu, LeWu, Xiaolin Li, Charu Aggarwal, and Hui Xiong. 2021. Automated feature selection: A reinforcement learning perspective. IEEE TKDE (2021).Google Scholar
- Siyi Liu, Chen Gao, Yihong Chen, Depeng Jin, and Yong Li. 2021. Learnable Embedding Sizes for Recommender Systems. In Proc. of ICLR.Google Scholar
- Yuanfei Luo, Mengshuo Wang, Hao Zhou, Quanming Yao, Wei-Wei Tu, Yuqiang Chen,Wenyuan Dai, and Qiang Yang. 2019. Autocross: Automatic feature crossing for tabular data in real-world applications. In Proc. of KDD.Google ScholarDigital Library
- Tom Mitchell. 1997. Machine learning. (1997).Google Scholar
- Boaz Nadler and Ronald R Coifman. 2005. The prediction error in CLS and PLS: the importance of feature selection prior to multivariate calibration. Journal of Chemometrics: A Journal of the Chemometrics Society (2005).Google Scholar
- Alexey Natekin and Alois Knoll. 2013. Gradient boosting machines, a tutorial. Frontiers in neurorobotics (2013).Google Scholar
- Hanchuan Peng, Fuhui Long, and Chris Ding. 2005. Feature selection based on mutual information criteria of max-dependency, max-relevance, and minredundancy. IEEE TPAMI (2005).Google Scholar
- Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. 2018. Efficient Neural Architecture Search via Parameters Sharing. In Proc. of ICML.Google Scholar
- Hassan Ramchoun, Mohammed Amine Janati Idrissi, Youssef Ghanou, and Mohamed Ettaouil. 2016. Multilayer Perceptron: Architecture Optimization and Training. Int. J. Interact. Multim. Artif. Intell. (2016).Google Scholar
- Steffen Rendle. 2010. Factorization machines. In Proc. of ICDM.Google ScholarDigital Library
- J Ben Schafer, Dan Frankowski, Jon Herlocker, and Shilad Sen. 2007. Collaborative filtering recommender systems. In The adaptive web.Google Scholar
- Bo Shu, Fuji Ren, and Yanwei Bao. 2018. Investigating Lstm with k-Max Pooling for Text Classification. In Proc. of ICICTA.Google ScholarCross Ref
- Qingquan Song, Dehua Cheng, Hanning Zhou, Jiyan Yang, Yuandong Tian, and Xia Hu. 2020. Towards automated neural interaction discovery for click-through rate prediction. In Proc. of KDD.Google ScholarDigital Library
- Yong Kiam Tan, Xinxing Xu, and Yong Liu. 2016. Improved recurrent neural networks for session-based recommendations. In Proc. of DLRS.Google ScholarDigital Library
- Robert Tibshirani. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological) (1996).Google Scholar
- Michael Tsang, Dehua Cheng, Hanpeng Liu, Xue Feng, Eric Zhou, and Yan Liu. 2020. Feature Interaction Interpretability: A Case for Explaining Ad- Recommendation Systems via Neural Interaction Detection. In Proc. of ICLR.Google Scholar
- YejingWang, Xiangyu Zhao, Tong Xu, and XianWu. 2022. AutoField: Automating Feature Selection in Deep Recommender Systems. In Proc. of WWW.Google Scholar
- SvanteWold, Kim Esbensen, and Paul Geladi. 1987. Principal component analysis. Chemometrics and intelligent laboratory systems (1987).Google Scholar
- Niannan Xue, Bin Liu, Huifeng Guo, Ruiming Tang, Fengwei Zhou, Stefanos P Zafeiriou, Yuzhou Zhang, Jun Wang, and Zhenguo Li. 2020. AutoHash: Learning Higher-order Feature Interactions for Deep CTR Prediction. IEEE TKDE (2020).Google Scholar
- Yiming Yang and Jan O Pedersen. 1997. A comparative study on feature selection in text categorization. In Icml.Google Scholar
- Lei Yu and Huan Liu. 2003. Feature selection for high-dimensional data: A fast correlation-based filter solution. In Proc. of ICML.Google Scholar
- Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019. Deep learning based recommender system: A survey and new perspectives. ACM CSUR (2019).Google Scholar
- Weinan Zhang, Xiangyu Zhao, Li Zhao, Dawei Yin, Grace Hui Yang, and Alex Beutel. 2020. Deep Reinforcement Learning for Information Retrieval: Fundamentals and Advances. In Proc. of SIGIR.Google ScholarDigital Library
- Xiangyu Zhao. 2022. Adaptive and automated deep recommender systems. ACM SIGWEB Newsletter (2022).Google Scholar
- Xiangyu Zhao, Haochen Liu, Wenqi Fan, Hui Liu, Jiliang Tang, and Chong Wang. 2021. AutoLoss: Automated Loss Function Search in Recommendations. In Proc. of KDD.Google ScholarDigital Library
- Xiangyu Zhao, Haochen Liu, Wenqi Fan, Hui Liu, Jiliang Tang, Chong Wang, Ming Chen, Xudong Zheng, Xiaobing Liu, and Xiwang Yang. 2021. Autoemb: Automated embedding dimensionality search in streaming recommendations. In Proc. of ICDM.Google Scholar
- Xiangyu Zhao, Haochen Liu, Hui Liu, Jiliang Tang, Weiwei Guo, Jun Shi, Sida Wang, Huiji Gao, and Bo Long. 2021. AutoDim: Field-aware Embedding Dimension Searchin Recommender Systems. In Proc. of WWW.Google ScholarDigital Library
- Xiangyu Zhao, Long Xia, Jiliang Tang, and Dawei Yin. 2019. Deep reinforcement learning for search, recommendation, and online advertising: a survey. ACM SIGWEB Newsletter (2019).Google Scholar
- Xiangyu Zhao, Long Xia, Liang Zhang, Zhuoye Ding, Dawei Yin, and Jiliang Tang. 2018. Deep Reinforcement Learning for Page-wise Recommendations. In Proc. of RecSys.Google ScholarDigital Library
- Xiangyu Zhao, Liang Zhang, Zhuoye Ding, Long Xia, Jiliang Tang, and Dawei Yin. 2018. Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning. In Proc. of KDD.Google ScholarDigital Library
- Barret Zoph and Quoc V. Le. 2017. Neural Architecture Search with Reinforcement Learning. arXiv:1611.01578 [cs.LG]Google Scholar
- Lixin Zou, Long Xia, Yulong Gu, Xiangyu Zhao, Weidong Liu, Jimmy Xiangji Huang, and Dawei Yin. 2020. Neural Interactive Collaborative Filtering. In Proc. of SIGIR.Google ScholarDigital Library
- Lixin Zou, Long Xia, Linfang Hou, Xiangyu Zhao, and Dawei Yin. 2021. Data-Efficient Reinforcement Learning for Malaria Control. arXiv e-prints (2021).Google Scholar
Index Terms
- AdaFS: Adaptive Feature Selection in Deep Recommender System
Recommendations
AutoField: Automating Feature Selection in Deep Recommender Systems
WWW '22: Proceedings of the ACM Web Conference 2022Feature quality has an impactful effect on recommendation performance. Thereby, feature selection is a critical process in developing deep learning-based recommender systems. Most existing deep recommender systems, however, focus on designing ...
MvFS: Multi-view Feature Selection for Recommender System
CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge ManagementFeature selection, which is a technique to select key features in recommender systems, has received increasing research attention. Recently, Adaptive Feature Selection (AdaFS) has shown remarkable performance by adaptively selecting features for each ...
Improving Accuracy of Recommender System by Item Clustering
Recommender System (RS) predicts user's ratings towards items, and then recommends highly-predicted items to user. In recent years, RS has been playing more and more important role in the agent research field. There have been a great deal of researches ...
Comments