ABSTRACT
While some powerful neural network architectures (e.g., Transformer, Graph Neural Networks) have achieved improved performance in sequential recommendation with high-order item dependency modeling, they may suffer from poor representation capability in label scarcity scenarios. To address the issue of insufficient labels, Contrastive Learning (CL) has attracted much attention in recent methods to perform data augmentation through embedding contrasting for self-supervision. However, due to the hand-crafted property of their contrastive view generation strategies, existing CL-enhanced models i) can hardly yield consistent performance on diverse sequential recommendation tasks; ii) may not be immune to user behavior data noise. In light of this, we propose a simple yet effective Graph Masked AutoEncoder-enhanced sequential Recommender system (MAERec) that adaptively and dynamically distills global item transitional information for self-supervised augmentation. It naturally avoids the above issue of heavy reliance on constructing high-quality embedding contrastive views. Instead, an adaptive data reconstruction paradigm is designed to be integrated with the long-range item dependency modeling, for informative augmentation in sequential recommendation. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art baseline models and can learn more accurate representations against data noise and sparsity. Our implemented model code is available at https://github.com/HKUDS/MAERec.
- Xuheng Cai, Chao Huang, Lianghao Xia, and Xubin Ren. 2023. LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation. In International Conference on Learning Representations (ICLR).Google Scholar
- Jianxin Chang, Chen Gao, Yu Zheng, Yiqun Hui, Yanan Niu, Yang Song, Depeng Jin, and Yong Li. 2021. Sequential recommendation with graph neural networks. In International Conference on Research and Development in Information Retrieval (SIGIR). 378--387.Google ScholarDigital Library
- Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2020. Bias and debias in recommender system: A survey and future directions. Transactions on Information Systems (TOIS) (2020).Google Scholar
- Lei Chen, Le Wu, Richang Hong, Kun Zhang, and Meng Wang. 2020. Revisiting graph based collaborative filtering: A linear residual graph convolutional network approach. In AAAI Conference on Artificial Intelligence (AAAI), Vol. 34. 27--34.Google ScholarCross Ref
- Tianwen Chen and Raymond Chi-Wing Wong. 2020. Handling information loss of graph neural networks for session-based recommendation. In International Conference on Knowledge Discovery and Data Mining (KDD). 1172--1180.Google ScholarDigital Library
- Yabo Chen, Yuchen Liu, Dongsheng Jiang, Xiaopeng Zhang, Wenrui Dai, Hongkai Xiong, and Qi Tian. 2022. Sdae: Self-distillated masked autoencoder. In European Conference on Computer Vision (ECCV). Springer, 108--124.Google ScholarDigital Library
- Yongjun Chen, Zhiwei Liu, Jia Li, Julian McAuley, and Caiming Xiong. 2022. Intent Contrastive Learning for Sequential Recommendation. In ACM Web Conference (WWW). 2172--2182.Google Scholar
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of ACL (NAACL).Google Scholar
- Xinyan Fan, Zheng Liu, Jianxun Lian, Wayne Xin Zhao, Xing Xie, and Ji-Rong Wen. 2021. Lighter and better: low-rank decomposed self-attention networks for next-item recommendation. In International Conference on Research and Development in Information Retrieval (SIGIR). 1733--1737.Google ScholarDigital Library
- Ziwei Fan, Zhiwei Liu, Yu Wang, Alice Wang, Zahra Nazari, Lei Zheng, Hao Peng, and Philip S Yu. 2022. Sequential Recommendation via Stochastic Self-Attention. In ACM Web Conference (WWW). 2036--2047.Google ScholarDigital Library
- Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. 2022. Masked autoencoders are scalable vision learners. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 16000--16009.Google ScholarCross Ref
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 770--778.Google ScholarCross Ref
- Sunan He, Taian Guo, Tao Dai, Ruizhi Qiao, Chen Wu, Xiujun Shu, and Bo Ren. 2022. VLMAE: Vision-Language Masked Autoencoder. arXiv preprint arXiv:2208.09374 (2022).Google Scholar
- Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In International Conference on Research and Development in Information Retrieval (SIGIR). 639--648.Google ScholarDigital Library
- Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based recommendations with recurrent neural networks. In International Conference on Learning Representations (ICLR).Google Scholar
- Zhenyu Hou, Xiao Liu, Yuxiao Dong, Chunjie Wang, Jie Tang, et al. 2022. Graph- MAE: Self-Supervised Masked Graph Autoencoders. In International Conference on Knowledge Discovery and Data Mining (KDD).Google Scholar
- Eric Jang, Shixiang Gu, and Ben Poole. 2017. Categorical Reparametrization with Gumble-Softmax. In International Conference on Learning Representations (ICLR).Google Scholar
- Baoyu Jing, Chanyoung Park, and Hanghang Tong. 2021. Hdmi: High-order deep multiplex infomax. In The Web Conference (WWW). 2414--2424.Google ScholarDigital Library
- Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In International Conference on Data Mining (ICDM). IEEE, 197--206.Google ScholarCross Ref
- Yejin Kim, Kwangseob Kim, Chanyoung Park, and Hwanjo Yu. 2019. Sequential and Diverse Recommendation with Long Tail.. In International Joint Conference on Artificial Intelligence (IJCAI), Vol. 19. 2740--2746.Google ScholarCross Ref
- Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017. Neural attentive session-based recommendation. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 1419--1428.Google ScholarDigital Library
- Guanyu Lin, Chen Gao, Yinfeng Li, Yu Zheng, Zhiheng Li, Depeng Jin, and Yong Li. 2022. Dual contrastive network for sequential recommendation. In International Conference on Research and Development in Information Retrieval (SIGIR). 2686--2691.Google ScholarDigital Library
- Siyi Liu and Yujia Zheng. 2020. Long-tail session-based recommendation. In ACM Conference on Recommender Systems (Recsys). 509--514.Google ScholarDigital Library
- Zhiwei Liu, Yongjun Chen, Jia Li, Philip S Yu, Julian McAuley, and Caiming Xiong. 2021. Contrastive self-supervised sequential recommendation with robust augmentation. arXiv preprint arXiv:2108.06479 (2021).Google Scholar
- Chen Ma, Liheng Ma, Yingxue Zhang, Jianing Sun, Xue Liu, and Mark Coates. 2020. Memory augmented graph neural networks for sequential recommendation. In AAAI Conference on Artificial Intelligence (AAAI), Vol. 34. 5045--5052.Google ScholarCross Ref
- Ruihong Qiu, Zi Huang, Hongzhi Yin, and Zijian Wang. 2022. Contrastive learning for representation degeneration problem in sequential recommendation. In International Conference on Web Search and Data Mining (WSDM). 813--823.Google ScholarDigital Library
- Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In International Conference on Information and Knowledge Management (CIKM). 1441--1450.Google ScholarDigital Library
- Qiaoyu Tan, Ninghao Liu, Xiao Huang, Rui Chen, Soo-Hyun Choi, and Xia Hu. 2023. S2GAE: Self-Supervised Graph Autoencoders Are Generalizable Learners with Graph Masking. In International Conference on Web Search and Data Mining (WSDM).Google ScholarDigital Library
- Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In International Conference on Web Search and Data Mining (WSDM). 565--573.Google ScholarDigital Library
- Changxin Tian, Yuexiang Xie, Yaliang Li, Nan Yang, and Wayne Xin Zhao. 2022. Learning to Denoise Unreliable Interactions for Graph Collaborative Filtering. In International Conference on Research and Development in Information Retrieval (SIGIR). 122--132.Google ScholarDigital Library
- Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. 2019. Deep Graph Infomax. In International Conference on Learning Representations (ICLR).Google Scholar
- Chenyang Wang, Weizhi Ma, Chong Chen, Min Zhang, Yiqun Liu, and Shaoping Ma. 2023. Sequential recommendation with multiple contrast signals. Transactions on Information Systems (TOIS) 41, 1 (2023), 1--27.Google ScholarDigital Library
- Jianling Wang, Kaize Ding, Liangjie Hong, Huan Liu, and James Caverlee. 2020. Next-item recommendation with sequential hypergraphs. In International Conference on Research and Development in Information Retrieval (SIGIR). 1101--1110.Google ScholarDigital Library
- Jianling Wang, Raphael Louca, Diane Hu, Caitlin Cellier, James Caverlee, and Liangjie Hong. 2020. Time to Shop for Valentine's Day: Shopping Occasions and Sequential Recommendation in E-commerce. In International Conference on Web Search and Data Mining (WSDM). 645--653.Google ScholarDigital Library
- Ziyang Wang, Wei Wei, Gao Cong, Xiao-Li Li, Xian-Ling Mao, and Minghui Qiu. 2020. Global context enhanced graph neural networks for session-based recommendation. In International Conference on Research and Development in Information Retrieval (SIGIR). 169--178.Google ScholarDigital Library
- Wei Wei, Chao Huang, Lianghao Xia, Yong Xu, Jiashu Zhao, and Dawei Yin. 2022. Contrastive meta learning with behavior multiplicity for recommendation. In International Conference on Web Search and Data Mining (WSDM). 1120--1128.Google ScholarDigital Library
- Yinwei Wei, Xiang Wang, Liqiang Nie, Xiangnan He, Richang Hong, and Tat-Seng Chua. 2019. MMGCN: Multi-modal graph convolution network for personalized recommendation of micro-video. In International Conference on Multimedia (MM). 1437--1445.Google ScholarDigital Library
- Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019. Session-based recommendation with graph neural networks. In AAAI Conference on Artificial Intelligence (AAAI), Vol. 33. 346--353.Google ScholarDigital Library
- Lianghao Xia, Chao Huang, and Chuxu Zhang. 2022. Self-supervised hypergraph transformer for recommender systems. In International Conference on Knowledge Discovery and Data Mining (KDD). 2100--2109.Google ScholarDigital Library
- Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Bolin Ding, and Bin Cui. 2021. Contrastive learning for sequential recommendation. In International Conference on Data Engineering (ICDE).Google Scholar
- An Yan, Shuo Cheng, Wang-Cheng Kang, Mengting Wan, and Julian McAuley. 2019. CosRec: 2D convolutional neural networks for sequential recommendation. In International Conference on Information and Knowledge Management (CIKM). 2173--2176.Google ScholarDigital Library
- Yuhao Yang, Chao Huang, Lianghao Xia, Chunzhen Huang, Da Luo, and Kangyi Lin. 2023. Debiased Contrastive Learning for Sequential Recommendation. In ACM Web Conference (WWW). 1063--1073.Google Scholar
- Yuhao Yang, Chao Huang, Lianghao Xia, Yuxuan Liang, Yanwei Yu, and Chenliang Li. 2022. Multi-behavior hypergraph-enhanced transformer for sequential recommendation. In International Conference on Knowledge Discovery and Data Mining (KDD). 2263--2274.Google ScholarDigital Library
- Wenwen Ye, Shuaiqiang Wang, Xu Chen, Xuepeng Wang, Zheng Qin, and Dawei Yin. 2020. Time matters: Sequential recommendation with complex temporal information. In International Conference on Research and Development in Information Retrieval (SIGIR). 1459--1468.Google ScholarDigital Library
- Shengyu Zhang, Dong Yao, Zhou Zhao, Tat-Seng Chua, and Fei Wu. 2021. Causerec: Counterfactual user sequence synthesis for sequential recommendation. In International Conference on Research and Development in Information Retrieval (SIGIR). 367--377.Google ScholarDigital Library
- Yixin Zhang, Yong Liu, Yonghui Xu, Hao Xiong, Chenyi Lei, Wei He, Lizhen Cui, and Chunyan Miao. 2022. Enhancing Sequential Recommendation with Graph Contrastive Learning. In International Joint Conference on Artificial Intelligence (IJCAI).Google Scholar
- Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. 2021. Graph contrastive learning with adaptive augmentation. In ACM Web Conference (WWW). 2069--2080.Google ScholarDigital Library
Index Terms
- Graph Masked Autoencoder for Sequential Recommendation
Recommendations
Multi-dimensional Graph Neural Network for Sequential Recommendation
Highlights- We design a multi-dimensional information integrated graph neural network, which integrates the category and time information into the graph embedding ...
AbstractGraph neural networks (GNNs) technology has been widely used in recommendation systems because most information in recommendation systems has a graph structure in nature, and GNNs have advantages in graph representation learning. In ...
Graph gating-mixer for sequential recommendation
AbstractRecent Transformer-based architectures have achieved encouraging performance for sequential recommendation, whereas their computational complexity is quadratic to the sequence length. MLP4Rec is a promising solution to settle this issue, which ...
Highlights- Proposing a novel sequential recommendation method named graph gating-mixer.
- Developing a dual gating block to infer short-term user interest.
- Designing a user-specific gating to adaptively integrate two components in GMRec.
- ...
AutoSeqRec: Autoencoder for Efficient Sequential Recommendation
CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge ManagementSequential recommendation demonstrates the capability to recommend items by modeling the sequential behavior of users. Traditional methods typically treat users as sequences of items, overlooking the collaborative relationships among them. Graph-based ...
Comments