Skip Abstract Section
Abstract
Mainstream solutions to sequential recommendation represent items with fixed vectors. These vectors have limited capability in capturing items’ latent aspects and users’ diverse preferences. As a new generative paradigm, diffusion models have achieved excellent performance in areas like computer vision and natural language processing. To our understanding, its unique merit in representation generation well fits the problem setting of sequential recommendation. In this article, we make the very first attempt to adapt the diffusion model to sequential recommendation and propose DiffuRec for item representation construction and uncertainty injection. Rather than modeling item representations as fixed vectors, we represent them as distributions in DiffuRec, which reflect a user’s multiple interests and an item’s various aspects adaptively. In the diffusion phase, DiffuRec corrupts the target item embedding into a Gaussian distribution via noise adding, which is further applied for sequential item distribution representation generation and uncertainty injection. Afterward, the item representation is fed into an approximator for target item representation reconstruction. In the reverse phase, based on a user’s historical interaction behaviors, we reverse a Gaussian noise into the target item representation, then apply a rounding operation for target item prediction. Experiments over four datasets show that DiffuRec outperforms strong baselines by a large margin.1
- [1] . 2022. Blended diffusion for text-driven editing of natural images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 18208–18218.Google Scholar
Cross Ref
- [2] . 1986. An analysis for unreplicated fractional factorials. Technometrics 28, 1 (1986), 11–18.Google ScholarCross Ref
- [3] . 2020. Learning gradient fields for shape generation. In Computer Vision—ECCV 2020. Lecture Notes in Computer Science, Vol. 12348. Springer, 364–381.Google Scholar
- [4] . 2020. Controllable multi-interest framework for recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2942–2951.Google ScholarDigital Library
- [5] . 2020. WaveGrad: Estimating gradients for waveform generation. arXiv preprint arXiv:2009.00713 (2020).Google Scholar
- [6] . 2018. Sequential recommendation with user memory networks. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining. 108–116.Google ScholarDigital Library
- [7] . 2022. Continuous diffusion for categorical data. arXiv preprint arXiv:2211.15089 (2022).Google Scholar
- [8] . 2021. Leveraging two types of global graph for sequential fashion recommendation. In Proceedings of the 2021 International Conference on Multimedia Retrieval. 73–81.Google ScholarDigital Library
- [9] . 2021. Modeling sequences as distributions with uncertainty for sequential recommendation. In Proceedings of the 30th ACM International Conference on Information and Knowledge Management. 3019–3023.Google ScholarDigital Library
- [10] . 2022. Sequential recommendation via stochastic self-attention. In Proceedings of the 2022 ACM Web Conference. 2036–2047.Google ScholarDigital Library
- [11] . 2021. Continuous-time sequential recommendation with temporal graph collaborative transformer. In Proceedings of the 30th ACM International Conference on Information and Knowledge Management. 433–442.Google ScholarDigital Library
- [12] . 2022. DiffuSeq: Sequence to sequence text generation with diffusion models. arXiv preprint arXiv:2210.08933 (2022).Google Scholar
- [13] . 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014).Google Scholar
- [14] . 2021. DA-GCN: A domain-aware attentive graph convolution network for shared-account cross-domain sequential recommendation. arXiv preprint arXiv:2105.03300 (2021).Google Scholar
- [15] . 2022. SSD-LM: Semi-autoregressive simplex-based diffusion language model for text generation and modular control. arXiv preprint arXiv:2210.17432 (2022).Google Scholar
- [16] . 2018. Adversarial personalized ranking for recommendation. In Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval. 355–364.Google ScholarDigital Library
- [17] . 2022. DiffusionBERT: Improving generative masked language models with diffusion models. arXiv preprint arXiv:2211.15029 (2022).Google Scholar
- [18] . 2018. Recurrent neural networks with top-k gains for session-based recommendations. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 843–852.Google ScholarDigital Library
- [19] . 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 (2015).Google Scholar
- [20] . 2020. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems 33 (2020), 6840–6851.Google Scholar
- [21] . 2022. Cascaded diffusion models for high fidelity image generation. Journal of Machine Learning Research 23 (2022), Article 47, 33 pages.Google Scholar
- [22] . 2022. Video diffusion models. arXiv preprint arXiv:2204.03458 (2022).Google Scholar
- [23] . 2021. Argmax flows and multinomial diffusion: Towards non-autoregressive language models. arXiv preprint arXiv:2102.05379 (2021).Google Scholar
- [24] . 2011. Novelty and diversity in top-n recommendation—Analysis and evaluation. ACM Transactions on Internet Technology 10, 4 (2011), 1–30.Google ScholarDigital Library
- [25] . 2018. Self-attentive sequential recommendation. In Proceedings of the 2018 IEEE International Conference on Data Mining (ICDM ’18). IEEE, Los Alamitos, CA, 197–206.Google ScholarCross Ref
- [26] . 2022. On sampled metrics for item recommendation. Communications of the ACM 65, 7 (2022), 75–83.Google ScholarDigital Library
- [27] . 2019. Multi-interest network with dynamic routing for recommendation at Tmall. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2615–2623.Google ScholarDigital Library
- [28] . 2022. Diffusion-LM improves controllable text generation. arXiv preprint arXiv:2205.14217 (2022).Google Scholar
- [29] . 2018. Variational autoencoders for collaborative filtering. In Proceedings of the 2018 World Wide Web Conference. 689–698.Google ScholarDigital Library
- [30] . 2023. TESS: Text-to-text self-conditioned simplex diffusion. arXiv preprint arXiv:2305.08379 (2023).Google Scholar
- [31] . 2021. Improved denoising diffusion probabilistic models. In Proceedings of the International Conference on Machine Learning. 8162–8171.Google Scholar
- [32] . 2022. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 (2022).Google Scholar
- [33] . 2010. Factorizing personalized Markov chains for next-basket recommendation. In Proceedings of the 19th International Conference on World Wide Web. 811–820.Google ScholarDigital Library
- [34] . 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10684–10695.Google ScholarCross Ref
- [35] . 2015. U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015. Lecture Notes in Computer Science, Vol. 9351. Springer, 234–241.Google Scholar
- [36] . 2019. Sequential variational autoencoders for collaborative filtering. In Proceedings of the 12th ACM International Conference on Web Search and Data Mining. 600–608.Google ScholarDigital Library
- [37] . 2023. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 4 (2023), 4713–4726.Google Scholar
- [38] . 2005. An MDP-based recommender system. Journal of Machine Learning Research 6, 9 (2005), 1265–1295.Google Scholar
- [39] . 2022. Sequential recommendation with user evolving preference decomposition. arXiv preprint arXiv:2203.16942 (2022).Google Scholar
- [40] . 2015. Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the International Conference on Machine Learning. 2256–2265.Google Scholar
- [41] . 2022. Self-conditioned embedding diffusion for text generation. arXiv preprint arXiv:2211.04236 (2022).Google Scholar
- [42] . 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from Transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 1441–1450.Google ScholarDigital Library
- [43] . 2021. Dynamic memory based attention network for sequential recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 4384–4392.Google ScholarCross Ref
- [44] . 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining. 565–573.Google ScholarDigital Library
- [45] . 2022. When multi-level meets multi-interest: A multi-grained neural model for sequential recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1632–1641.Google ScholarDigital Library
- [46] . 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, 11 (2008), 2579–2605.Google Scholar
- [47] . 2017. Attention is all you need. Advances in Neural Information Processing Systems 30 (2017), 1–11.Google Scholar
- [48] . 2022. Target interest distillation for multi-interest recommendation. In Proceedings of the 31st ACM International Conference on Information and Knowledge Management. 2007–2016.Google ScholarDigital Library
- [49] . 2020. KERL: A knowledge-guided reinforcement learning model for sequential recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 209–218.Google ScholarDigital Library
- [50] . 2020. SSE-PT: Sequential recommendation via personalized Transformer. In Proceedings of the 14th ACM Conference on Recommender Systems. 328–337.Google ScholarDigital Library
- [51] . 2019. Session-based recommendation with graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 346–353.Google ScholarDigital Library
- [52] . 2019. Hierarchical neural variational model for personalized sequential recommendation. In Proceedings of the 2019 World Wide Web Conference. 3377–3383.Google ScholarDigital Library
- [53] . 2021. Adversarial and contrastive variational autoencoder for sequential recommendation. In Proceedings of the 2021 Web Conference. 449–459.Google ScholarDigital Library
- [54] . 2022. Diffusion models: A comprehensive survey of methods and applications. arXiv preprint arXiv:2209.00796 (2022).Google Scholar
- [55] . 2020. Future data helps training: Modeling future contexts for session-based recommendation. In Proceedings of the 2020 Web Conference. 303–313.Google ScholarDigital Library
- [56] . 2022. Re4: Learning to re-contrast, re-attend, re-construct for multi-interest recommendation. In Proceedings of the 2022 ACM Web Conference. 2216–2226.Google ScholarDigital Library
- [57] . 2021. A model of two tales: Dual transfer learning framework for improved long-tail item recommendation. In Proceedings of the 2021 Web Conference. 2220–2231.Google ScholarDigital Library
- [58] . 2012. Auralist: Introducing serendipity into music recommendation. In Proceedings of the 5th ACM International Conference on Web Search and Data Mining. 13–22.Google ScholarDigital Library
- [59] . 2019. InfoVAE: Balancing learning and inference in variational autoencoders. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 5885–5892.Google ScholarDigital Library
- [60] . 2018. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1059–1068.Google ScholarDigital Library
- [61] . 2022. Filter-enhanced MLP is all you need for sequential recommendation. In Proceedings of the 2022 ACM Web Conference. 2388–2399.Google ScholarDigital Library
- [62] . 2013. Using temporal data for making recommendations. arXiv preprint arXiv:1301.2320 (2013).Google Scholar
Index Terms
- DiffuRec: A Diffusion Model for Sequential Recommendation
Recommendations
Diffusion Augmentation for Sequential Recommendation
CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge ManagementSequential recommendation (SRS) has become the technical foundation in many applications recently, which aims to recommend the next item based on the user's historical interactions. However, sequential recommendation often faces the problem of data ...
User Popularity Preference Aware Sequential Recommendation
Computational Science – ICCS 2023AbstractIn recommender systems, users’ preferences for item popularity are diverse and dynamic, which reveals the different items that users prefer. Therefore, identifying user popularity preferences are significant for personalized recommendations. ...
Sequential Recommendation with Dual Side Neighbor-based Collaborative Relation Modeling
WSDM '20: Proceedings of the 13th International Conference on Web Search and Data MiningSequential recommendation task aims to predict user preference over items in the future given user historical behaviors. The order of user behaviors implies that there are resourceful sequential patterns embedded in the behavior history which reveal the ...
Comments