skip to main content
research-article

Double Attention Convolutional Neural Network for Sequential Recommendation

Published:08 December 2022Publication History
Skip Abstract Section

Abstract

The explosive growth of e-commerce and online service has led to the development of recommender system. Aiming to provide a list of items to meet a user’s personalized need by analyzing his/her interaction1 history, recommender system has been widely studied in academic and industrial communities. Different from conventional recommender systems, sequential recommender systems attempt to capture the pattern of users’ sequential behaviors and the evolution of users’ preferences. Most of the existing sequential recommendation models only focus on user interaction sequence, but neglect item interaction sequence. An item interaction sequence also contains rich contextual information for capturing the item’s dynamic characteristic, since an item’s dynamic characteristic can be reflected by the users who interact with it in a period. Furthermore, existing dual sequential models use the same method to handle the user interaction sequence and item interaction sequence, and do not consider their different characteristics. Hence, we propose a novel Double Attention Convolution Neural Network (DACNN), which incorporates user interaction sequence and item interaction sequence into an integrated neural network framework. DACNN leverages the strength of attention mechanism to capture the temporary suitability and adopts CNN to extract local sequential features. Experimental evaluations on the real datasets show that DACNN outperforms the baseline approaches.

REFERENCES

  1. [1] Adomavicius Gediminas and Tuzhilin Alexander. 2005. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17, 6 (2005), 734749.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. [2] Bahdanau Dzmitry, Cho Kyunghyun, and Bengio Yoshua. 2015. Neural machine translation by jointly learning to align and translate. In ICLR.Google ScholarGoogle Scholar
  3. [3] Bell Robert M. and Koren Yehuda. 2007. Improved neighborhood-based collaborative filtering. In KDD Cup and Workshop at the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Citeseer, 714.Google ScholarGoogle Scholar
  4. [4] Bogina Veronika and Kuflik Tsvi. 2017. Incorporating dwell time in session-based recommendations with recurrent neural networks. In RecTemp@RecSys (CEUR Workshop Proceedings), Vol. 1922. CEUR-WS.org, 5759.Google ScholarGoogle Scholar
  5. [5] Chen Jingyuan, Zhang Hanwang, He Xiangnan, Nie Liqiang, Liu Wei, and Chua Tat-Seng. 2017. Attentive collaborative filtering: Multimedia recommendation with item- and component-level attention. In SIGIR. ACM, 335344.Google ScholarGoogle Scholar
  6. [6] Chen Tong, Yin Hongzhi, Nguyen Quoc Viet Hung, Peng Wen-Chih, Li Xue, and Zhou Xiaofang. 2020. Sequence-aware factorization machines for temporal predictive analytics. In ICDE. IEEE, 14051416.Google ScholarGoogle Scholar
  7. [7] Garcin Florent, Dimitrakakis Christos, and Faltings Boi. 2013. Personalized news recommendation with context trees. In RecSys. ACM, 105112.Google ScholarGoogle Scholar
  8. [8] Ghoshal Abhijeet, Menon Syam, and Sarkar Sumit. 2015. Recommendations using information from multiple association rules: A probabilistic approach. Inf. Syst. Res. 26, 3 (2015), 532551.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. [9] Ghoshal Abhijeet and Sarkar Sumit. 2014. Association rules for recommendations with multiple items. INFORMS J. Comput. 26, 3 (2014), 433448.Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Gong Yuyun and Zhang Qi. 2016. Hashtag recommendation using attention-based convolutional neural network. In IJCAI. IJCAI/AAAI Press, 27822788.Google ScholarGoogle Scholar
  11. [11] Guo Lei, Yin Hongzhi, Wang Qinyong, Chen Tong, Zhou Alexander, and Hung Nguyen Quoc Viet. 2019. Streaming session-based recommendation. In KDD. ACM, 15691577.Google ScholarGoogle Scholar
  12. [12] He Ruining and McAuley Julian J.. 2016. Fusing similarity models with Markov chains for sparse sequential recommendation. In ICDM. IEEE Computer Society, 191200.Google ScholarGoogle Scholar
  13. [13] He Xiangnan, He Zhankui, Song Jingkuan, Liu Zhenguang, Jiang Yu-Gang, and Chua Tat-Seng. 2018. NAIS: Neural attentive item similarity model for recommendation. IEEE Trans. Knowl. Data Eng. 30, 12 (2018), 23542366.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. [14] Hidasi Balázs, Quadrana Massimo, Karatzoglou Alexandros, and Tikk Domonkos. 2016. Parallel recurrent neural network architectures for feature-rich session-based recommendations. In RecSys. ACM, 241248.Google ScholarGoogle Scholar
  15. [15] Huang Xiaowen, Qian Shengsheng, Fang Quan, Sang Jitao, and Xu Changsheng. 2018. CSAN: Contextual self-attention network for user sequential recommendation. In ACM Multimedia. ACM, 447455.Google ScholarGoogle Scholar
  16. [16] Ji Mingi, Joo Weonyoung, Song Kyungwoo, Kim Yoon-Yeong, and Moon Il-Chul. 2020. Sequential recommendation with relation-aware kernelized self-attention. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI’20, New York, NY, USA, February 7-12, 2020. AAAI Press, 43044311.Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] Joachims Thorsten, Freitag Dayne, and Mitchell Tom M.. 1997. Web Watcher: A tour guide for the world wide web. In IJCAI (1). Morgan Kaufmann, 770777.Google ScholarGoogle Scholar
  18. [18] Kingma Diederik P. and Ba Jimmy. 2015. Adam: A method for stochastic optimization. In ICLR (Poster).Google ScholarGoogle Scholar
  19. [19] Koren Yehuda, Bell Robert M., and Volinsky Chris. 2009. Matrix factorization techniques for recommender systems. IEEE Computer 42, 8 (2009), 3037.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. [20] Kumar Srijan, Zhang Xikun, and Leskovec Jure. 2019. Predicting dynamic embedding trajectory in temporal interaction networks. In KDD. ACM, 12691278.Google ScholarGoogle Scholar
  21. [21] Lee Daniel D. and Seung H. Sebastian. 2000. Algorithms for non-negative matrix factorization. In NIPS. MIT Press, 556562.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. [22] Li Chenliang, Niu Xichuan, Luo Xiangyang, Chen Zhenzhong, and Quan Cong. 2019. A review-driven neural model for sequential recommendation. In IJCAI. ijcai.org, 28662872.Google ScholarGoogle Scholar
  23. [23] Li Jing, Ren Pengjie, Chen Zhumin, Ren Zhaochun, Lian Tao, and Ma Jun. 2017. Neural attentive session-based recommendation. In CIKM. ACM, 14191428.Google ScholarGoogle Scholar
  24. [24] Liu Jian, Zhao Pengpeng, Liu Yanchi, Xu Jiajie, Fang Junhua, Zhao Lei, Sheng Victor S., and Cui Zhiming. 2019. Attention and convolution enhanced memory network for sequential recommendation. In DASFAA (2) (Lecture Notes in Computer Science), Vol. 11447. Springer, 333349.Google ScholarGoogle Scholar
  25. [25] Liu Kuan, Shi Xing, Kumar Anoop, Zhu Linhong, and Natarajan Prem. 2016. Temporal learning and sequence modeling for a job recommender system. In RecSys. Challenge. ACM, 7:1–7:4.Google ScholarGoogle Scholar
  26. [26] Liu Qiang, Wu Shu, Wang Diyi, Li Zhaokang, and Wang Liang. 2016. Context-aware sequential recommendation. In ICDM. IEEE Computer Society, 10531058.Google ScholarGoogle Scholar
  27. [27] Liu Yining, Liu Yong, Shen Yanming, and Li Keqiu. 2018. Recommendation in a changing world: Exploiting temporal dynamics in ratings and reviews. ACM Trans. Web 12, 1 (2018), 3:1–3:20.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. [28] Moghaddam Samaneh, Jamali Mohsen, and Ester Martin. 2012. ETF: Extended tensor factorization model for personalizing prediction of review helpfulness. In WSDM. ACM, 163172.Google ScholarGoogle Scholar
  29. [29] Mooney Raymond J. and Roy Loriene. 2000. Content-based book recommending using learning for text categorization. In ACM DL. ACM, 195204.Google ScholarGoogle Scholar
  30. [30] Pan Rong, Zhou Yunhong, Cao Bin, Liu Nathan Nan, Lukose Rajan M., Scholz Martin, and Yang Qiang. 2008. One-class collaborative filtering. In ICDM. IEEE Computer Society, 502511.Google ScholarGoogle Scholar
  31. [31] Pazzani Michael J., Muramatsu Jack, and Billsus Daniel. 1996. Syskill & Webert: Identifying interesting web sites. In AAAI/IAAI, Vol. 1. AAAI Press/The MIT Press, 5461.Google ScholarGoogle Scholar
  32. [32] Qin Jiarui, Ren Kan, Fang Yuchen, Zhang Weinan, and Yu Yong. 2020. Sequential recommendation with dual side neighbor-based collaborative relation modeling. In WSDM. ACM, 465473.Google ScholarGoogle Scholar
  33. [33] Qiu Ruihong, Huang Zi, Li Jingjing, and Yin Hongzhi. 2020. Exploiting cross-session information for session-based recommendation with graph neural networks. ACM Trans. Inf. Syst. 38, 3 (2020), 22:1–22:23.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] Qiu Ruihong, Yin Hongzhi, Huang Zi, and Chen Tong. 2020. GAG: Global attributed graph neural network for streaming session-based recommendation. In SIGIR. ACM, 669678.Google ScholarGoogle Scholar
  35. [35] Quadrana Massimo, Karatzoglou Alexandros, Hidasi Balázs, and Cremonesi Paolo. 2017. Personalizing session-based recommendations with hierarchical recurrent neural networks. In RecSys. ACM, 130137.Google ScholarGoogle Scholar
  36. [36] Rendle Steffen, Freudenthaler Christoph, Gantner Zeno, and Schmidt-Thieme Lars. 2009. BPR: Bayesian personalized ranking from implicit feedback. (2009), 452461.Google ScholarGoogle Scholar
  37. [37] Rendle Steffen, Freudenthaler Christoph, and Schmidt-Thieme Lars. 2010. Factorizing personalized Markov chains for next-basket recommendation. (2010), 811820.Google ScholarGoogle Scholar
  38. [38] Salakhutdinov Ruslan and Mnih Andriy. 2007. Probabilistic matrix factorization. In NIPS. Curran Associates, Inc., 12571264.Google ScholarGoogle Scholar
  39. [39] Sarwar Badrul. 2001. Item-based collaborative filtering recommendation algorithms. In Proc. International World Wide Web Conference.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. [40] Senecal Sylvain and Nantel Jacques. 2004. The influence of online product recommendations on consumers’ online choices. Journal of Retailing 80, 2 (2004), 159169.Google ScholarGoogle ScholarCross RefCross Ref
  41. [41] Srivastava Nitish, Hinton Geoffrey E., Krizhevsky Alex, Sutskever Ilya, and Salakhutdinov Ruslan. 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1 (2014), 19291958.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. [42] Su Xiaoyuan and Khoshgoftaar Taghi M.. 2009. A survey of collaborative filtering techniques. Adv. Artificial Intellegence 2009 (2009), 421425:1–421425:19.Google ScholarGoogle Scholar
  43. [43] Sutskever Ilya, Vinyals Oriol, and Le Quoc V.. 2014. Sequence to sequence learning with neural networks. In NIPS. 31043112.Google ScholarGoogle Scholar
  44. [44] Tang Jiaxi and Wang Ke. 2018. Personalized top-N sequential recommendation via convolutional sequence embedding. In WSDM. ACM, 565573.Google ScholarGoogle Scholar
  45. [45] Wang Shoujin, Hu Liang, Wang Yan, Cao Longbing, Sheng Quan Z., and Orgun Mehmet A.. 2019. Sequential recommender systems: Challenges, progress and prospects. In IJCAI. ijcai.org, 63326338.Google ScholarGoogle Scholar
  46. [46] Wickramaratna Kasun, Kubat Miroslav, and Premaratne Kamal. 2009. Predicting missing items in shopping carts. IEEE Trans. Knowl. Data Eng. 21, 7 (2009), 985998.Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. [47] Wu Chao-Yuan, Ahmed Amr, Beutel Alex, Smola Alexander J., and Jing How. 2017. Recurrent recommender networks. In WSDM. ACM, 495503.Google ScholarGoogle Scholar
  48. [48] Wu Qitian, Gao Yirui, Gao Xiaofeng, Weng Paul, and Chen Guihai. 2019. Dual sequential prediction models linking sequential recommendation and information dissemination. In KDD. ACM, 447457.Google ScholarGoogle Scholar
  49. [49] Wu Yuehong and Yang Juan. 2021. Dual sequential recommendation integrating high-order collaborative relations via graph attention networks. In International Joint Conference on Neural Networks, IJCNN’21, Shenzhen, China, July 18-22, 2021. IEEE, 18.Google ScholarGoogle ScholarCross RefCross Ref
  50. [50] Wu Zhiang, Li Changsheng, Cao Jie, and Ge Yong. 2020. On scalability of association-rule-based recommendation: A unified distributed-computing framework. ACM Trans. Web 14, 3 (2020), 13:1–13:21.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. [51] Xia Xin, Yin Hongzhi, Yu Junliang, Wang Qinyong, Cui Lizhen, and Zhang Xiangliang. 2021. Self-supervised hypergraph convolutional networks for session-based recommendation. In AAAI. AAAI Press, 45034511.Google ScholarGoogle Scholar
  52. [52] Xu Chengfeng, Zhao Pengpeng, Liu Yanchi, Xu Jiajie, Sheng Victor S., Cui Zhiming, Zhou Xiaofang, and Xiong Hui. 2019. Recurrent convolutional neural network for sequential recommendation. In WWW. ACM, 33983404.Google ScholarGoogle Scholar
  53. [53] Yu Feng, Liu Qiang, Wu Shu, Wang Liang, and Tan Tieniu. 2016. A dynamic recurrent model for next basket recommendation. In SIGIR. ACM, 729732.Google ScholarGoogle Scholar
  54. [54] Zhang Shuai, Tay Yi, Yao Lina, and Sun Aixin. 2018. Next item recommendation with self-attention. CoRR abs/1808.06414.Google ScholarGoogle Scholar
  55. [55] Zhang Sheng, Wang Weihong, Ford James, Makedon Fillia, and Pearlman Justin D.. 2005. Using singular value decomposition approximation for collaborative filtering. In CEC. IEEE Computer Society, 257264.Google ScholarGoogle Scholar
  56. [56] Zhang Tingting, Zhao Pengpeng, Liu Yanchi, Sheng Victor S., Xu Jiajie, Wang Deqing, Liu Guanfeng, and Zhou Xiaofang. 2019. Feature-level deeper self-attention network for sequential recommendation. In IJCAI. ijcai.org, 43204326.Google ScholarGoogle Scholar
  57. [57] Zhang Yuyu, Dai Hanjun, Xu Chang, Feng Jun, Wang Taifeng, Bian Jiang, Wang Bin, and Liu Tie-Yan. 2014. Sequential click prediction for sponsored search with recurrent neural networks. In AAAI. AAAI Press, 13691375.Google ScholarGoogle Scholar
  58. [58] Zhou Chang, Bai Jinze, Song Junshuai, Liu Xiaofei, Zhao Zhengchao, Chen Xiusi, and Gao Jun. 2018. ATRank: An attention-based user behavior modeling framework for recommendation. In AAAI. AAAI Press, 45644571.Google ScholarGoogle Scholar

Index Terms

  1. Double Attention Convolutional Neural Network for Sequential Recommendation

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on the Web
      ACM Transactions on the Web  Volume 16, Issue 4
      November 2022
      165 pages
      ISSN:1559-1131
      EISSN:1559-114X
      DOI:10.1145/3571715
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 8 December 2022
      • Online AM: 9 August 2022
      • Accepted: 8 July 2022
      • Revised: 17 June 2022
      • Received: 11 July 2021
      Published in tweb Volume 16, Issue 4

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed
    • Article Metrics

      • Downloads (Last 12 months)133
      • Downloads (Last 6 weeks)20

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format