Bo Yan
ABSTRACT
The heterogeneous information network (HIN), which contains rich semantics depicted by meta-paths, has emerged as a potent tool for mitigating data sparsity in recommender systems. Existing HIN-based recommender systems operate under the assumption of centralized storage and model training. However, real-world data is often distributed due to privacy concerns, leading to the semantic broken issue within HINs and consequent failures in centralized HIN-based recommendations. In this paper, we suggest the HIN is partitioned into private HINs stored on the client side and shared HINs on the server. Following this setting, we propose a federated heterogeneous graph neural network (FedHGNN) based framework, which facilitates collaborative training of a recommendation model using distributed HINs while protecting user privacy. Specifically, we first formalize the privacy definition for HIN-based federated recommendation (FedRec) in the light of differential privacy, with the goal of protecting user-item interactions within private HIN as well as users' high-order patterns from shared HINs. To recover the broken meta-path based semantics and ensure proposed privacy measures, we elaborately design a semantic-preserving user interactions publishing method, which locally perturbs user's high-order patterns and related user-item interactions for publishing. Subsequently, we introduce an HGNN model for recommendation, which conducts node- and semantic-level aggregations to capture recovered semantics. Extensive experiments on four datasets demonstrate that our model outperforms existing methods by a substantial margin (up to 34% in HR@10 and 42% in NDCG@10) under a reasonable privacy budget (e.g., ε=1).
Supplemental Material
- Muhammad Ammad-ud-din, Elena Ivannikova, Suleiman A. Khan, Were Oyomno, Qiang Fu, Kuan Eeik Tan, and Adrian Flanagan. 2019. Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System. CoRR , Vol. abs/1901.09888 (2019).Google Scholar
- Jinheon Baek, Wonyong Jeong, Jiongdao Jin, Jaehong Yoon, and Sung Ju Hwang. 2023. Personalized Subgraph Federated Learning. In ICML (Proceedings of Machine Learning Research, Vol. 202). PMLR, 1396--1415.Google Scholar
- Di Chai, Leye Wang, Kai Chen, and Qiang Yang. 2021. Secure Federated Matrix Factorization. IEEE Intell. Syst. , Vol. 36, 5 (2021), 11--20.Google ScholarCross Ref
- Chaochao Chen, Huiwen Wu, Jiajie Su, Lingjuan Lyu, Xiaolin Zheng, and Li Wang. 2022. Differential Private Knowledge Transfer for Privacy-Preserving Cross-Domain Recommendation. In WWW. ACM, 1455--1465.Google Scholar
- Junyang Chen, Ziyi Chen, Mengzhu Wang, Ge Fan, Guo Zhong, Ou Liu, Wenfeng Du, Zhenghua Xu, and Zhiguo Gong. 2023. A Neural Inference of User Social Interest for Item Recommendation. Data Sci. Eng. , Vol. 8, 3 (2023), 223--233.Google Scholar
- Cynthia Dwork and Aaron Roth. 2014. The Algorithmic Foundations of Differential Privacy. Found. Trends Theor. Comput. Sci. , Vol. 9, 3--4 (2014), 211--407.Google ScholarDigital Library
- Shaohua Fan, Junxiong Zhu, Xiaotian Han, Chuan Shi, Linmei Hu, Biyu Ma, and Yongliang Li. 2019. Metapath-guided Heterogeneous Graph Neural Network for Intent Recommendation. In KDD. ACM, 2478--2486.Google Scholar
- Xinyu Fu, Jiani Zhang, Ziqiao Meng, and Irwin King. 2020. MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding. In WWW. ACM / IW3C2, 2331--2341.Google Scholar
- Chaoyang He, Keshav Balasubramanian, Emir Ceyani, Yu Rong, Peilin Zhao, Junzhou Huang, Murali Annavaram, and Salman Avestimehr. 2021. FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks. CoRR , Vol. abs/2104.07145 (2021).Google Scholar
- Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yong-Dong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. In SIGIR. ACM, 639--648.Google Scholar
- Seira Hidano and Takao Murakami. 2022. Degree-Preserving Randomized Response for Graph Neural Networks under Local Differential Privacy. CoRR , Vol. abs/2202.10209 (2022).Google Scholar
- Binbin Hu, Chuan Shi, Wayne Xin Zhao, and Tianchi Yang. 2018a. Local and Global Information Fusion for Top-N Recommendation in Heterogeneous Information Network. In CIKM. ACM, 1683--1686.Google Scholar
- Binbin Hu, Chuan Shi, Wayne Xin Zhao, and Philip S. Yu. 2018b. Leveraging Meta-path based Context for Top- N Recommendation with A Neural Co-Attention Model. In KDD. ACM, 1531--1540.Google Scholar
- Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. 2020. Heterogeneous Graph Transformer. In WWW. ACM / IW3C2, 2704--2710.Google Scholar
- Houye Ji, Junxiong Zhu, Xiao Wang, Chuan Shi, Bai Wang, Xiaoye Tan, Yanghua Li, and Shaojian He. 2021. Who You Would Like to Share With? A Study of Share Recommendation in Social E-commerce. In AAAI. AAAI Press, 232--239.Google Scholar
- Zhi-Yuan Li, Man-Sheng Chen, Yuefang Gao, and Chang-Dong Wang. 2023. Signal Contrastive Enhanced Graph Collaborative Filtering for Recommendation. Data Sci. Eng. , Vol. 8, 3 (2023), 318--328.Google Scholar
- Feng Liang, Weike Pan, and Zhong Ming. 2021. FedRec: Lossless Federated Recommendation with Explicit Feedback. In AAAI. AAAI Press, 4224--4231.Google Scholar
- Zhaohao Lin, Weike Pan, and Zhong Ming. 2021. FR-FMSS: Federated Recommendation via Fake Marks and Secret Sharing. In RecSys. ACM, 668--673.Google Scholar
- Zhaohao Lin, Weike Pan, Qiang Yang, and Zhong Ming. 2023. A Generic Federated Recommendation Framework via Fake Marks and Secret Sharing. ACM Trans. Inf. Syst. , Vol. 41, 2 (2023), 40:1--40:37.Google ScholarDigital Library
- Siwei Liu, Iadh Ounis, Craig Macdonald, and Zaiqiao Meng. 2020. A Heterogeneous Graph Neural Model for Cold-start Recommendation. In SIGIR. ACM, 2029--2032.Google ScholarDigital Library
- Yaoqi Liu, Cheng Yang, Tianyu Zhao, Hui Han, Siyuan Zhang, Jing Wu, Guangyu Zhou, Hai Huang, Hui Wang, and Chuan Shi. 2023. GammaGL: A Multi-Backend Library for Graph Neural Networks. In SIGIR. ACM, 2861--2870.Google Scholar
- Zhiwei Liu, Liangwei Yang, Ziwei Fan, Hao Peng, and Philip S. Yu. 2022. Federated Social Recommendation with Graph Neural Network. ACM Trans. Intell. Syst. Technol. , Vol. 13, 4 (2022), 55:1--55:24.Google ScholarDigital Library
- Yuanfu Lu, Yuan Fang, and Chuan Shi. 2020. Meta-learning on Heterogeneous Information Networks for Cold-start Recommendation. In KDD. ACM, 1563--1573.Google Scholar
- Sichun Luo, Yuanzhang Xiao, and Linqi Song. 2022. Personalized Federated Recommendation via Joint Representation Learning, User Clustering, and Model Adaptation. In CIKM. ACM, 4289--4293.Google Scholar
- Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agü era y Arcas. 2017. Communication-Efficient Learning of Deep Networks from Decentralized Data. In AISTATS (Proceedings of Machine Learning Research, Vol. 54). PMLR, 1273--1282.Google Scholar
- Khalil Muhammad, Qinqin Wang, Diarmuid O'Reilly-Morgan, Elias Z. Tragos, Barry Smyth, Neil Hurley, James Geraci, and Aonghus Lawlor. 2020. FedFast: Going Beyond Average for Faster Training of Federated Recommender Systems. In KDD. ACM, 1234--1242.Google Scholar
- Zhan Qin, Ting Yu, Yin Yang, Issa Khalil, Xiaokui Xiao, and Kui Ren. 2017. Generating Synthetic Decentralized Social Graphs with Local Differential Privacy. In CCS. ACM, 425--438.Google Scholar
- Liang Qu, Ningzhi Tang, Ruiqi Zheng, Quoc Viet Hung Nguyen, Zi Huang, Yuhui Shi, and Hongzhi Yin. 2023. Semi-decentralized Federated Ego Graph Learning for Recommendation. In WWW. ACM, 339--348.Google Scholar
- Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In UAI. AUAI Press, 452--461.Google ScholarDigital Library
- Michael Sejr Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. 2018. Modeling Relational Data with Graph Convolutional Networks. In ESWC (Lecture Notes in Computer Science, Vol. 10843). Springer, 593--607.Google Scholar
- Chuan Shi, Binbin Hu, Wayne Xin Zhao, and Philip S. Yu. 2019. Heterogeneous Information Network Embedding for Recommendation. IEEE Trans. Knowl. Data Eng. , Vol. 31, 2 (2019), 357--370.Google ScholarDigital Library
- Chuan Shi, Yitong Li, Jiawei Zhang, Yizhou Sun, and Philip S. Yu. 2017. A Survey of Heterogeneous Information Network Analysis. IEEE Trans. Knowl. Data Eng. , Vol. 29, 1 (2017), 17--37.Google ScholarDigital Library
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In NIPS. 5998--6008.Google Scholar
- Xiao Wang, Deyu Bo, Chuan Shi, Shaohua Fan, Yanfang Ye, and Philip S. Yu. 2023. A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources. IEEE Trans. Big Data, Vol. 9, 2 (2023), 415--436.Google ScholarCross Ref
- Xiao Wang, Peng Cui, Jing Wang, Jian Pei, Wenwu Zhu, and Shiqiang Yang. 2017. Community Preserving Network Embedding. In AAAI. AAAI Press, 203--209.Google Scholar
- Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019a. Neural Graph Collaborative Filtering. In SIGIR. ACM, 165--174.Google Scholar
- Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S. Yu. 2019b. Heterogeneous Graph Attention Network. In WWW. ACM, 2022--2032.Google Scholar
- Xiao Wang, Nian Liu, Hui Han, and Chuan Shi. 2021. Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning. In KDD. ACM, 1726--1736.Google Scholar
- Chuhan Wu, Fangzhao Wu, Lingjuan Lyu, Tao Qi, Yongfeng Huang, and Xing Xie. 2022. A federated graph neural network framework for privacy-preserving personalization. Nature Communications, Vol. 13, 1 (2022), 3091.Google ScholarCross Ref
- Fengli Xu, Jianxun Lian, Zhenyu Han, Yong Li, Yujian Xu, and Xing Xie. 2019. Relation-Aware Graph Convolutional Networks for Agent-Initiated Social E-Commerce Recommendation. In CIKM. ACM, 529--538.Google Scholar
- Qiang Yang, Lixin Fan, and Han Yu (Eds.). 2020. Federated Learning - Privacy and Incentive. Lecture Notes in Computer Science, Vol. 12500. Springer.Google Scholar
- Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. 2019. Federated Machine Learning: Concept and Applications. ACM Trans. Intell. Syst. Technol. , Vol. 10, 2 (2019), 12:1--12:19.Google ScholarDigital Library
- Qingqing Ye, Haibo Hu, Man Ho Au, Xiaofeng Meng, and Xiaokui Xiao. 2022. LF-GDPR: A Framework for Estimating Graph Metrics With Local Differential Privacy. IEEE Trans. Knowl. Data Eng. , Vol. 34, 10 (2022), 4905--4920.Google ScholarDigital Library
- Senci Ying. 2020. Shared MF: A privacy-preserving recommendation system. CoRR , Vol. abs/2008.07759 (2020).Google Scholar
- Wei Yuan, Chaoqun Yang, Quoc Viet Hung Nguyen, Lizhen Cui, Tieke He, and Hongzhi Yin. 2023. Interaction-level Membership Inference Attack Against Federated Recommender Systems. In WWW. ACM, 1053--1062.Google Scholar
- Chunxu Zhang, Guodong Long, Tianyi Zhou, Peng Yan, Zijian Zhang, Chengqi Zhang, and Bo Yang. 2023. Dual Personalization on Federated Recommendation. In IJCAI. ijcai.org, 4558--4566.Google Scholar
- Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V. Chawla. 2019. Heterogeneous Graph Neural Network. In KDD. ACM, 793--803.Google ScholarDigital Library
- Mengmei Zhang, Xiao Wang, Meiqi Zhu, Chuan Shi, Zhiqiang Zhang, and Jun Zhou. 2022. Robust Heterogeneous Graph Neural Networks against Adversarial Attacks. In AAAI. AAAI Press, 4363--4370.Google Scholar
- Shijie Zhang and Hongzhi Yin. 2022. Comprehensive Privacy Analysis on Federated Recommender System against Attribute Inference Attacks. CoRR , Vol. abs/2205.11857 (2022).Google Scholar
- Huan Zhao, Quanming Yao, Jianda Li, Yangqiu Song, and Dik Lun Lee. 2017. Meta-Graph Based Recommendation Fusion over Heterogeneous Information Networks. In KDD. ACM, 635--644.Google Scholar
- Jiawei Zheng, Qianli Ma, Hao Gu, and Zhenjing Zheng. 2021. Multi-view Denoising Graph Auto-Encoders on Heterogeneous Information Networks for Cold-start Recommendation. In KDD. ACM, 2338--2348.Google Scholar
- Jiayin Zheng, Juanyun Mai, and Yanlong Wen. 2022. Explainable Session-based Recommendation with Meta-path Guided Instances and Self-Attention Mechanism. In SIGIR. ACM, 2555--2559.Google Scholar
- Zhihui Zhou, Lilin Zhang, and Ning Yang. 2023. Contrastive Collaborative Filtering for Cold-Start Item Recommendation. In WWW. ACM, 928--937. ioGoogle Scholar
Index Terms
- Federated Heterogeneous Graph Neural Network for Privacy-preserving Recommendation
Recommendations
Privacy-preserving Cross-domain Recommendation with Federated Graph Learning
As people inevitably interact with items across multiple domains or various platforms, cross-domain recommendation (CDR) has gained increasing attention. However, the rising privacy concerns limit the practical applications of existing CDR models, since ...
Heterogeneous information network-based interest composition with graph neural network for recommendation
AbstractHeterogeneous information networks (HINs) are widely applied to recommendation systems due to their capability of modeling various auxiliary information with meta-paths. However, existing HIN-based recommendation models usually fuse the ...
A Privacy-Preserving Friend Recommendation Mechanism for Online Social Networks
ICCSP 2020: Proceedings of the 2020 4th International Conference on Cryptography, Security and PrivacyFriend recommendation systems is widely applied in the context of online social networks (OSNs). Such systems aim to expand users' social network to increase users' engagement with related OSNs. However, during the cold-start stage, in which situation ...
Comments