Skip to main content

Advertisement

Springer Nature Link
Log in

Exploring privacy measurement in federated learning

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Federated learning (FL) is a collaborative artificial intelligence (AI) approach that enables distributed training of AI models without data sharing, thereby promoting privacy by design. However, it is essential to acknowledge that FL only offers a partial solution to safeguard the confidentiality of AI and machine learning (ML) models. Unfortunately, many studies fail to report the results of privacy measurement when applying FL, mainly due to assumptions that privacy is implicitly achieved as FL is a privacy-by-design approach. This trend can also be attributed to the complexity of understanding privacy measurement metrics and methods. This paper presents a survey of privacy measurement in FL, aimed at evaluating its effectiveness in protecting the privacy of sensitive data during the training of AI and ML models. While FL is a promising approach for preserving privacy during model training, ensuring privacy is genuinely achieved in practice is crucial. By evaluating privacy measurement metrics and methods in FL, we can identify the gaps in existing approaches and propose new techniques to enhance FL’s privacy. A comprehensive study investigating “privacy measurement and metrics” in FL is therefore required to support the field’s growth. Our survey provides a critical analysis of the current state of privacy measurement in FL, identifies gaps in existing research, and offers insights into potential research directions. Moreover, this paper presents a case study that evaluates the effectiveness of various privacy techniques in a specific FL scenario. This case study serves as tangible evidence of the real-world implications of privacy measurements, providing insightful and practical guidelines for researchers and practitioners to optimize privacy preservation while balancing other crucial factors such as communication overhead and accuracy. Finally, our paper outlines a future roadmap for advancing privacy in FL, combining traditional techniques with innovative technologies such as quantum computing and Trusted Execution Environments to fortify data protection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Al Hayajneh A, Bhuiyan MZA, McAndrew I (2020) Improving internet of things (IOT) security with software-defined networking (SDN). Computers 9(1):8

    Article  Google Scholar 

  2. Ungurean I, Gaitan NC (2020) A software architecture for the industrial internet of things-a conceptual model. Sensors 20(19):5603

    Article  Google Scholar 

  3. Truong N, Sun K, Wang S, Guitton F, Guo Y (2021) Privacy preservation in federated learning: an insightful survey from the GDPR perspective. Comput Secur 110:102402

    Article  Google Scholar 

  4. McMahan B, Moore E, Ramage D, Hampson S, Arcas BAY (2017) Communication-efficient learning of deep networks from decentralized data. In: Artificial intelligence and statistics. PMLR, pp 1273–1282

  5. Bellavista P, Foschini L, Montanari R, Romandini N (2022) Flowchain: the playground for federated learning in industrial internet of things environments. IEEE Internet Things Mag 5(2):78–83. https://doi.org/10.1109/IOTM.001.2100188

    Article  Google Scholar 

  6. Kang J, Li X, Nie J, Liu Y, Xu M, Xiong Z, Niyato D, Yan Q (2022) Communication-efficient and cross-chain empowered federated learning for artificial intelligence of things. IEEE Trans Netw Sci Eng 9(5):2966–2977

    Article  Google Scholar 

  7. Lv C, Zhou X, Zhong L, Yan C, Srinivasan M, Seh ZW, Liu C, Pan H, Li S, Wen Y et al (2022) Machine learning: an advanced platform for materials development and state prediction in lithium-ion batteries. Adv Mater 34(25):2101474

    Article  Google Scholar 

  8. Yazdinejad A, Parizi RM, Dehghantanha A, Karimipour H (2021) Federated learning for drone authentication. Ad Hoc Netw 120:102574

    Article  Google Scholar 

  9. Mothukuri V, Parizi RM, Pouriyeh S, Mashhadi A (2022) Cloudfl: a zero-touch federated learning framework for privacy-aware sensor cloud. In: Proceedings of the 17th International Conference on Availability, Reliability and Security, ARES ’22. Association for Computing Machinery, New York

  10. Nguyen DC, Ding M, Pathirana PN, Seneviratne A, Li J, Poor HV (2021) Federated learning for internet of things: a comprehensive survey. IEEE Commun Surv Tutor 23(3):1622–1658

    Article  Google Scholar 

  11. Briggs C, Fan Z, Andras P (2021) A review of privacy-preserving federated learning for the internet-of-things. Federated Learn Syst Towards Next Gen A I:21–50

    Article  Google Scholar 

  12. Shokri R, Stronati M, Song C, Shmatikov V (2017) Membership inference attacks against machine learning models. In: IEEE Symposium on Security and Privacy. IEEE, pp 3–18

  13. Truex S, Baracaldo N, Anwar A, Steinke T, Ludwig H, Zhang R, Zhou Y (2019) A hybrid approach to privacy-preserving federated learning. In: Proceedings of the 12th ACM workshop on artificial intelligence and security, pp 1–11

  14. Bonawitz K, Ivanov V, Kreuter B, Marcedone A, McMahan HB, Patel S, Ramage D, Segal A, Seth K (2016) Practical secure aggregation for federated learning on user-held data. arXiv preprint arXiv:1611.04482

  15. Yin X, Zhu Y, Hu J (2021) A comprehensive survey of privacy-preserving federated learning: a taxonomy, review, and future directions. ACM Comput Surv 54(6):1–36

    Article  Google Scholar 

  16. Liu Z, Guo J, Yang W, Fan J, Lam K-Y, Zhao J (2022) Privacy-preserving aggregation in federated learning: a survey. IEEE Trans Big Data 25:92

    Google Scholar 

  17. Kaissis GA, Makowski MR, Rückert D, Braren RF (2020) Secure, privacy-preserving and federated machine learning in medical imaging. Nat Mach Intell 2(6):305–311

    Article  Google Scholar 

  18. Yang Q, Liu Y, Chen T, Tong Y (2019) Federated machine learning: concept and applications. ACM Trans Intell Syst Technol 10(2):1–19

    Article  Google Scholar 

  19. Li Q, Wen Z, Wu Z, Hu S, Wang N, Li Y, Liu X, He B (2021) A survey on federated learning systems: vision, hype and reality for data privacy and protection. IEEE Trans Knowl Data Eng 16:24

    Google Scholar 

  20. Bharati S, Mondal M, Podder P, Prasath V (2022) Federated learning: applications, challenges and future scopes. Int J Hybrid Intell Syst 8:1–17

    Google Scholar 

  21. Blanco-Justicia A, Domingo-Ferrer J, Martínez S, Sánchez D, Flanagan A, Tan KE (2021) Achieving security and privacy in federated learning systems: survey, research challenges and future directions. Eng Appl Artif Intell 106:104468

    Article  Google Scholar 

  22. Mothukuri V, Parizi RM, Pouriyeh S, Huang Y, Dehghantanha A, Srivastava G (2021) A survey on security and privacy of federated learning. Futur Gener Comput Syst 115:619–640

    Article  Google Scholar 

  23. Gong X, Sharma A, Karanam S, Wu Z, Chen T, Doermann D, Innanje A (2022) Preserving privacy in federated learning with ensemble cross-domain knowledge distillation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 36, pp 11891–11899

  24. Bonawitz K, Ivanov V, Kreuter B, Marcedone A, McMahan HB, Patel S, Ramage D, Segal A, Seth K (2017) Practical secure aggregation for privacy-preserving machine learning. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp 1175–1191

  25. Elkordy AR, Zhang J, Ezzeldin YH, Psounis K, Avestimehr S (2022) How much privacy does federated learning with secure aggregation guarantee? arXiv preprint arXiv:2208.02304

  26. Kim J, Park G, Kim M, Park S (2023) Cluster-based secure aggregation for federated learning. Electronics 12(4):870

    Article  Google Scholar 

  27. El Ouadrhiri A, Abdelhadi A (2022) Differential privacy for deep and federated learning: a survey. IEEE Access 10:22359–22380

    Article  Google Scholar 

  28. Wei K, Li J, Ding M, Ma C, Yang HH, Farokhi F, Jin S, Quek TQ, Poor HV (2020) Federated learning with differential privacy: algorithms and performance analysis. IEEE Trans Inf Forensics Secur 15:3454–3469

    Article  Google Scholar 

  29. Mugunthan V, Polychroniadou A, Byrd D, Balch TH (2019) Smpai: secure multi-party computation for federated learning. In: Proceedings of the NeurIPS 2019 Workshop on Robust AI in Financial Services

  30. Hosseini SM, Sikaroudi M, Babaei M, Tizhoosh HR (2022) Cluster based secure multi-party computation in federated learning for histopathology images. In: International Workshop on Distributed, Collaborative, and Federated Learning. Springer, pp 110–118

  31. Yu S, Cui L (2022) Secure multi-party computation in federated learning. In: Security and Privacy in Federated Learning. Springer, pp 89–98

  32. Fang H, Qian Q (2021) Privacy preserving machine learning with homomorphic encryption and federated learning. Future Internet 13(4):94

    Article  Google Scholar 

  33. Wibawa F, Catak FO, Kuzlu M, Sarp S, Cali U (2022) Homomorphic encryption and federated learning based privacy-preserving cnn training: Covid-19 detection use-case. In: Proceedings of the 2022 European Interdisciplinary Cybersecurity Conference, pp 85–90

  34. Tian H, Wen Y, Zhang F, Shao Y, Li B (2022) A distributed threshold additive homomorphic encryption for federated learning with dropout resiliency based on lattice. In: International Symposium on Cyberspace Safety and Security. Springer, pp 277–292

  35. Gul M (2023) Fully homomorphic encryption with applications to privacy-preserving machine learning, Ph.D. thesis

  36. Yazdinejad A, Dehghantanha A, Parizi RM, Hammoudeh M, Karimipour H, Srivastava G (2022) Block hunter: federated learning for cyber threat hunting in blockchain-based iiot networks. IEEE Trans Ind Inf 18(11):8356–8366

    Article  Google Scholar 

  37. Nguyen DC, Ding M, Pham Q-V, Pathirana PN, Le LB, Seneviratne A, Li J, Niyato D, Poor HV (2021) Federated learning meets blockchain in edge computing: opportunities and challenges. IEEE Internet Things J 8(16):12806–12825

    Article  Google Scholar 

  38. Goh E, Kim D, Kim DY, Lee K (2023) Blockchain-enabled federated learning: a reference architecture incorporating a did access system. arXiv preprint arXiv:2306.10841

  39. Nasr M, Shokri R, Houmansadr A (2019) Comprehensive privacy analysis of deep learning: passive and active white-box inference attacks against centralized and federated learning. In: IEEE Symposium on Security and Privacy. IEEE, pp 739–753

  40. Zhu L, Liu Z, Han S (2019) Deep leakage from gradients. Adv Neural Inf Process Syst 32:24

    Google Scholar 

  41. Balle B, Bell J, Gascón A, Nissim K (2019) The privacy blanket of the shuffle model. In: Advances in Cryptology-CRYPTO 2019: 39th Annual International Cryptology Conference, Santa Barbara, CA, USA, August 18–22, 2019, Proceedings, Part II 39. Springer, pp 638–667

  42. McMahan B, Moore E, Ramage D, Hampson S, Arcas BA (2017) Communication-efficient learning of deep networks from decentralized data. In: Artificial intelligence and statistics. PMLR, pp 1273–1282

  43. Mammen PM (2021) Federated learning: opportunities and challenges. arXiv preprint arXiv:2101.05428

  44. Yazdinejad A, Dehghantanha A, Srivastava G (2023) Ap2fl: auditable privacy-preserving federated learning framework for electronics in healthcare. IEEE Trans Consum Electron. https://doi.org/10.1109/TCE.2023.3318509

    Article  Google Scholar 

  45. Bonawitz K, Eichner H, Grieskamp W, Huba D, Ingerman A, Ivanov V, Kiddon C, Konečnỳ J, Mazzocchi S, McMahan B et al (2019) Towards federated learning at scale: system design. Proc Mach Learn Syst 1:374–388

    Google Scholar 

  46. Bonawitz KA, Ivanov V, Kreuter B, Marcedone A, McMahan HB, Patel S, Ramage D, Segal A, Seth A (2016) Practical secure aggregation for federated learning on user-held data. In: NIPS Workshop on Private Multi-Party Machine Learning

  47. Kholod I, Yanaki E, Fomichev D, Shalugin E, Novikova E, Filippov E, Nordlund M (2020) Open-source federated learning frameworks for iot: a comparative review and analysis. Sensors 21(1):167

    Article  Google Scholar 

  48. Ziller A, Trask A, Lopardo A, Szymkow B, Wagner B, Bluemke E, Nounahon J-M, Passerat-Palmbach J, Prakash K, Rose N et al (2021) Pysyft: a library for easy federated learning. Federated Learn Syst Towards Next Gen A I:111–139

    Article  Google Scholar 

  49. Beutel DJ, Topal T, Mathur A, Qiu X, Parcollet T, de Gusmão PP, Lane ND (2020) Flower: a friendly federated learning research framework. arXiv preprint arXiv:2007.14390

  50. Zheng Z, Zhou Y, Sun Y, Wang Z, Liu B, Li K (2022) Applications of federated learning in smart cities: recent advances, taxonomy, and open challenges. Connect Sci 34(1):1–28

    Article  Google Scholar 

  51. Yazdinejad A, Parizi RM, Dehghantanha A, Zhang Q, Choo K-KR (2020) An energy-efficient sdn controller architecture for iot networks with blockchain-based security. IEEE Trans Serv Comput 13(4):625–638

    Article  Google Scholar 

  52. Nakhodchi S, Zolfaghari B, Yazdinejad A, Dehghantanha A (2021) Steeleye: an application-layer attack detection and attribution model in industrial control systems using semi-deep learning. In: 2021 18th International Conference on Privacy, Security and Trust (PST), pp 1–8. https://doi.org/10.1109/PST52912.2021.9647777

  53. Luo S, Fu S, Luo Y, Liu L, Deng Y, Wang S (2023) Privacy-preserving federated learning with hierarchical clustering to improve training on non-iid data. In: International Conference on Network and System Security. Springer, pp 195–216

  54. Gu X, Zhu T, Li J, Zhang T, Ren W (2020) The impact of differential privacy on model fairness in federated learning. In: Network and System Security: 14th International Conference, NSS 2020, Melbourne, VIC, Australia, November 25–27, Proceedings 14. Springer, pp 419–430

  55. Kairouz P, McMahan HB, Avent B, Bellet A, Bennis M, Bhagoji AN, Bonawitz K, Charles Z, Cormode G, Cummings R et al (2021) Advances and open problems in federated learning. Found Trends Mach Learn 14(1–2):1–210

    Article  Google Scholar 

  56. Lindell Y (2005) Secure multiparty computation for privacy preserving data mining. In: Encyclopedia of data warehousing and mining. IGI Global, pp 1005–1009

  57. Dwork C (2011) A firm foundation for private data analysis. Commun ACM 54(1):86–95

    Article  Google Scholar 

  58. Abadi M, Chu A, Goodfellow I, McMahan HB, Mironov I, Talwar K, Zhang L (2016) Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp 308–318

  59. Cosic T, Misetic R, Stefancic H (2018) Observed lifespan differential-global trends, policy impact and computational methods. arXiv preprint arXiv:1812.07839

  60. Fu Y, Zhou Y, Wu D, Yu S, Wen Y, Li C (2021) On the practicality of differential privacy in federated learning by tuning iteration times. arXiv preprint arXiv:2101.04163

  61. Tolpegin V, Truex S, Gursoy ME, Liu L (2020) Data poisoning attacks against federated learning systems. In: Computer Security-ESORICS 2020: 25th European Symposium on Research in Computer Security, ESORICS 2020, Guildford, UK, September 14–18, 2020, Proceedings, Part I 25. Springer, pp 480–501

  62. Cao X, Gong NZ (2022) Mpaf: model poisoning attacks to federated learning based on fake clients. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 3396–3404

  63. Bagdasaryan E, Veit A, Hua Y, Estrin D, Shmatikov V (2020) How to backdoor federated learning. In: International Conference on Artificial Intelligence and Statistics. PMLR, pp 2938–2948

  64. Zakeri H, Antsaklis PJ (2019) A data-driven adaptive controller reconfiguration for fault mitigation: a passivity approach. In: 2019 27th Mediterranean Conference on Control and Automation (MED). IEEE, pp 25–30

  65. EUR-Lex-32016R0679-EN-EUR-Lex—eur-lex.europa.eu. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32016R0679. Accessed 28 Feb 2023

  66. Yazdinejad A, Dehghantanha A, Parizi RM, Epiphaniou G (2023) An optimized fuzzy deep learning model for data classification based on nsga-ii. Neurocomputing 522:116–128

    Article  Google Scholar 

  67. California Consumer Privacy Act (CCPA)—oag.ca.gov. https://oag.ca.gov/privacy/ccpa. Accessed 28 Feb 2023

  68. HIPAA Home—hhs.gov. https://www.hhs.gov/hipaa/index.html. Accessed 28 Feb 2023

  69. Children’s Online Privacy Protection Rule (“COPPA")—ftc.gov. https://www.ftc.gov/enforcement/rules/rulemaking-regulatory-reform-proceedings/childrens-online-privacy-protection-rule. Accessed 28 Feb 2023

  70. Li B, Fan L, Gu H, Li J, Yang Q (2022) Fedipr: ownership verification for federated deep neural network models. IEEE Trans Pattern Anal Mach Intell 45:4521–4536

    Google Scholar 

  71. Li J, Li M, Lu G, Zhang B, Yin H, Zhang D (2020) Similarity and diversity induced paired projection for cross-modal retrieval. Inf Sci 539:215–228

    Article  MathSciNet  Google Scholar 

  72. Geyer RC, Klein T, Nabi M (2017) Differentially private federated learning: a client level perspective. arXiv preprint arXiv:1712.07557

  73. Yazdinejad A, Dehghantanha A, Karimipour H, Srivastava G, Parizi RM (2022) An efficient packet parser architecture for software-defined 5g networks. Phys. Commun. 53:101677. https://doi.org/10.1016/j.phycom.2022.101677

    Article  Google Scholar 

  74. Johnson AE, Pollard TJ, Shen L, Lehman L-WH, Feng M, Ghassemi M, Moody B, Szolovits P, Anthony Celi L, Mark RG (2016) Mimic-iii, a freely accessible critical care database. Sci. Data 3(1):1–9

    Article  Google Scholar 

  75. Pytorch (2021). https://pytorch.org/blog/pytorch-1.9-released/

  76. Dias MLD (2019) fuzzy-c-means: an implementation of fuzzy \(c\)-means clustering algorithm. https://doi.org/10.5281/zenodo.3066222

  77. Raschka S (2018) Mlxtend: providing machine learning and data science utilities and extensions to python’s scientific computing stack. J Open Source Softw. https://doi.org/10.21105/joss.00638

    Article  Google Scholar 

  78. C. Data61, Python paillier library (2013). https://github.com/data61/python-paillier

  79. Paillier library (2016). https://python-paillier.readthedocs.io/en/develop/

  80. PySyft + Intel SGX. https://blog.openmined.org/pysyft-pytorch-intel-sgx/

  81. SciKit-Fuzzy (2019). https://github.com/scikit-fuzzy/scikit-fuzzy

  82. Yazdinejad A, Bohlooli A, Jamshidi K (2018) Efficient design and hardware implementation of the openflow v1.3 switch on the virtex-6 fpga ml605. J Supercomput 74:1299–1320

    Article  Google Scholar 

  83. Nielsen MA, Chuang I (2002) Quantum computation and quantum information

  84. Kaewpuang R, Xu M, Niyato D, Yu H, Xiong Z et al (2022) Adaptive resource allocation in quantum key distribution (qkd) for federated learning. arXiv preprint arXiv:2208.11270

  85. Sav S, Pyrgelis A, Troncoso-Pastoriza JR, Froelicher D, Bossuat JP, Sousa JS, Hubaux JP (2020) Poseidon: privacy-preserving federated neural network learning. arXiv preprint arXiv:2009.00349

  86. Chen SY-C, Yoo S (2021) Federated quantum machine learning. Entropy 23(4):460

    Article  Google Scholar 

  87. Quantum Random Number Generator (QRNG)—quside.com. https://quside.com/quantum-random-number-generators-why-how-where/#:~:text=What%20is%20a%20quantum%20random,%2C%20by%20nature%20non%2Ddeterministic. Accessed 17 Mar 2023

  88. Quoc DL, Fetzer C (2021) Secfl: confidential federated learning using tees. arXiv preprint arXiv:2110.00981

  89. Mo F, Haddadi H (2019) Efficient and private federated learning using tee. In: Proceedings of EuroSystem Conference, Dresden, Germany

  90. Mo F, Haddadi H, Katevas K, Marin E, Perino D, Kourtellis N (2021) Ppfl: privacy-preserving federated learning with trusted execution environments. In: Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services, pp 94–108

  91. Yazdinejad A, Rabieinejad E, Hasani T, Srivastava G (2023) A bert-based recommender system for secure blockchain-based cyber physical drug supply chain management. Clust Comput 8:1–15

    Google Scholar 

  92. Chen Y, Luo F, Li T, Xiang T, Liu Z, Li J (2020) A training-integrity privacy-preserving federated learning scheme with trusted execution environment. Inf Sci 522:69–79

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Decentralized Science Lab, Kennesaw State University, Kennesaw, GA, USA

    Gopi Krishna Jagarlamudi & Reza M. Parizi

  2. Cyber Science Lab, Canada Cyber Foundry, School of Computer Science, University of Guelph, Guelph, ON, Canada

    Abbas Yazdinejad

  3. Department of Information Technology, Kennesaw State University, Kennesaw, GA, USA

    Seyedamin Pouriyeh

Authors
  1. Gopi Krishna Jagarlamudi
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Abbas Yazdinejad
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Reza M. Parizi
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Seyedamin Pouriyeh
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Abbas Yazdinejad.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jagarlamudi, G.K., Yazdinejad, A., Parizi, R.M. et al. Exploring privacy measurement in federated learning. J Supercomput 80, 10511–10551 (2024). https://doi.org/10.1007/s11227-023-05846-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05846-4

Keywords

Profiles

  1. Gopi Krishna Jagarlamudi View author profile