Peihua Yu
ABSTRACT
Object detection with deep learning model has achieved good results in many fields, but in some fields that think highly of data privacy, such as medical care, its applications is greatly limited by data. And Federated Learning allows clients to train a model together, while leaving their data in the local, without sharing with the server or other clients. Using the methods of Federated Learning, such as Federated Averaging(FedAvg), to train models can provide privacy, security benefits. Nonetheless, there is little experiment applying Federated Learning algorithms to train the model with a large number of parameters, such as deep learning object detection model. With non-IID data, the accuracy of object detection model trained by FedAvg reduces significantly, and need more rounds to coverage. In this work, we use Kullback-Leibler divergence(KLD) measure the weights divergence between different model trained with non-IID data. And we propose a useful scheme to improve FedAvg based Abnormal Weights Supression, reducing the influence of the weights divergence caused by non-IID and unbalanced data. As a representative of object detection, we choose Single Shot MultiBox Detector(SSD) as the base model. The results of the experiments show that the Mean Average Precision(mAP) get obvious improvement in Pascal VOC 2007 test dataset.
- Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. Technical Report. Citeseer.Google Scholar
- Andre Esteva, Alexandre Robicquet, Bharath Ramsundar, Volodymyr Kuleshov, Mark DePristo, Katherine Chou, Claire Cui, Greg Corrado, Sebastian Thrun, and Jeff Dean. 2019. A guide to deep learning in healthcare. Nature medicine 25, 1 (2019), 24.Google Scholar
- Boyi Liu, Lujia Wang, Ming Liu, and Chengzhong Xu. 2019. Lifelong federated reinforcement learning: a learning architecture for navigation in cloud robotic systems. arXiv preprint arXiv:1901.06455 (2019).Google Scholar
- Brendan McMahan and Daniel Ramage. [n.d.]. Federated Learning: Collaborative Machine Learning without Centralized Training Data. https://ai.googleblog.com/2017/04/federated-learningcollaborative.html. Accessed April 6, 2017.Google Scholar
- Felix A Gers, Jürgen Schmidhuber, and Fred Cummins. 1999. Learning to forget: Continual prediction with LSTM. (1999).Google Scholar
- Felix Sattler, Simon Wiedemann, Klaus-Robert Müller, and Wojciech Samek. 2019. Robust and communication-efficient federated learning from non-iid data. arXiv preprint arXiv:1903.02891 (2019).Google Scholar
- H Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, et al. 2016. Communication-efficient learning of deep networks from decentralized data. arXiv preprint arXiv:1602.05629 (2016).Google Scholar
- Jakub Konečny, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. 2016. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492 (2016).Google Scholar
- Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 779-788.Google ScholarCross Ref
- Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. 2017. Practical secure aggregation for privacy-preserving machine learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. ACM, 1175--1191.Google ScholarDigital Library
- M. Everingham, S. M. A. Eslami, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. 2015. The Pascal Visual Object Classes Challenge: A Retrospective. International Journal of Computer Vision 111, 1 (Jan. 2015), 98--136.Google ScholarDigital Library
- Reza Shokri and Vitaly Shmatikov. 2015. Privacy-preserving deep learning. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security. ACM, 1310--1321.Google Scholar
- Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster rcnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems. 91-99.Google Scholar
- Sixin Zhang, Anna E Choromanska, and Yann LeCun. 2015. Deep learning with elastic averaging SGD. In Advances in Neural Information Processing Systems. 685-693.Google ScholarDigital Library
- Tsung-Yi Lin, Michael Maire, Serge J. Belongie, Lubomir D. Bourdev, Ross B. Girshick, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. CoRR abs/1405.0312 (2014). arXiv:1405.0312 http://arxiv.org/abs/1405.0312Google Scholar
- Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. 2016. Ssd: Single shot multibox detector. In European conference on computer vision. Springer, 21--37.Google ScholarCross Ref
- William Shakespeare. 2007. The complete works of William Shakespeare. Wordsworth Editions.Google Scholar
- Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. 2019. On the Convergence of FedAvg on Non-IID Data. arXiv preprint arXiv:1907.02189 (2019).Google Scholar
- Yang Liu, Tianjian Chen, and Qiang Yang. 2018. Secure Federated Transfer Learning. arXiv preprint arXiv:1812.03337 (2018).Google Scholar
- Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, et al. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278--2324.Google ScholarCross Ref
- Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. 2018. Federated learning with non-iid data. arXiv preprint arXiv:1806.00582 (2018).Google Scholar
- Yujun Lin, Song Han, Huizi Mao, Yu Wang, and William J Dally. 2017. Deep gradient compression: Reducing the communication bandwidth for distributed training. arXiv preprint arXiv:1712.01887 (2017).Google Scholar
Index Terms
- Federated Object Detection: Optimizing Object Detection Model with Federated Learning
Recommendations
Non-IID data re-balancing at IoT edge with peer-to-peer federated learning for anomaly detection
WiSec '21: Proceedings of the 14th ACM Conference on Security and Privacy in Wireless and Mobile NetworksThe increase of the computational power in edge devices has enabled the penetration of distributed machine learning technologies such as federated learning, which allows to build collaborative models performing the training locally in the edge devices, ...
Single shot object detection with refined feature
AbstractObject classification and localization are two significant aspects of object detector based on the Single Shot MultiBox Detector (SSD). In general, the more feature maps there are, the better the object classification performance will be. However, ...
Federated Learning based Object Detection using Dampened Harmonic Optimization
ICMLSC '23: Proceedings of the 2023 7th International Conference on Machine Learning and Soft ComputingObject Detection is the task of detecting and localizing objects of importance in visual media. The rapid increase in the number of powerful end devices such as mobiles and surveillance cameras have to lead to increasing in both resources and media ...
Comments