Skip to main content
SpringerLink
Log in

Elderly Care - Human Activity Recognition Using Radar with an Open Dataset and Hybrid Maps

  • Conference paper
  • First Online:
Body Area Networks. Smart IoT and Big Data for Intelligent Health Management (BODYNETS 2021)

Included in the following conference series:

Abstract

Population ageing has become a severe problem worldwide. Human activity recognition (HAR) can play an important role to provide the elders with in-time healthcare. With the advantages of environmental insensitivity, contactless sensing and privacy protection, radar has been widely used for human activity detection. The micro-Doppler signatures (spectrograms) contain much information about human motion and are often applied in HAR. However, spectrograms only interpret magnitude information, resulting in suboptimal performances. We propose a radar-based HAR system using deep learning techniques. The data applied came from the open dataset “Radar signatures of human activities” created by the University of Glasgow. A new type of hybrid map was proposed, which concatenated the spectrograms amplitude and phase. After cropping the hybrid maps to focus on useful information, a convolutional neural network (CNN) based on LeNet-5 was designed for feature extraction and classification. In addition, the idea of transfer learning was applied for radar-based HAR to evaluate the classification performance of a pre-trained network. For this, GoogLeNet was taken and trained on the newly-produced hybrid maps. These initial results showed that the LeNet-5 CNN using only the spectrograms obtained an accuracy of 80.5%, while using the hybrid maps reached an accuracy of 84.3%, increasing by 3.8%. The classification result of transfer learning using GoogLeNet was 86.0%.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Department of Economic and Social Affairs United Nations: Population Division World Population Ageing 2019 (2020)

    Google Scholar 

  2. Bulling, A., Blanke, U., Schiele, B.: A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput. Surv. 46(3), 33 (2014). https://doi.org/10.1145/2499621

  3. Kernec, J.L., et al.: Radar signal processing for sensing in assisted living: the challenges associated with real-time implementation of emerging algorithms. IEEE Signal Process. Mag. 36(4), 29–41 (2019). https://doi.org/10.1109/MSP.2019.2903715

    Article  Google Scholar 

  4. Imran, M.A., Ghannam, R., Abbasi, Q.H., Fioranelli, F., Kernec, J.L.: Contactless radar sensing for health monitoring. In: Imran, M.A., Ghannam, R., Abbasi, Q.H. (eds.) Engineering and Technology for Healthcare (2021).. https://doi.org/10.1002/9781119644316.ch2

  5. Fioranelli, F., Kernec, J.L., Shah, S.A.: Radar for Health care: recognizing human activities and monitoring vital signs. IEEE Potent. 38(4), 16–23 (2019). https://doi.org/10.1109/MPOT.2019.2906977

    Article  Google Scholar 

  6. Jia, M., Li, S., Kernec, J.L., Yang, S., Fioranelli, F., Romain, O.: Human activity classification with radar signal processing and machine learning. International Conference on UK-China Emerging Technologies (UCET) 2020, 1–5 (2020). https://doi.org/10.1109/UCET51115.2020.9205461

    Article  Google Scholar 

  7. Li, S., Jia, M., Kernec, J.L., Yang, S., Fioranelli, F., Romain, O.: Elderly Care: Using Deep Learning for Multi-Domain Activity Classification. International Conference on UK-China Emerging Technologies (UCET) 2020, 1–4 (2020). https://doi.org/10.1109/UCET51115.2020.9205464

    Article  Google Scholar 

  8. Yang, S., Kernec, J.L., Fioranelli, F., Romain, O.: Human Activities Classification in a Complex Space Using Raw Radar Data. International Radar Conference (RADAR) 2019, 1–4 (2019). https://doi.org/10.1109/RADAR41533.2019.171367

    Article  Google Scholar 

  9. Li, X., He, Y., Jing, X.: A survey of deep learning-based human activity recognition in radar. Remote. Sens. 11, 1068 (2019)

    Article  Google Scholar 

  10. Gurbuz, S.Z., Amin, M.G.: Radar-based human-motion recognition with deep learning: promising applications for indoor monitoring. IEEE Sig. Process. Mag. 36(4), 16–28 (2019). https://doi.org/10.1109/MSP.2018.2890128

    Article  Google Scholar 

  11. Kernec, J.L., Fioranelli, F., Yang, S., Lorandel, J., Romain, O.: Radar for assisted living in the context of Internet of Things for Health and beyond. In: 2018 IFIP/IEEE International Conference on Very Large Scale Integration (VLSI-SoC), pp. 163–167 (2018), https://doi.org/10.1109/VLSI-SoC.2018.8644816

  12. Chen, V.C.: The Micro-Doppler Effect in Radar. Artech House Publishers, Boston (2011)

    Google Scholar 

  13. Kim, Y., Ling, H.: Human activity classification based on micro-doppler signatures using a support vector machine. IEEE Trans. Geosci. Remote Sens. 47(5), 1328–1337 (2009). https://doi.org/10.1109/TGRS.2009.2012849

    Article  Google Scholar 

  14. Fioranelli, F., et al.: Radar-based evaluation of lameness detection in ruminants: preliminary results. In: 2019 IEEE MTT-S International Microwave Biomedical Conference (IMBioC), pp. 1–4 (2019). https://doi.org/10.1109/IMBIOC.2019.8777830

  15. Linardopoulou, K., Viora, L., Abbasi, Q.H., Fioranelli, F., Le Kernec, J., Jonsson, N.: Lameness detection in dairy cows: evaluation of the agreement and repeatability of mobility scoring. In: 74th Annual AVTRW Conference, 14–15 Septemper 2020 (2020)

    Google Scholar 

  16. Busin, V., et al.: Radar sensing as a novel tool to detect lameness in sheep. In: Ontario Small Ruminant Veterinary Conference, Ontario, Canada, 17–19 June 2019

    Google Scholar 

  17. Shrestha, A., et al.: Animal lameness detection with radar sensing. IEEE Geosci. Remote Sens. Lett. 15(8), 1189–1193. https://doi.org/10.1109/LGRS.2018.2832650

  18. Busin, V., et al.: Evaluation of lameness detection using radar sensing in ruminants. Vet. Record 185(18), 572 (2019). https://doi.org/10.1136/vr.105407. PMID:31554712

    Article  Google Scholar 

  19. Shrestha, A., Le Kernec, J., Fioranelli, F., Marshall, J.F., Voute, L.: Gait analysis of horses for lameness detection with radar sensors. In: RADAR 2017: International Conference on Radar Systems, Belfast, UK, 23–26 October 2017. ISBN:9781785616730. https://doi.org/10.1049/cp.2017.0427

  20. Li, H., et al.: Multisensor data fusion for human activities classification and fall detection. In: IEEE Sensors 2017, Glasgow, UK, 30 October–01 November 2017. ISBN: 9781509010127. https://doi.org/10.1109/ICSENS.2017.8234179

  21. Li, X., Li, Z., Fioranelli, F., Yang, S., Romain, O., Kernec, J.L.: Hierarchical radar data analysis for activity and personnel recognition. Remote Sens. 12(14), 2237 (2020)

    Article  Google Scholar 

  22. Fioranelli, F., Shah, S.A., Li, H., Shrestha, A., Yang, S., Le Kernec, J.: Radar sensing for healthcare. Electr. Lett. 55(19), 1022-1024 (2019). https://doi.org/10.1049/el.2019.2378

  23. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015). https://doi.org/10.1038/nature14539

  24. Smith, K.A., Csech, C., Murdoch, D., Shaker, G.: Gesture recognition using mm-wave sensor for human-car interface. IEEE Sens. Lett. 2(2), 1–4 (2018). https://doi.org/10.1109/LSENS.2018.2810093

    Article  Google Scholar 

  25. Kim, B.K., Kang, H., Park, S.: Drone classification using convolutional neural networks with merged doppler images. IEEE Geosci. Remote Sens. Lett. 14(1), 38–42 (2017). https://doi.org/10.1109/LGRS.2016.2624820

    Article  Google Scholar 

  26. Kim, Y., Moon, T.: Human detection and activity classification based on micro-doppler signatures using deep convolutional neural networks. IEEE Geosci. Remote Sens. Lett. 13(1), 8–12 (2016). https://doi.org/10.1109/LGRS.2015.2491329

    Article  Google Scholar 

  27. Jokanovic, B., Amin, M., Erol, B.: Multiple joint-variable domains recognition of human motion. In: 2017 IEEE Radar Conference (RadarConf), 8–12 May 2017. pp. 0948–0952 (2017). https://doi.org/10.1109/RADAR.2017.7944340

  28. Loukas, C., Fioranelli, F., Le Kernec, J., Yang, S.: Activity classification using raw range and I & Q radar data with long short term memory layers. In: 2018 IEEE 16th Intl Conferenceon Dependable, Autonomic and Secure Computing, 16th Intl Conference on Pervasive Intelligence and Computing, 4th International Conference on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech), pp. 441–445 (2018). https://doi.org/10.1109/DASC/PiCom/DataCom/CyberSciTec.2018.00088

  29. Guendel, R.G., Fioranelli, F., Yarovoy, A.: Phase-based classification for arm gesture and gross-motor activities using histogram of oriented gradients. IEEE Sensors J. 21(6), 7918–7927 (2021). https://doi.org/10.1109/JSEN.2020.3044675

  30. Fioranelli, F., Shah, S.A., Li, H., Shrestha, A., Yang, S., Le Kernec, J.: Radar signatures of human activities. https://doi.org/10.5525/gla.researchdata.848

  31. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998). https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  32. Szegedy, C., et al.: Going deeper with convolutions. Proc. CVPR (2015). arXiv:1409.4842

  33. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. Proc. CVPR (2014). arXiv:1409.1556

  34. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. Proc. CVPR (2016). arXiv:1512.03385

  35. Olivas, E.S., Guerrero, J.D.M., Sober, M.M., Benedito, J.R.M., Lopez, A.J.S.: Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods and Techniques - 2 Volumes. IGI Publishing (2009)

    Google Scholar 

  36. MATLAB: Pretrained Deep Neural Networks. The MathWorks Inc. https://www.mathworks.com/help/deeplearning/ug/pretrained-convolutional-neural-networks.html. Accessed 19 Apr 2021

  37. Bruna, J., Chintala, S., LeCun, Y., Piantino, S., Szlam, A., Tygert, M.: A mathematical motivation for complex-valued convolutional networks (2015). arXiv:1503.03438v3

  38. Gandhi, A.: Data augmentation|how to use deep learning when you have limited data — Part 2. Nanonets. https://nanonets.com/blog/data-augmentation-how-to-use-deep-learning-when-you-have-limited-data-part-2/. Accessed 19 Apr 2021

  39. Li, J., Shrestha, A., Le Kernec, J., Fioranelli, F.: From Kinect skeleton data to hand gesture recognition with radar. J. Eng. 2019(20), 6914–6919 (2019). https://doi.org/10.1049/joe.2019.0557

    Article  Google Scholar 

  40. Li, H., le Kernec, J., Mehul, A., Gurbuz, S.Z., Fioranelli, F.: Distributed radar information fusion for gait recognition and fall detection. In: 2020 IEEE Radar Conference (RadarConf 2020), pp. 1–6 (2020). https://doi.org/10.1109/RadarConf2043947.2020.9266319

  41. Li, H., Mehul, A., Le Kernec, J., Gurbuz, S.Z., Fioranelli, F.: Sequential human gait classification with distributed radar sensor fusion. In: IEEE Sensors J. 21(6), 7590–7603 (2021). https://doi.org/10.1109/JSEN.2020.3046991

Download references

Author information

Authors and Affiliations

  1. University of Glasgow, University Avenue, Glasgow, G12 8QQ, UK

    Xinyu Zhang, Qammer H. Abbasi & Julien Le Kernec

  2. University of Electronic Science and Technology of China, Qingshuihe Campus: Chengdu High-tech Zone (West) 2006 West Avenue, Chengdu, 611731, China

    Xinyu Zhang & Julien Le Kernec

  3. TU Delft, Mekelweg 4, 2628 CD, Delft, The Netherlands

    Francesco Fioranelli

  4. University Cergy-Pontoise, 6 Avenue du ponceau, 95000, Cergy-Pontoise, France

    Olivier Romain

Authors
  1. Xinyu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qammer H. Abbasi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Francesco Fioranelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Olivier Romain
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Julien Le Kernec
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Julien Le Kernec .

Editor information

Editors and Affiliations

  1. University of Glasgow, Glasgow, UK

    Masood Ur Rehman

  2. University of Glasgow, Glasgow, UK

    Ahmed Zoha

Rights and permissions

Reprints and permissions

Copyright information

© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, X., Abbasi, Q.H., Fioranelli, F., Romain, O., Le Kernec, J. (2022). Elderly Care - Human Activity Recognition Using Radar with an Open Dataset and Hybrid Maps. In: Ur Rehman, M., Zoha, A. (eds) Body Area Networks. Smart IoT and Big Data for Intelligent Health Management. BODYNETS 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 420. Springer, Cham. https://doi.org/10.1007/978-3-030-95593-9_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-95593-9_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-95592-2

  • Online ISBN: 978-3-030-95593-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation