Skip to main content

Advertisement

SpringerLink
Log in

Smartphone based human activity monitoring and recognition using ML and DL: a comprehensive survey

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Human activity monitoring and recognition (HAMR) based on smartphone sensor data is a field that promotes a lot of observation in current era due to its notable desire in various Ambient Intelligent applications such as healthcare, sports, surveillance, and remote health monitoring. In this context, many research works have unveiled incredible results using various smartphone sensors such as accelerometer, gyroscope, magnetometer, digital compass, microphone, GPS and camera. The waveform of sensor motion is quite different in several smartphone placements even for the identical activity. This makes it challenging to apprehend varied completely different activities with high precision. Due to the difference in behavioral habits, gender and age, the movement patterns of various individuals vary greatly, which boosts the problem of dividing boundaries of various activities. In HAMR, the main computational tasks are quantitative analysis of human motion and its automatic classification. These cause the inception of Machine Learning (ML) and Deep Learning (DL) techniques to automatically recognize various human activity signals collected using smartphone sensors. This paper presents a comprehensive survey of smartphone sensor based human activity monitoring and recognition using various ML and DL techniques to address the above mentioned challenges. This study unveils the “research gaps in the field of HAMR, to provide the future research directions in HAMR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Amiribesheli M, Benmansour A, Bouchachia A (2015) A review of smart homes in healthcare. J Ambient Intell Humaniz Comput 6:495–517

    Article  Google Scholar 

  • Anjum A, Ilyas MU (2013) Activity recognition using smartphone sensors. In IEEE conference on consumer communications and networking

  • Antos SA, Albert MV, Kording KP (2013) Hand, belt, pocket or bag: practical activity tracking with mobile phones. J Neorosci Methods

  • Barua A, Masum AKM, Hossain ME, Bahadur EH, Alam MS (2019) A study on human activity recognition using gyroscope, accelerometer, temperature and humidity data. In 2019 international conference on electrical, computer and communication engineering (ecce), 1–6

  • Bayat A, Pomplun M, Tran DA (2014) A study on human activity recognition using accelerometer data from smartphones. Proc Comput Sci 34:450–457

    Article  Google Scholar 

  • Bayndr L (2017) A survey of people-centric sensing studies utilizing mobile phone sensors. J Ambient Intell Smart Environ 9(4):421–448

    Article  Google Scholar 

  • Bengio Y (2012) Practical recommendations for gradient-based training of deep architectures. CoRR arXiv:abs/1206.5533

  • Bhattacharya S, Nurmi P, Hammerla N, Plotz T (2014) Using unlabeled data in a sparse-coding framework for human activity recognition. Pervasive Mobile Comput 15:242–262

    Article  Google Scholar 

  • Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  MATH  Google Scholar 

  • Bulbul E, Cetin A, Dogru IA (2018) Human activity recognition using smartphones. In 2018 2nd international symposium on multidisciplinary studies and innovative technologies (ismsit), 1–6

  • Cabrita M, Lousberg R, Tabak M, Hermens HJ, Vollenbroek-Hutten MMR (2017) An exploratory study on the impact of daily activities on the pleasure and physical activity of older adults. Eur Rev Aging Phys Act 14(1):1

    Article  Google Scholar 

  • Calvaresi D, Cesarini D, Sernani P, Marinoni M, Dragoni AF, Sturm A (2017) Exploring the ambient assisted living domain: a systematic review. J Ambient Intell Humaniz Comput 8:239–257

    Article  Google Scholar 

  • Cardoso HL, Moreira JM (2016) Human activity recognition by means of online semi-supervised learning. IEEE International Conference on Mobile Data Management (MDM), IEEE, Porto

  • Catal C, SelinTufekci EP, Kocabag G (2015) On the use of ensemble of classifiers for accelerometer-based activity recognition. Appl Soft Comput 37:1018–1022

    Article  Google Scholar 

  • Chen Y, Shen C (2017) Performance analysis of smartphone-sensor behavior for human activity recognition. IEEE Access 5:3095–3110

    Article  Google Scholar 

  • Chen Z, Jiang C, Xie L (2019) A novel ensemble elm for human activity recognition using smartphone sensors. IEEE Trans Ind Inf 15(5):2691–2699

    Article  Google Scholar 

  • Chen Y, Xue Y (2015) A deep learning approach to human activity recognition based on single accelerometer. In 2015 ieee international conference on systems, man, and cybernetics, 1488–1492

  • Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  • Cvetkovic B, Szeklicki R, Janko V, Lutomski P, Lustrek M (2017) Real-time activity monitoring with a wristband and a smartphone. Inf Fus

  • Dangu Elu Beily M, Badjowawo MD, Bekak DO, Dana S (2016) A sensor based on recognition activities using smartphone. In 2016 international seminar on intelligent technology and its applications (isitia), 393–398

  • Garcia-Ceja E, Riegler M, Nordgreen T, Jakobsen P, Oedegaard KJ, Trresen J (2018) Mental health monitoring with multimodal sensing and machine learning: a survey. Pervas Mobile Comput

  • Ghosh S, Mitra J, Karunanithi M, Dowling J (2015) Human activity recognition from smart-phone sensor data using a multi-class ensemble learning in home monitoring. Stud Health Technol Inform 214:62–67

    Google Scholar 

  • Gravenhorst F, Muaremi A, Bardram J, Grnerbl A, Mayora O, Wurzer G, Frost M, Osmani V, Arnrich B, Lukowicz P, Trster G (2015) Mobile phones as medical devices in mental disorder treatment: an overview. Pers Ubiquit Comput 19(2):335–353

    Article  Google Scholar 

  • Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1):489–501

    Article  Google Scholar 

  • Jain A, Kanhangad V (2018) Human activity classification in smartphones using accelerometer and gyroscope sensors. IEEE Sens J 18(3):1169–1177

    Article  Google Scholar 

  • Kakria P, Tripathi NK, Kitipawang P (2015) A realtime health monitoring system for remote cardiac patients using smartphone and wearable sensors. Int J Telemed Appl

  • Kim Y, Ghorpade A, Zhao F, Pereira FC, Zegras PC, Ben-Akiva M (2018) Activity recognition for a smartphone and web-based human mobility sensing system. IEEE Intell Syst 33(4):5–23

    Article  Google Scholar 

  • Kwapisz Jennifer R, Weiss Gary M, Moore Samuel A (2011) Activity recognition using cell phone accelerometers. ACM SigKDD Explorations Newslett 12 (2)

  • Lara OD, Labrador MA (2013) A survey on human activity recognition using wearable sensors. IEEE Commun Surv Tutor 15(3):1192–1209

    Article  Google Scholar 

  • Lee YS, Cho SB (2013) Activity recognition with android phone using mixture-of-experts co-trained with labeled and unlabled data. Neurocomputing

  • Lee SM, Yoon SM, Cho H (2017) Human activity recognition from accelerometer data using convolutional neural network. In 2017 ieee international conference on big data and smart computing (bigcomp), 131–134

  • Li P, Wang Y, Tian Y, Zhou T, Li J (2017) An automatic user-adapted physical activity classification method using smartphones. IEEE Trans Biomed Eng 64(3):706–714

    Google Scholar 

  • Li X, He Y, Jing X (2019) A survey of deep learning-based human activity recognition in radar. Remote Sens 11(9)

  • Malasinghe LP, Ramzan N, Dahal K (2019) Remote patient monitoring: a comprehensive study. J Ambient Intell Humaniz Comput 10(1):57–76

    Article  Google Scholar 

  • Mejia-Ricart LF, Helling P, Olmsted A (2017) Evaluate action primitives for human activity recognition using unsupervised learning approach. In 2017 12th international conference for internet technology and secured transactions (icitst), 186–188

  • Miao F, He Y, Liu J, Li Y, Ayoola I (2015) Identifying typical physical activity on smartphone with varying positions and orientations. Biomed Eng Online

  • Mohr DC, Zhang M, Schueller SM (2017) Personal sensing: understanding mental health using ubiquitous sensors and machine learning. Annu Rev Clin Psychol 13:23–47

    Article  Google Scholar 

  • Nicholas J, Larsen ME, Proudfoot J, Christensen H (2015) Mobile apps for bipolar disorder: a systematic review of features and content quality. J Med Internet Res 17(8)

  • Nurhanim K, Elamvazuthi I, Izhar LI, Ganesan T (2017) Classification of human activity based on smartphone inertial sensor using support vector machine. In 2017 ieee 3rd international symposium in robotics and manufacturing automation (roma), 1–5

  • Pantelopoulos A, Bourbakis NG (2010) A survey on wearable sensor-based systems for health monitoring and prognosis. IEEE Trans Syst Man Cybern Part C Appl Rev 40(1):1–12

    Article  Google Scholar 

  • Prabowo OM, Mutijarsa K, Supangkat SH (2016) Missing data handling using machine learning for human activity recognition on mobile device. In 2016 international conference on ict for smart society (iciss), 59–62

  • Ravi D, Wong C, Lo B, Yang G (2017) A deep learning approach to on-node sensor data analytics for mobile or wearable devices. IEEE J Biomed Health Inform 21(1):56–64

    Article  Google Scholar 

  • Reyes-Ortiz JL, Oneto L, Sam A, Parra X, Anguita D (2016) Transition-aware human activity recognition using smartphones. Neurocomputing 171:754–767

    Article  Google Scholar 

  • Ronao CA, Cho S-B (2016) Human activity recognition with smartphone sensors using deep learning neural networks. Expert Syst Appl 59:235–244

    Article  Google Scholar 

  • Saha J, Chakraborty S, Chowdhury C, Biswas S, Aslam N (2017) Designing device independent two-phase activity recognition framework for smartphones. In 2017 ieee 13th international conference on wireless and mobile computing, networking and communications (wimob), 257–264

  • Sorkun MC, Dani?man AE, ?ncel D (2018) Human activity recognition with mobile phone sensors: Impact of sensors and window size. In 2018 26th signal processing and communications applications conference (siu), 1–4

  • Stephens J, Allen J (2013) Mobile phone interventions to increase physical activity and reduce weight: a systematic review. J Cardiovasc Nurs 28(4):320

    Article  Google Scholar 

  • Sun Z, Tang S, Huang H, Zhu Z, Guo H, Sun YE, Huang LS (2017) Sos: real-time and accurate physical assault detection using smartphone. Peer-to-Peer Netw Appl 10(2):395–410

    Article  Google Scholar 

  • Suto J, Oniga S (2018) Efficiency investigation of artificial neural networks in human activity recognition. J Ambient Intell Human Comput 9 (1049)

  • Tian Y, Chen W (2016) Mems-based human activity recognition using smartphone. In 2016 35th chinese control conference (ccc), 3984–3989

  • Tran DN, Phan DD (2016) Human activities recognition in android smartphone using support vector machine. In 2016 7th international conference on intelligent systems, modelling and simulation (isms), 64–68

  • Voicu RA, Dobre C, Bajenaru L, Ciobanu RI (2019) Human physical activity recognition using smartphone sensors. Sensors 19 (3)

  • Wang J, Wang Y, Wei C, Yao N, Yuan A, Shan Y, Yuan C (2014) Smartphone interventions for long-term health management of chronic diseases: an integrative review. Telemed e-Health 20(6):570–583

    Article  Google Scholar 

  • Wang A, Chen G, Yang J, Zhao S, Chang C (2016) A comparative study on human activity recognition using inertial sensors in a smartphone. IEEE Sens J 16(11):4566–4578

    Article  Google Scholar 

  • Wang J, Chen Y, Hao S, Peng X, Hu L (2018) Deep learning for sensor-based activity recognition. Pattern Recogn Lett

  • Weenk M, Alken APB, Engelen LJLPG, Bredie SJH, van de Belt TH, van Goor H (2018) Stress measurement in surgeons and residents using a smart patch. Am J Surg 216(2):361–368

    Article  Google Scholar 

  • Yang JB, Nguyen MN, San PP, Li XL, Krishnaswamy S (2015) Deep convolutional neural networks on multichannel time series for human activity recognition. In Proceedings of the twenty-fourth international joint conference on artificial intelligence (ijcai ), 995–4001

  • Yuan G, Wang Z, Meng F, Yan Q, Xia S (2018) An overview of human activity recognition based on smartphone. Sensor Rev

  • Yu S, Qin L (2018) Human activity recognition with smartphone inertial sensors using bidir-lstm networks. In 2018 3rd international conference on mechanical, control and computer engineering (icmcce), 219–224

  • Zeng M, Nguyen LT, Yu B, Mengshoel OJ, Zhu J, Wu P, Zhang J (2014) Convolutional neural networks for human activity recognition using mobile sensors. In 6th international conference on mobile computing, applications and services, 197–205

  • Zhang S, McCullagh P, Zheng H, Nugent C (2017) Situation awareness inferred from posture transition and location: derived from smartphone and smart home sensors. IEEE Trans Hum Mach Syst 47(6):814–821

    Article  Google Scholar 

  • Zhu W, Lan C, Xing J, Zeng W, Li Y, Shen L, Xie X (2016) Co-occurrence feature learning for skeleton based action recognition using regularized deep LSTM networks, Vol. 2

Download references

Author information

Authors and Affiliations

  1. Banasthali Vidyapith, Tonk, Rajasthan, India

    Dipanwita Thakur

  2. Maulana Abul Kalam Azad University of Technology, Kolkata, WB, India

    Suparna Biswas

Authors
  1. Dipanwita Thakur
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Suparna Biswas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dipanwita Thakur.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Thakur, D., Biswas, S. Smartphone based human activity monitoring and recognition using ML and DL: a comprehensive survey. J Ambient Intell Human Comput 11, 5433–5444 (2020). https://doi.org/10.1007/s12652-020-01899-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-020-01899-y

Keywords

Search

Navigation