Skip to main content
SpringerLink
Log in

Real-Time Hand Gesture Recognition: A Long Short-Term Memory Approach with Electromyography

  • Conference paper
  • First Online:
Advances and Applications in Computer Science, Electronics and Industrial Engineering (CSEI 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1078))

Abstract

Hand gestures are a non-verbal type of communication ideally suited for Human-Machine Interaction. Nevertheless, accuracy rates and response times still are a matter of research. One unattended problem has been the difficulty and vagueness of the evaluation of the models proposed in the literature. In this paper, a protocol for evaluating recognition is proposed. A Hand Gesture Recognition system using electromyography signals (EMG) is also presented. This model works in real time, is user dependent and is based in Long Short-Term Memory Networks. The model recognizes 5 different classes (wave in, wave out, fist, open, pinch) apart from the relax state. A data set with 120 people was collected using the commercial device Myo Armband. The data set was divided 50% for tuning and 50% for testing. Following the evaluation protocol proposed, the presented model achieves a 95.79% in classification and a 88.1% in recognition accuracy. An analysis of the characteristics of this model shows the advantage over similar models and its capability for being applied in all sort of fields.

The authors gratefully acknowledge the financial support provided by the Escuela Politécnica Nacional for the development of the research project PIJ-16-13.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Abhishek, K.S., Qubeley, L.C.K., Ho, D.: Glove-based hand gesture recognition sign language translator using capacitive touch sensor. In: 2016 IEEE International Conference on Electron Devices and Solid-State Circuits (EDSSC), pp. 334–337. IEEE (2016)

    Google Scholar 

  2. Ameur, S., Khalifa, A.B., Bouhlel, M.S.: A comprehensive leap motion database for hand gesture recognition. In: 2016 7th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications, SETIT 2016, pp. 514–519. IEEE (2017). https://doi.org/10.1109/SETIT.2016.7939924. http://ieeexplore.ieee.org/document/7939924/

  3. Athira, P.K., Sruthi, C.J., Lijiya, A.: A signer independent sign language recognition with co-articulation elimination from live videos: an indian scenario. J. King Saud Univ. Comput. Inf. Sci. (2019). https://doi.org/10.1016/j.jksuci.2019.05.002. https://www.sciencedirect.com/science/article/pii/S131915781831228X

  4. Barros, P., Maciel-Junior, N.T., Fernandes, B.J., Bezerra, B.L., Fernandes, S.M.: A dynamic gesture recognition and prediction system using the convexity approach. Comput. Vis. Image Underst. 155, 139–149 (2017). https://doi.org/10.1016/j.cviu.2016.10.006. https://www.sciencedirect.com/science/article/pii/S107731421630159X

    Article  Google Scholar 

  5. Belgacem, S., Chatelain, C., Paquet, T.: Gesture sequence recognition with one shot learned CRF/HMM hybrid model. Image Vis. Comput. 61, 12–21 (2017). https://doi.org/10.1016/j.imavis.2017.02.003. https://www.sciencedirect.com/science/article/pii/S0262885617300471

    Article  Google Scholar 

  6. Benalcázar, M.E., Anchundia, C.E., Zea, J.A., Zambrano, P., Jaramillo, A.G., Segura, M.: Real-time hand gesture recognition based on artificial feed-forward neural networks and EMG. In: European Signal Processing Conference, September 2018, pp. 1492–1496 (2018). https://doi.org/10.23919/EUSIPCO.2018.8553126

  7. Benalcazar, M.E., Motoche, C., Zea, J.A., Jaramillo, A.G., Anchundia, C.E., Zambrano, P., Segura, M., Benalcazar Palacios, F., Perez, M.: Real-time hand gesture recognition using the Myo armband and muscle activity detection. In: 2017 IEEE 2nd Ecuador Technical Chapters Meeting, ETCM 2017, January 2017, pp. 1–6 (2018). https://doi.org/10.1109/ETCM.2017.8247458

  8. Benatti, S., Rovere, G., Bosser, J., Montagna, F., Farella, E., Glaser, H., Schonle, P., Burger, T., Fateh, S., Huang, Q., Benini, L.: A sub-10 mW real-Time implementation for EMG hand gesture recognition based on a multi-core biomedical SoC. In: Proceedings - 2017 7th International Workshop on Advances in Sensors and Interfaces, IWASI 2017, pp. 139–144. IEEE (2017)

    Google Scholar 

  9. Bonato, P., D’Alessio, T., Knaflitz, M.: A statistical method for the measurement of muscle activation intervals from surface myoelectric signal during gait. IEEE Trans. Biomed. Eng. 45(3), 287–299 (1998). https://doi.org/10.1109/10.661154. http://www.ncbi.nlm.nih.gov/pubmed/9509745, http://ieeexplore.ieee.org/document/661154/

    Article  Google Scholar 

  10. Cartagena, P., Naranjo, J., Saltos, L., Garcia, C., Garcia, M.: Multifunctional exoskeletal orthosis for hand rehabilitation based on virtual reality. Adv. Intell. Syst. Comput. 884, 209–221 (2019). https://doi.org/10.1007/978-3-030-02828-2_16

    Article  Google Scholar 

  11. Dimitrova, N., Dimitrov, G.: Electromyography (EMG) Modeling. In: Wiley Encyclopedia of Biomedical Engineering. Wiley, Hoboken (2006). https://doi.org/10.1002/9780471740360.ebs0656. http://doi.wiley.com/10.1002/9780471740360.ebs0656

  12. Duda, R.O., Hart, P.E.P.E., Stork, D.G.: Pattern Classification. Wiley (2001). https://www.wiley.com/en-ec/Pattern+Classification%2C+2nd+Edition-p-9780471056690

  13. Farina, D., Jiang, N., Rehbaum, H., Holobar, A., Graimann, B., Dietl, H., Aszmann, O.C.: The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges. IEEE Trans. Neural Syst. Rehabil. Eng. 22(4), 797–809 (2014)

    Article  Google Scholar 

  14. Grif, H.S., Farcas, C.C.: Mouse cursor control system based on hand gesture. Procedia Technol. 22, 657–661 (2016). https://doi.org/10.1016/J.PROTCY.2016.01.137. https://www.sciencedirect.com/science/article/pii/S2212017316001389

    Article  Google Scholar 

  15. Huang, Y., Guo, W., Liu, J., He, J., Xia, H., Sheng, X., Wang, H., Feng, X., Shull, P.B.: Preliminary testing of a hand gesture recognition wristband based on EMG and inertial sensor fusion. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 9244, pp. 359–367. Springer, Cham (2015)

    Google Scholar 

  16. Iyer, D., Mohammad, F., Guo, Y., Al Safadi, E., Smiley, B.J., Liang, Z., Jain, N.K.: Generalized hand gesture recognition for wearable devices in IoT: application and implementation challenges. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9729, pp. 346–355. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41920-6_26. http://link.springer.com/10.1007/978-3-319-41920-626

  17. Jacob, M.G., Wachs, J.P., Packer, R.A.: Hand-gesture-based sterile interface for the operating room using contextual cues for the navigation of radiological images. J. Am. Med. Inform. Assoc. 20(E1), e183–e186 (2013). https://doi.org/10.1136/amiajnl-2012-001212. http://www.ncbi.nlm.nih.gov/pubmed/23250787, http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC3715344, https://academic.oup.com/jamia/article-lookup/doi/10.1136/amiajnl-2012-001212

  18. Jiang, X., Merhi, L.K., Xiao, Z.G., Menon, C.: Exploration of force myography and surface electromyography in hand gesture classification. Med. Eng. Phys. 41, 63–73 (2017). https://doi.org/10.1016/j.medengphy.2017.01.015. http://www.ncbi.nlm.nih.gov/pubmed/28161107, https://linkinghub.elsevier.com/retrieve/pii/S1350453317300176

    Article  Google Scholar 

  19. Joshi, A., Monnier, C., Betke, M., Sclaroff, S.: Comparing random forest approaches to segmenting and classifying gestures. Image Vis. Comput. 58, 86–95 (2017). https://doi.org/10.1016/j.imavis.2016.06.001. https://www.sciencedirect.com/science/article/pii/S0262885616300993

    Article  Google Scholar 

  20. Kim, S.Y., Han, H.G., Kim, J.W., Lee, S., Kim, T.W.: A hand gesture recognition sensor using reflected impulses. IEEE Sensors J. 17(10), 2975–2976 (2017). https://doi.org/10.1109/JSEN.2017.2679220. http://ieeexplore.ieee.org/document/7874149/

    Article  Google Scholar 

  21. Li, G., Zhang, R., Ritchie, M., Griffiths, H.: Sparsity-based dynamic hand gesture recognition using micro-Doppler signatures. In: 2017 IEEE Radar Conference (RadarConf). pp. 0928–0931. IEEE (2017). https://doi.org/10.1109/RADAR.2017.7944336. http://ieeexplore.ieee.org/document/7944336/

  22. Li Yang, Tian Yantao, Chen Wanzhong: Multi-pattern recognition of sEMG based on improved BP neural network algorithm. In: Proceedings of the 29th Chinese Control Conference. IEEE (2010). https://ieeexplore.ieee.org/abstract/document/5573567/similar#similar

  23. Liu, H., Wang, L.: Gesture recognition for human-robot collaboration: a review. Int. J. Ind. Ergon. 68, 355–367 (2018). https://doi.org/10.1016/j.ergon.2017.02.004. https://www.sciencedirect.com/science/article/pii/S0169814117300690

    Article  Google Scholar 

  24. Mohanty, A., Rambhatla, S.S., Sahay, R.R.: Deep gesture: static hand gesture recognition using CNN. In: Advances in Intelligent Systems and Computing, AISC, vol. 460, pp. 449–461. Springer, Singapore (2017). https://doi.org/10.1007/978-981-10-2107-7_41. http://link.springer.com/10.1007/978-981-10-2107-741

  25. Mordhorst, M., Heidlauf, T., Röhrle, O.: Mathematically modelling surface EMG signals. PAMM 14(1), 123–124 (2014). https://doi.org/10.1002/pamm.201410049. http://doi.wiley.com/10.1002/pamm.201410049

    Article  Google Scholar 

  26. Moschetti, A., Fiorini, L., Esposito, D., Dario, P., Cavallo, F.: Recognition of daily gestures with wearable inertial rings and bracelets. Sensors 16(8), 1341 (2016). https://doi.org/10.3390/s16081341. http://www.ncbi.nlm.nih.gov/pubmed/27556473, http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC5017504, http://www.mdpi.com/1424-8220/16/8/1341

  27. Motoche, C., Benalcázar, M.E.: Real-time hand gesture recognition based on electromyographic signals and artificial neural networks. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), LNCS, vol. 11139, pp. 352–361. Springer, Cham (2018)

    Google Scholar 

  28. Palmeri, M., Vella, F., Infantino, I., Gaglio, S.: Sign languages recognition based on neural network architecture. In: Smart Innovation, Systems and Technologies, vol. 76, pp. 109–118. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-59480-4_12. http://link.springer.com/10.1007/978-3-319-59480-412

  29. Raez, M.B.I., Hussain, M.S., Mohd-Yasin, F.: Techniques of EMG signal analysis: detection, processing, classification and applications. Biol. Proced. Online 8, 11–35 (2006). https://doi.org/10.1251/bpo115. http://www.ncbi.nlm.nih.gov/pubmed/16799694, http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC1455479

  30. Rodriguez-Falces, J., Navallas, J., Mal, A.: EMG modeling. In: Computational Intelligence in Electromyography Analysis - A Perspective on Current Applications and Future Challenges. InTech (2012). https://doi.org/10.5772/50304

  31. Saggio, G., Orengo, G., Pallotti, A., Errico, V., Ricci, M.: Sensory systems for human body gesture recognition and motion capture. In: 2018 International Symposium on Networks, Computers and Communications, ISNCC 2018, pp. 1–6. IEEE (2018). https://doi.org/10.1109/ISNCC.2018.8531054. https://ieeexplore.ieee.org/document/8531054/

  32. Scherer, R., Rao, R.: Non-manual control devices. In: Handbook of Research on Personal Autonomy Technologies and Disability Informatics, pp. 233–250. IGI Global (2011). https://doi.org/10.4018/978-1-60566-206-0.ch015. http://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/978-1-60566-206-0.ch015

  33. Sidig, A.a.I., Luqman, H., Mahmoud, S.A.: Arabic sign language recognition using optical flow-based features and HMM. In: Lecture Notes on Data Engineering and Communications Technologies, pp. 297–305. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59427-9_32. http://link.springer.com/10.1007/978-3-319-59427-932

  34. Sonkusare, J.S., Chopade, N.B., Sor, R., Tade, S.L.: A review on hand gesture recognition system. In: 2015 International Conference on Computing Communication Control and Automation, pp. 790–794. IEEE (2 2015). https://doi.org/10.1109/ICCUBEA.2015.158. http://ieeexplore.ieee.org/document/7155956/

  35. Tavakoli, M., Benussi, C., Lourenco, J.L.: Single channel surface EMG control of advanced prosthetic hands: a simple, low cost and efficient approach. Expert Syst. Appl. 79, 322–332 (2017). https://doi.org/10.1016/j.eswa.2017.03.012. https://www.sciencedirect.com/science/article/pii/S0957417417301574

    Article  Google Scholar 

  36. Wachs, J., Stern, H., Edan, Y., Gillam, M., Feied, C., Smith, M., Handler, J.: A real-time hand gesture interface for medical visualization applications. In: Advances in Soft Computing, vol. 36, pp. 153–162. Springer, Heidelberg (2006). https://doi.org/10.1007/978-3-540-36266-1_15. http://link.springer.com/10.1007/978-3-540-36266-115

  37. Wang, C., Liu, Z., Zhu, M., Zhao, J., Chan, S.C.: A hand gesture recognition system based on canonical superpixel-graph. Signal Process. Image Commun. 58, 87–98 (2017). https://doi.org/10.1016/j.image.2017.06.015. https://www.sciencedirect.com/science/article/abs/pii/S0923596517301224

    Article  Google Scholar 

  38. Wang, N., Lao, K., Zhang, X.: Design and myoelectric control of an anthropomorphic prosthetic hand. J. Bionic Eng. 14(1), 47–59 (2017). https://doi.org/10.1016/S1672-6529(16)60377-3. https://www.sciencedirect.com/science/article/pii/S1672652916603773

    Article  Google Scholar 

  39. Weiss, L.D., Weiss, J.M., Silver, J.K.: Easy EMG: A Guide to Performing Nerve Conduction Studies and Electromyography. Elsevier (2015)

    Google Scholar 

  40. Wipfli, R., Dubois-Ferrière, V., Budry, S., Hoffmeyer, P., Lovis, C.: Gesture-controlled image management for operating room: a randomized crossover study to compare interaction using gestures, mouse, and third person relaying. PLoS ONE 11(4), e0153596 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departamento de Informática y Ciencias de la Computación (DICC), Escuela Politécnica Nacional, Quito, Ecuador

    Jonathan A. Zea & Marco E. Benalcázar

Authors
  1. Jonathan A. Zea
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marco E. Benalcázar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jonathan A. Zea or Marco E. Benalcázar .

Editor information

Editors and Affiliations

  1. Faculty of Information Technology and Communication Sciences, Tampere University, Tampere, Finland

    Jyrki Nummenmaa

  2. Faculty of Engineering in Bilbao, University of the Basque Country (UPV/EHU), Bilbao, Spain

    Federico Pérez-González

  3. Department of Management, Escola Tècnica Superior d’Enginyeria Industrial de Barcelona (ETSEIB), Universitat Politècnica de Catalunya, Barcelona, Spain

    Bruno Domenech-Lega

  4. Division of Geotechnical Engineering and Geosciences; Department of Civil and Environmental Engineering, Universitat Politècnica de Catalunya, Barcelona, Spain

    Jean Vaunat

  5. Faculty of Software, Electronics and Industrial Engineering, Universidad Técnica de Ambato, Ambato, Ecuador

    Félix Oscar Fernández-Peña

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zea, J.A., Benalcázar, M.E. (2020). Real-Time Hand Gesture Recognition: A Long Short-Term Memory Approach with Electromyography. In: Nummenmaa, J., Pérez-González, F., Domenech-Lega, B., Vaunat, J., Oscar Fernández-Peña, F. (eds) Advances and Applications in Computer Science, Electronics and Industrial Engineering. CSEI 2019. Advances in Intelligent Systems and Computing, vol 1078. Springer, Cham. https://doi.org/10.1007/978-3-030-33614-1_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-33614-1_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-33613-4

  • Online ISBN: 978-3-030-33614-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation