Abstract
Sign language has been used to communicate with people who are hard of hearing in conveying their thoughts and ideas to ordinary people. People may readily express thoughts using this sort of gesture-based language, which reduces barriers caused by hearing problems. The major issue is that the vast majority of the population lacks the knowledge of using sign language. Sign language detection from live video footage is a challenging problem that can bridge the communication gap. This paper proposes a method for identifying sign language motions in real-time video data that combines computer vision and deep learning approaches. The major contribution of the proposed model is stacking the long short-term memory (LSTM) and gated recurrent unit (GRU) architecture called LSTM-GRU to detect and classify signs from sign language videos. Our built model is run on a real-time video stream using the camera input, and by stacking the LSTM and GRU, we optimize model performance and reach 94.4% prediction accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bilge YC, Cinbis RG, Ikizler-Cinbis N (2022) Towards zero-shot sign language recognition. IEEE Trans Patt Anal Mach Intell 45(1):1217–1232
Minu RI (2023) A extensive survey on sign language recognition methods. In: 2023 7th international conference on computing methodologies and communication (ICCMC), IEEE, pp 613–619
Tamiru NK, Tekeba M, Salau AO (2022) Recognition of Amharic sign language with Amharic alphabet signs using ANN and SVM. Visual Comp, pp 1–16
Akash, SK, Chakraborty D, Kaushik MM, Babu BS, Rahman Zishan, MdS (2023) Action recognition based real-time bangla sign language detection and sentence formation. In: 2023 3rd international conference on robotics, electrical and signal processing techniques (ICREST). IEEE, pp 311–315
Sreemathy R, Turuk M, Kulkarni I, Khurana S (2023) Sign language recognition using artificial intelligence. Educ Inf Tech 28(5):5259–5278
Aldhahri E, Aljuhani R, Alfaidi A, Alshehri B, Alwadei H, Aljojo N, Alshutayri A, Almazroi A (2023) Arabic sign language recognition using convolutional neural network and mobilenet. Arab J Sci Eng 48(2):2147–2154
Shin J, Miah ASM, Hasan MAM, Hirooka K, Suzuki K, Lee H-S, Jang S-W (2023) Korean sign language recognition using transformer-based deep neural network. Appl Sci 13(5):3029
Nandi U, Ghorai A, Marjit Singh M, Changdar C, Bhakta S, Kumar Pal R (2023) Indian sign language alphabet recognition system using CNN with diffGrad optimizer and stochastic pooling. Multimedia Tools Appl 82(7):9627–9648
Grishchenko I, Bazarevsky V, Zanfir A, Bazavan EG, Zanfir M, Richard Yee, Raveendran K, Zhdanovich M (2022) Matthias grundmann, and cristian sminchisescu. Blazepose ghum holistic: real-time 3d human landmarks and pose estimation. arXiv preprint arXiv:2206.11678
Gong, S, Li M, Feng J, Wu Z, Kong LP (2022) Diffuseq: sequence to sequence text generation with diffusion models. ArXiv preprint arXiv:2210.08933
Goyal K (2023) Indian sign language recognition using mediapipe holistic. arXiv preprint arXiv:2304.10256
Kothadiya D, Bhatt C, Sapariya K, Patel K, Gil-González A-B, Corchado JM (2022) Deepsign: sign language detection and recognition using deep learning. Electronics 11(11):1780
Bora J, Dehingia S, Boruah A, Chetia AA, Gogoi D (2023) Real-time assamese sign language recognition using mediapipe and deep learning. Proc Comput Sci 218:1384–1393
Fathima MD, Hariharan R, Singh PK, Kumar P, Ramya S, Ammal MSSR (2023) Real time face mask detection using Mobilenetv2 algorithm.“ In: Recent trends in computational intelligence and its application: proceedings of the 1st international conference on recent trends in information technology and its application (ICRTITA, 22). CRC Press, p 215
Shewalkar A, Nyavanandi D, Ludwig SA (2019) Performance evaluation of deep neural networks applied to speech recognition: RNN, LSTM and GRU. J Artif Intell Soft Comput Res 9(4):235–245
Dhilsath FM, Justin Samuel S, Raja SP (2023) HDDSS: an enhanced heart disease decision support system using RFE-ABGNB algorithm. Int J Interact Multimedia and Artif Intell 527
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Dhilsath Fathima, M., Hariharan, R., Shome, S., Kharsyiemlieh, M., Deepa, J., Jayanthi, K. (2024). Sign Language Interpreter Using Stacked LSTM-GRU. In: Sharma, H., Chakravorty, A., Hussain, S., Kumari, R. (eds) Artificial Intelligence: Theory and Applications. AITA 2023. Lecture Notes in Networks and Systems, vol 844. Springer, Singapore. https://doi.org/10.1007/978-981-99-8479-4_30
Download citation
DOI: https://doi.org/10.1007/978-981-99-8479-4_30
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8478-7
Online ISBN: 978-981-99-8479-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)