Skip to main content
SpringerLink
Log in

mRMR-PSO: A Hybrid Feature Selection Technique with a Multiobjective Approach for Sign Language Recognition

  • Research Article-Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

In this paper, a hybrid feature selection technique named mRMR-PSO has been proposed with a multiobjective approach for automatic recognition of sign language. The features are extracted by histogram of oriented gradient (HOG) for input gestures. Here, mRMR is used as a pre-processor for the removal of redundant and irrelevant features reducing the computational burden of PSO. Further, PSO chooses a feature subset having maximum accuracy with minimum features based on the classifier performance. A multi-class support vector machine is used as a classifier. The effectiveness of the proposed approach has been exhaustively tested on seven publically available benchmark datasets for three different sign languages with both uniform and complex backgrounds. The experimental results obtained by mRMR-PSO achieve more accurate classification with reduced feature vector size as compared to HOG (no FS), mRMR, PSO. Furthermore, Friedman’s test has been conducted to show the significance of mRMR-PSO in comparison to others.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. https://www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-loss

  2. Cheok, M.J.; Omar, Z.; Jaward, M.H.: A review of hand gesture and sign language recognition techniques. Int. J. Mach. Learn. Cybern. 10(1), 131–153 (2019)

    Article  Google Scholar 

  3. Rastgoo, R.; Kiani, K.; Escalera, S.: Sign language recognition: a deep survey. Expert Syst. Appl. 164, 113794 (2021)

    Article  Google Scholar 

  4. Wadhawan, A.; Kumar, P.: Deep learning-based sign language recognition system for static signs. Neural Comput. Appl. 32(12), 7957–7968 (2020)

    Article  Google Scholar 

  5. Rastgoo, R.; Kiani, K.; Escalera, S.: Hand sign language recognition using multi-view hand skeleton. Expert Syst. Appl. 150, 113336 (2020)

    Article  Google Scholar 

  6. Rastgoo, R.; Kiani, K.; Escalera, S.: Hand pose aware multimodal isolated sign language recognition. Multimed. Tools Appl. 80(1), 127–163 (2021)

    Article  Google Scholar 

  7. Rastgoo, R.; Kiani, K.; Escalera, S.; Sabokrou, M.: Sign language production: a review. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3451–3461 (2021)

  8. Wadhawan, A.; Kumar, P.: Sign language recognition systems: a decade systematic literature review. Arch. Comput. Methods Eng. 28(3), 785–813 (2021)

    Article  Google Scholar 

  9. Dalal, N.; Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), vol 1, pp 886–893. IEEE (2005)

  10. Brezočnik, L.; Fister, I.; Podgorelec, V.: Swarm intelligence algorithms for feature selection: a review. Appl. Sci. 8(9), 1521 (2018)

    Article  Google Scholar 

  11. Xue, B.; Zhang, M.; Browne, W.N.: Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans. Cybern. 43(6), 1656–1671 (2012)

    Article  Google Scholar 

  12. Chen, B.; Hong, J.; Wang, Y.: The minimum feature subset selection problem. J. Comput. Sci. Technol. 12(2), 145–153 (1997)

    Article  MathSciNet  Google Scholar 

  13. Balasaraswathi, V.R.; Sugumaran, M.; Hamid, Y.: Feature selection techniques for intrusion detection using non-bio-inspired and bio-inspired optimization algorithms. J. Commun. Inf. Netw. 2(4), 107–119 (2017)

    Article  Google Scholar 

  14. Kamruzzaman, M.M.: Arabic sign language recognition and generating arabic speech using convolutional neural network. Wirel. Commun. Mobile Comput. 2020, 1–9 (2020). https://doi.org/10.1155/2020/3685614

    Article  Google Scholar 

  15. Cai, J.; Luo, J.; Wang, S.; Yang, S.: Feature selection in machine learning: a new perspective. Neurocomputing 300, 70–79 (2018)

    Article  Google Scholar 

  16. Tadist, K.; Najah, S.; Nikolov, N.S.; Mrabti, F.; Zahi, A.: Feature selection methods and genomic big data: a systematic review. J. Big Data 6(1), 1–24 (2019)

    Article  Google Scholar 

  17. Shroff, K.P.; Maheta, H.H.: A comparative study of various feature selection techniques in high-dimensional data set to improve classification accuracy. In: 2015 International Conference on Computer Communication and Informatics (ICCCI), pp. 1–6. IEEE (2015)

  18. Wu, J.; Sun, L.; Jafari, R.: A wearable system for recognizing American sign language in real-time using IMU and surface EMG sensors. IEEE J. Biomed. Health Inform. 20(5), 1281–1290 (2016)

    Article  Google Scholar 

  19. Engelbrecht, A.P.: Computational Intelligence: An Introduction. Wiley (2007)

    Book  Google Scholar 

  20. Kennedy, J.; Eberhart, R.: Particle swarm optimization. In: Proceedings of ICNN'95-International Conference on Neural Networks, vol. 4, pp. 1942–1948. IEEE (1995)

  21. Kennedy, J.; Eberhart, R.C.: A discrete binary version of the particle swarm algorithm. In: 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation, vol. 5, pp 4104–4108. IEEE (1997)

  22. Tran, B.; Xue, B.; Zhang, M.: Improved PSO for feature selection on high-dimensional datasets. In: Dick, G., et al. (Eds.) Simulated Evolution and Learning. SEAL 2014. Lecture Notes in Computer Science, Vol. 8886. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-13563-2_43

    Chapter  Google Scholar 

  23. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  24. Bay, H.; Ess, A.; Tuytelaars, T.; Van Gool, L.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008)

    Article  Google Scholar 

  25. Ojala, T.; Pietikäinen, M.; Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recognit. 29(1), 51–59 (1996)

    Article  Google Scholar 

  26. El-Gayar, M.M.; Soliman, H.: A comparative study of image low level feature extraction algorithms. Egypt. Inform. J. 14(2), 175–181 (2013)

    Article  Google Scholar 

  27. Tyagi, A.; Bansal, S.; & Kashyap, A.: Comparative analysis of feature detection and extraction techniques for vision-based ISLR system. In: 2020 Sixth International Conference on Parallel, Distributed and Grid Computing (PDGC), pp. 515–520. IEEE (2020)

  28. Joshi, G.; Singh, S.; Vig, R.: Taguchi-TOPSIS based HOG parameter selection for complex background sign language recognition. J. Visual Commun. Image Represent. 71, 102834 (2020)

    Article  Google Scholar 

  29. Patel, P.; Patel, N.: Vision based real-time recognition of hand gestures for Indian sign language using histogram of oriented gradients features. Int. J. Next Gener. Comput. 10(2), 92–102 (2019)

    Google Scholar 

  30. Kika, A.; Koni, A.: Hand Gesture Recognition Using Convolutional Neural Network and Histogram of Oriented Gradients Features. In: RTA-CSIT (pp. 75–79) (2018)

  31. Sharma, A.; Mittal, A.; Singh, S.; Awatramani, V.: Hand gesture recognition using image processing and feature extraction techniques. Procedia Comput. Sci. 173, 181–190 (2020)

    Article  Google Scholar 

  32. Jmaa, A.B.; Mahdi, W.; Jemaa, Y.B.; Hamadou, A.B.: Arabic sign language recognition based on HOG descriptor. In: Eighth International Conference on Graphic and Image Processing (ICGIP 2016), vol. 10225, p. 102250H. International Society for Optics and Photonics (2017)

  33. Sun, C.; Zhang, T.; Bao, B.K.; Xu, C.: Latent support vector machine for sign language recognition with Kinect. In: 2013 IEEE International Conference on Image Processing, pp. 4190–4194. IEEE (2013)

  34. Jangyodsuk, P.; Conly, C.; Athitsos, V.: Sign language recognition using dynamic time warping and hand shape distance based on histogram of oriented gradient features. In: Proceedings of the 7th International Conference on PErvasive Technologies Related to Assistive Environments, pp. 1–6 (2014)

  35. Agrawal, S.C.; Jalal, A.S.; Bhatnagar, C.:. Recognition of Indian Sign Language using feature fusion. In: 2012 4th International Conference on Intelligent Human Computer Interaction (IHCI), pp. 1–5. IEEE (2012)

  36. Aryanie, D.; Heryadi, Y.: American sign language-based finger-spelling recognition using k-Nearest Neighbors classifier. In: 2015 3rd International Conference on Information and Communication Technology (ICoICT), pp. 533–536. IEEE (2015)

  37. Hamed, A.; Belal, N.A.; Mahar, K.M.: Arabic sign language alphabet recognition based on HOG-PCA using Microsoft kinect in complex backgrounds. In: 2016 IEEE 6th International Conference on Advanced Computing (IACC), pp. 451–458. IEEE (2016)

  38. Guo, D.; Zhou, W.; Li, H.; Wang, M.: Online early-late fusion based on adaptive HMM for sign language recognition. ACM Trans. Multimed. Comput. Commun. Appl. (TOMM) 14(1), 1–18 (2017)

    Google Scholar 

  39. Wu, J.; Tian, Z.; Sun, L.; Estevez, L.; Jafari, R.: Real-time American sign language recognition using wrist-worn motion and surface EMG sensors. In: 2015 IEEE 12th International Conference on Wearable and Implantable Body Sensor Networks (BSN), pp. 1–6. IEEE (2015)

  40. Zamani, M.; Kanan, H.R.: Saliency based alphabet and numbers of American sign language recognition using linear feature extraction. In: 2014 4th International Conference on Computer and Knowledge Engineering (ICCKE), pp. 398–403. IEEE (2014)

  41. Fregoso, J.; Gonzalez, C.I.; Martinez, G.E.: Parameter optimization of a convolutional neural network using particle swarm optimization. In: Castillo, O.; Melin, P. (Eds.) Fuzzy Logic Hybrid Extensions of Neural and Optimization Algorithms: Theory and Applications. Studies in Computational Intelligence, Vol. 940. Springer, Cham (2021)

    Google Scholar 

  42. Alshamlan, H.; Badr, G.; Alohali, Y.: mRMR-ABC: a hybrid gene selection algorithm for cancer classification using microarray gene expression profiling. Int. BioMed Res. (2015). https://doi.org/10.1155/2015/604910

    Article  Google Scholar 

  43. Pirgazi, J.; Alimoradi, M.; Abharian, T.E.; Olyaee, M.H.: An efficient hybrid filter-wrapper metaheuristic-based gene selection method for high dimensional datasets. Sci. Rep. 9(1), 1–15 (2019)

    Article  Google Scholar 

  44. Ouarda, W.; Trichili, H.; Alimi, A. M.; & Solaiman, B.: Combined local features selection for face recognition based on Naïve Bayesian classification. In: 13th International Conference on Hybrid Intelligent Systems (HIS 2013), pp. 240–245. IEEE (2013)

  45. Khan, A.; Baig, A.R.: Multi-objective feature subset selection using mRMR based enhanced ant colony optimization algorithm (mRMR-EACO). J. Exp. Theor. Artif. Intell. 28(6), 1061–1073 (2016)

    Article  Google Scholar 

  46. Das, S.P.; Talukdar, A.K.; Sarma, K.K.: Sign language recognition using facial expression. Procedia Comput. Sci. 58, 210–216 (2015)

    Article  Google Scholar 

  47. Peng, H.; Long, F.; Ding, C.: Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1226–1238 (2005)

    Article  Google Scholar 

  48. Tharwat, A.; Gaber, T.; Hassanien, A.E.; Shahin, M.K.; Refaat, B.: Sift-based arabic sign language recognition system. In: Afro-European Conference for Industrial Advancement, pp. 359–370. Springer, Cham (2015)

  49. Dahmani, D.; Larabi, S.: User-independent system for sign language finger spelling recognition. J. Vis. Commun. Image Represent. 25(5), 1240–1250 (2014)

    Article  Google Scholar 

  50. Abdel-Basset, M.; El-Shahat, D.; El-henawy, I.; de Albuquerque, V.H.C.; Mirjalili, S.: A new fusion of grey wolf optimizer algorithm with a two-phase mutation for feature selection. Expert Syst. Appl. 139, 112824 (2020)

    Article  Google Scholar 

  51. https://github.com/imRishabhGupta/Indian-Sign-Language-Recognition

  52. https://www.kaggle.com/ardamavi/sign-language-digits-dataset

  53. Triesch, J.; von der Malsburg, C.: Classification of hand postures against complex backgrounds using elastic graph matching. Image Vis. Comput. 20(13–14), 937–943 (2002)

    Article  Google Scholar 

  54. www.ece.nus.edu.sg/stfpage/elepv/NUS-HandSet/

  55. www.ieee-dataport.org/open-access/static-hand-gesture-asl-dataset

  56. Latif, G.; Mohammad, N.; Alghazo, J.; AlKhalaf, R.; AlKhalaf, R.: ArASL: Arabic Alphabets Sign Language dataset. Data Brief 23, 103777 (2019). https://doi.org/10.1016/j.dib.2019.103777

    Article  Google Scholar 

  57. Li, Y.; Xu, L.; Shu, W.; Mei, K.: AutoGesNet: auto gesture recognition network based on neural architecture search. In: 2020 12th International Conference on Advanced Computational Intelligence (ICACI), pp. 257–262. IEEE (2020)

  58. Adithya, V.; Rajesh, R.: A deep convolutional neural network approach for static hand gesture recognition. Procedia Comput. Sci. 171, 2353–2361 (2020)

    Article  Google Scholar 

  59. Mohanty, A.; Rambhatla, S.S.; Sahay, R.R.: Deep gesture: static hand gesture recognition using CNN. In: Proceedings of International Conference on Computer Vision and Image Processing, pp. 449–461. Springer, Singapore (2017)

  60. Kasukurthi, N.; Rokad, B.; Bidani, S.; & Dennisan, D.: American Sign Language Alphabet recognition using deep learning (2019). arXiv preprint arXiv:1905.05487

  61. Bheda, V.; Radpour, D.: Using deep convolutional networks for gesture recognition in American sign language (2017). arXiv preprint arXiv:1710.06836

  62. Oyedotun, O.K.; Khashman, A.: Deep learning in vision-based static hand gesture recognition. Neural Comput. Appl. 28(12), 3941–3951 (2017)

    Article  Google Scholar 

  63. Bantupalli, K.; Xie, Y.: American sign language recognition using deep learning and computer vision. In: 2018 IEEE International Conference on Big Data (Big Data), pp. 4896–4899. IEEE (2018)

  64. Sajanraj, T.D.; Beena, M.V.: Indian sign language numeral recognition using region of interest convolutional neural network. In: 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT), pp. 636–640. IEEE (2018)

Download references

Author information

Author notes

  1. Sandhya Rani Bansal & Rajeev Goel

    Present address: M.M.E.C, Mullana, Haryana, India

Authors and Affiliations

  1. MMICTBM, Mullana, Haryana, India

    Savita Wadhawan

  2. Government College, Naraingarh, Haryana, India

    Rajeev Goel

Authors
  1. Sandhya Rani Bansal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Savita Wadhawan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rajeev Goel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sandhya Rani Bansal.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bansal, S.R., Wadhawan, S. & Goel, R. mRMR-PSO: A Hybrid Feature Selection Technique with a Multiobjective Approach for Sign Language Recognition. Arab J Sci Eng 47, 10365–10380 (2022). https://doi.org/10.1007/s13369-021-06456-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-021-06456-z

Keywords

Search

Navigation