Skip to main content
SpringerLink
Log in

Shape oriented object recognition on grasp using features from enclosure based exploratory procedure

  • Regular Paper
  • Published:
International Journal of Intelligent Robotics and Applications Aims and scope Submit manuscript

Abstract

The potential of humans to recognize known objects while grasping, without the help of vision, is an exciting supposition to the robotics community. With a focus on reproducing such a natural aptitude in prosthetic hands, this paper reports a kinematic approach to exploring the human hand’s object recognition functionality during a grasp. Finger kinematics vary while grasping objects of different shapes and sizes. The authors emphasized learning the variations while grasping different objects through a forward kinematics model of the human hand. Finger joint kinematics for objects of two specific shape categories: spherical and cylindrical, were recorded during grasping experiments using a customized data glove to deduce the fingertip coordinates. An algorithm has been developed to derive novel three-dimensional grasp polyhedrons from fingertip coordinates. Areas of these polyhedrons and finger kinematics have been used as features to train classification algorithms. Comparing the recognition results using only finger kinematics as features revealed that the inclusion of the shape primitives increases the accuracies of the classifiers by 2–6% while recognizing the objects. This work analytically confirms that finger kinematics and the object’s shape primitives are vital information for visionless object recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  • Alexander, B., Viktor, K.: Proportions of hand segments. Int. J. Morphol. 28(3), 755–758 (2010)

    Google Scholar 

  • Allen, P., Michelman, P.: Acquisition and interpretation of 3-d sensor data from touch. IEEE Trans. Robot. Autom. 6(4), 397–404 (1990). https://doi.org/10.1109/70.59353

    Article  Google Scholar 

  • Ansuini, C., Cavallo, A., Koul, A., Jacono, M., Yang, Y., Becchio, C.: Predicting object size from hand kinematics: a temporal perspective. PLoS One 10(3), 0120–0432 (2015)

    Google Scholar 

  • Asada, H., Kitagawa, M.: Kinematic analysis and planning for form closure grasps by robotic hands. Robot. Comput. Integr. Manuf. 5(4), 293–299 (1989)

    Google Scholar 

  • Asif, U., Bennamoun, M., Sohel, F.A.: Rgb-d object recognition and grasp detection using hierarchical cascaded forests. IEEE Trans. Robot. 33(3), 547–564 (2017)

    Google Scholar 

  • Ayzenberg, V., Lourenco, S.F.: Skeletal descriptions of shape provide unique perceptual information for object recognition. Sci. Rep. 9(1), 1–13 (2019)

    Google Scholar 

  • Bandou, Y., Fukuda, O., Okumura, H., Arai, K., Bu, N.: Development of a prosthetic hand control system based on general object recognition analysis of recognition accuracy during approach phase. In: International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS), pp. 110–114. IEEE (2017)

  • Borra, S., Di Ciaccio, A.: Measuring the prediction error. a comparison of cross-validation, bootstrap and covariance penalty methods. Comput. Stat. Data Anal. 54(12), 2976–2989 (2010)

    MathSciNet  MATH  Google Scholar 

  • Boruah, A., Kakoty, N.M., Ali, T.: Object recognition based on surface detection—a review. Proc. Comput. Sci. 133, 63–74 (2018)

    Google Scholar 

  • Boruah, A., Kakoty, N.M., Ali, T.: Reasoning on objects’ geometric shapes for prosthetic hand grasping. In: Proceedings of the Advances in Robotics, pp. 1–6 (2019)

  • Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    MATH  Google Scholar 

  • Bullock, I.M., Zheng, J.Z., De La Rosa, S., Guertler, C., Dollar, A.M.: Grasp frequency and usage in daily household and machine shop tasks. IEEE Trans. Haptics 6(3), 296–308 (2013)

    Google Scholar 

  • Buxbaum, L.J., Kyle, K.M., Tang, K., Detre, J.A.: Neural substrates of knowledge of hand postures for object grasping and functional object use: evidence from fmri. Brain Res. 1117(1), 175–185 (2006)

    Google Scholar 

  • Cai, J., Luo, J., Wang, S., Yang, S.: Feature selection in machine learning: a new perspective. Neurocomputing 300, 70–79 (2018)

    Google Scholar 

  • Calli, B., Singh, A., Bruce, J., Walsman, A., Konolige, K., Srinivasa, S., Abbeel, P., Dollar, A.M.: Yale-CMU-Berkeley dataset for robotic manipulation research. Int. J. Robot. Res. 36(3), 261–268 (2017)

    Google Scholar 

  • Chakraborty, B.K., Sarma, D., Bhuyan, M.K., MacDorman, K.F.: Review of constraints on vision-based gesture recognition for human-computer interaction. IET Comput. Vis. 12(1), 3–15 (2018)

    Google Scholar 

  • Chen Chen, F., Appendino, S., Battezzato, A., Favetto, A., Mousavi, M., Pescarmona, F.: Constraint study for a hand exoskeleton: human hand kinematics and dynamics. J. Robot. 2013 (2013)

  • Cobos, S., Ferre, M., Ángel Sánchez-Urán, M., Ortego, J., Aracil, R.: Human hand descriptions and gesture recognition for object manipulation. Comput. Methods Biomech. Biomed. Eng. 13(3), 305–317 (2010)

    Google Scholar 

  • Cognolato, M., Graziani, M., Giordaniello, F., Saetta, G., Bassetto, F., Brugger, P., Caputo, B., Müller, H., Atzori, M.: Semi-automatic training of an object recognition system in scene camera data using gaze tracking and accelerometers. In: International Conference on Computer Vision Systems, pp. 175–184. Springer (2017)

  • Cordella, F., Zollo, L., Salerno, A., Accoto, D., Guglielmelli, E., Siciliano, B.: Human hand motion analysis and synthesis of optimal power grasps for a robotic hand. Int. J. Adv. Robot. Syst. 11(3), 37–49 (2014)

    Google Scholar 

  • Cotugno, G., Konstantinova, J., Althoefer, K., Nanayakkara, T.: Modelling the structure of object-independent human affordances of approaching to grasp for robotic hands. PLoS One 13(12–40) (2018)

  • Dang, H., Weisz, J., Allen, P.K.: Blind grasping: Stable robotic grasping using tactile feedback and hand kinematics. In: IEEE International Conference on Robotics and Automation, pp. 5917–5922 (2011)

  • Delgado, A., Jara, C., Torres, F.: In-hand recognition and manipulation of elastic objects using a servo-tactile control strategy. Robot. Comput. Integr. Manuf. 48, 102–112 (2017)

    Google Scholar 

  • Falco, P., Lu, S., Natale, C., Pirozzi, S., Lee, D.: A transfer learning approach to cross-modal object recognition: from visual observation to robotic haptic exploration. IEEE Trans. Robot. 35(4), 987–998 (2019)

    Google Scholar 

  • Feix, T., Romero, J., Schmiedmayer, H.B., Dollar, A.M., Kragic, D.: The grasp taxonomy of human grasp types. IEEE Trans. Hum. Mach. Syst. 46(1), 66–77 (2015)

    Google Scholar 

  • Genuer, R., Poggi, J.M., Tuleau-Malot, C.: Variable selection using random forests. Pattern Recogn. Lett. 31(14), 2225–2236 (2010)

    Google Scholar 

  • Gerlach, C.: Different measures of structural similarity tap different aspects of visual object processing. Front. Psychol. 8, 1404 (2017)

    Google Scholar 

  • Gorges, N., Navarro, S.E., Göger, D., Wörn, H.: Haptic object recognition using passive joints and haptic key features. In: International Conference on Robotics and Automation, pp. 2349–2355 (2010)

  • Gorges, N., Navarro, S.E., Wörn, H.: Haptic object recognition using statistical point cloud features. In: IEEE 15th International Conference on Advanced Robotics, pp. 15–20 (2011)

  • Hartenberg, R.S., Denavit, J.: A kinematic notation for lower pair mechanisms based on matrices. J. Appl. Mech. 77(2), 215–221 (1955)

    MathSciNet  MATH  Google Scholar 

  • Ho, T.K.: Random decision forests. In: IEEE 3rd International Conference on Document Analysis and Recognition, vol. 1, pp. 278–282 (1995)

  • Hussain, M., Wajid, S.K., Elzaart, A., Berbar, M.: A comparison of svm kernel functions for breast cancer detection. In: IEEE 8th International Conference Computer Graphics, Imaging and Visualization, pp. 145–150 (2011)

  • Iivarinen, J., Visa, A.J.: Shape recognition of irregular objects. In: Intelligent Robots and Computer Vision XV: Algorithms, Techniques, Active Vision, and Materials Handling, vol. 2904, pp. 25–32. International Society for Optics and Photonics (1996)

  • Iivarinen, J., Peura, M., Särelä, J., Visa, A.: Comparison of combined shape descriptors for irregular objects. In: BMVC. Citeseer (1997)

  • Jarque-Bou, N.J., Scano, A., Atzori, M., Müller, H.: Kinematic synergies of hand grasps: a comprehensive study on a large publicly available dataset. J. Neuroeng. Rehabil. 16(1), 63–77 (2019)

    Google Scholar 

  • Kamper, D.G., Cruz, E.G., Siegel, M.P.: Stereotypical fingertip trajectories during grasp. J. Neurophysiol. 90(6), 3702–3710 (2003)

    Google Scholar 

  • Kim, B.H., Yi, B.J., Oh, S.R., Suh, I.H.: Non-dimensionalized performance indices based optimal grasping for multi-fingered hands. Mechatronics 14(3), 255–280 (2004)

    Google Scholar 

  • Kimoto, A., Matsue, Y.: A new multifunctional tactile sensor for detection of material hardness. Trans. Instrum. Meas. 60(4), 1334–1339 (2011)

    Google Scholar 

  • Kira, K., Rendell, L.A.: A practical approach to feature selection. In: Machine Learning Proceedings 1992, pp. 249–256. Elsevier (1992)

  • Kucherhan, D.J., Goubran, M., da Fonseca, V.P., de Oliveira, T.E.A., Petriu, E.M., Groza, V.: Object recognition through manipulation using tactile enabled prosthetic fingers and feedback glove-experimental study. In: International Symposium on Medical Measurements and Applications (MeMeA), pp. 1–6. IEEE (2018)

  • Lakshiminarayana, K.: Mechanics of form closure. ASME paper, 78-DET-32 (1978)

  • Lederman, S.J., Klatzky, R.L.: Hand movements: a window into haptic object recognition. Cogn. Psychol. 19(3), 342–368 (1987)

    Google Scholar 

  • Lederman, S.J., Klatzky, R.L.: Extracting object properties through haptic exploration. Acta Physiol. (Oxf) 84(1), 29–40 (1993)

    Google Scholar 

  • Lenarčič, J., Bajd, T., Stanišić, M.M.: Kinematic model of the human hand. In: Robot Mechanisms, pp. 313–326. Springer (2013)

  • León, B., Sancho-Bru, J.L., Jarque-Bou, N.J., Morales, A., Roa, M.A.: Evaluation of human prehension using grasp quality measures. Int. J. Adv. Robot. Syst. 9(4), 112–124 (2012)

    Google Scholar 

  • Liu, H., Guo, D., Sun, F.: Object recognition using tactile measurements: kernel sparse coding methods. Trans. Instrum. Meas. 65(3), 656–665 (2016)

    Google Scholar 

  • Loncaric, S.: A survey of shape analysis techniques. Pattern Recogn. 31(8), 983–1001 (1998)

    Google Scholar 

  • Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Ojea, J.A., Goldberg, K.: Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics (2017). arXiv:1703.09312

  • Martinez-Hernandez, U., Dodd, T.J., Prescott, T.J.: Feeling the shape: active exploration behaviors for object recognition with a robotic hand. IEEE Trans. Syst. Man Cybern. Syst. 48(12), 2339–2348 (2017)

    Google Scholar 

  • Martinez-Martin, E., Del Pobil, A.P.: Object detection and recognition for assistive robots: experimentation and implementation. IEEE Robot. Autom. Mag. 24(3), 123–138 (2017)

    Google Scholar 

  • Ma’touq, J., Hu, T., Haddadin, S.: Sub-millimetre accurate human hand kinematics: from surface to skeleton. Comput. Methods Biomech. Biomed. Eng. 21(2), 113–128 (2018)

    Google Scholar 

  • Montaño, A., Suárez, R.: Dexterous manipulation of unknown objects using virtual contact points. Robotics 8(4), 86–106 (2019)

    Google Scholar 

  • Murali, A., Li, Y., Gandhi, D., Gupta, A.: Learning to grasp without seeing (2018). arXiv:1805.04201

  • Pacchierotti, C., Sinclair, S., Solazzi, M., Frisoli, A., Hayward, V., Prattichizzo, D.: Wearable haptic systems for the fingertip and the hand: taxonomy, review, and perspectives. IEEE Trans. Haptics 10(4), 580–600 (2017)

    Google Scholar 

  • Papazov, C., Haddadin, S., Parusel, S., Krieger, K., Burschka, D.: Rigid 3d geometry matching for grasping of known objects in cluttered scenes. Int. J. Robot. Res. 31(4), 538–553 (2012)

    Google Scholar 

  • Parasuraman, S., Zhen, C.C.S.: Development of robot assisted hand stroke rehabilitation system. In: IIEEE International Conference on Computer and Automation Engineering, pp. 70–74 (2009)

  • Petković, D., Shamshirband, S., Anuar, N.B., Sabri, A.Q.M., Rahman, Z.B.A., Pavlović, N.D.: Retracted article: input displacement neuro-fuzzy control and object recognition by compliant multi-fingered passively adaptive robotic gripper. J. Intell. Robot. Syst. 82(2), 177–187 (2016)

    Google Scholar 

  • Rijpkema, H., Girard, M.: Computer animation of knowledge-based human grasping. ACM Siggraph Comput. Graph. 25(4), 339–348 (1991)

    Google Scholar 

  • Säfström, D., Edin, B.B.: Prediction of object contact during grasping. Exp. Brain Res. 190(3), 265–277 (2008)

    Google Scholar 

  • Saha, S.K.: Introduction to Robotics. Tata McGraw-Hill Education, New York (2014)

    Google Scholar 

  • Sartori, L., Straulino, E., Castiello, U.: How objects are grasped: the interplay between affordances and end-goals. PLoS One 6(9), e25203 (2011)

    Google Scholar 

  • Schmidt, P., Vahrenkamp, N., Wächter, M., Asfour, T.: Grasping of unknown objects using deep convolutional neural networks based on depth images. In: IEEE International Conference on Robotics and Automation, pp. 6831–6838 (2018)

  • Spiers, A.J., Liarokapis, M.V., Calli, B., Dollar, A.M.: Single-grasp object classification and feature extraction with simple robot hands and tactile sensors. IEEE Trans. Haptics 9(2), 207–220 (2016)

    Google Scholar 

  • Stival, F., Michieletto, S., Cognolato, M., Pagello, E., Müller, H., Atzori, M.: A quantitative taxonomy of human hand grasps. J. Neuroeng. Rehabil. 16(1), 28–45 (2019)

    Google Scholar 

  • Temoche, P., Ramírez, E., Rodríguez, O.: A low-cost data glove for virtual reality. In: Proceedings of XI International Congress of Numerical Methods in Enginnering and Applied Sciences, vol. 2012, pp. 1–6 (2012)

  • Tian, L., Thalmann, N.M., Thalmann, D., Fang, Z., Zheng, J.: Object grasping of humanoid robot based on yolo. In: Computer Graphics International Conference, pp. 476–482. Springer (2019)

  • Vàsquez, A., Perdereau, V.: Proprioceptive shape signatures for object manipulation and recognition purposes in a robotic hand. Robot. Auton. Syst. 98, 135–146 (2017)

    Google Scholar 

  • Vásquez, A., Dapogny, A., Bailly, K., Perdereau, V.: Sequential recognition of in-hand object shape using a collection of neural forests. In: IEEE International Conference on Intelligent Robots and Systems, pp. 3081–3086 (2017)

  • Watanabe, T., Yamazaki, K., Yokokohji, Y.: Survey of robotic manipulation studies intending practical applications in real environments-object recognition, soft robot hand, and challenge program and benchmarking. Adv. Robot. 31(19–20), 1114–1132 (2017)

    Google Scholar 

  • Yoshikawa, T., Koeda, M., Fujimoto, H.: Shape recognition and grasping by robotic hands with soft fingers and omnidirectional camera. In: International Conference on Robotics and Automation, pp. 299–304. IEEE (2008)

  • Yu, J., Weng, K., Liang, G., Xie, G.: A vision-based robotic grasping system using deep learning for 3d object recognition and pose estimation. In: IEEE International Conference on Robotics and Biomimetics, pp. 1175–1180 (2013)

Download references

Acknowledgements

The support received under the ASEAN-India R &D Scheme, SERB-DST for the project No. CRD/2018/000049 and I-Hub Foundation for COBOTICS, Technology Innovation Hub of IIT Delhi by DST, project No. GP/2021/RR/018, Government of India, are gratefully acknowledged.

Author information

Authors and Affiliations

  1. Department of CSE, DUIET, Dibrugarh University, Dibrugarh, India

    Abhijit Boruah

  2. Embedded Systems and Robotics Laboratory, School of Engineering, Tezpur University, Tezpur, India

    Nayan M. Kakoty

  3. Department of Mathematics, Dibrugarh University, Dibrugarh, India

    Tazid Ali

  4. School of Biomedical and Health Science Engineering, Universiti Teknologi Malaysia, Johor Bahru, Malaysia

    M. B. Malarvili

Authors
  1. Abhijit Boruah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nayan M. Kakoty
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tazid Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M. B. Malarvili
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abhijit Boruah.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Table 11.

Table 11 Descriptions of the abbreviated notations used in Tables 3, 4, 5, 6 and 7 for representation of the DH parameters of individual fingers
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Boruah, A., Kakoty, N.M., Ali, T. et al. Shape oriented object recognition on grasp using features from enclosure based exploratory procedure. Int J Intell Robot Appl 7, 48–64 (2023). https://doi.org/10.1007/s41315-022-00244-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41315-022-00244-0

Keywords

Search

Navigation