Skip to main content
SpringerLink
Log in

Grasp2Hardness: fuzzy hardness inference of cylindrical objects for grasp force adjustment of force sensor-less robots

  • Original Research Paper
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

Service robots frequently operate various cylindrical objects with unknown physical properties, which demands the grippers of robots being equipped with force sensors to control grasp force. But force sensors are unnecessary and expensive for imprecise grasp force control for most operations in domestic environment. So as a substitute, this paper introduced the fuzzy hardness (FH) for imprecise grasp force evaluation. In addition, a method to infer the FH of objects was proposed, through vision and supervised learning. In this method, the deformation of objects related to the close degree of gripper was treated as a key variable and measured via visual methods. Based on the measured deformation data, long short-term memory network (LSTM) was introduced to conduct supervised learning synchronously. Then, several predicted deformation curves can be obtained through these LSTM blocks. Subsequently, the FH of objects would be clear when the errors between measured data and the predicted ones were calculated from the curves. The verification experiments showed that the maximum inference accuracy can reach 100% on TPU(80A) with 2 mm wall thickness. Moreover, after FH being applied, the deformation of TPU(80A) objects with 2 mm wall thickness decreased approximately 84.4% compared with using classical method. And all these results indicate that the FH inference method can be applied to adjust the grasp force for force sensor-less robots.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Marton ZC, Balint-Benczedi F et al (2014) Part-based geometric categorization and object reconstruction in cluttered table-top scenes. J Intell Robot Syst 76(1):35–56

    Article  Google Scholar 

  2. Marton ZC, Pangercic D, Blodow N et al (2011) Combined 2D–3D categorization and classification for multimodal perception systems. Int J Robot Res 30(11):1378–1402

    Article  Google Scholar 

  3. Calli B, Walsman A, Singh A et al (2015) Benchmarking in manipulation research: using the Yale-CMU-Berkeley object and model set. IEEE Robot Autom Mag 22(3):36–52

    Article  Google Scholar 

  4. Choi YS, Deyle T, Chen T et al (2009) A list of household objects for robotic retrieval prioritized by people with ALS. In: IEEE international conference on rehabilitation robotics, Kyoto, Japan, pp 510–517

  5. Kapusta A, Kemp CC et al (2019) Task-centric optimization of configurations for assistive robots. Auton Robot 43:2033–2054

    Article  Google Scholar 

  6. Ceccarelli M, Cafolla D, Carbone G et al (2017) HeritageBot service robot assisting in cultural heritage, general, and low-cost. In: IEEE international conference on robotic computing, Taichung, Taiwan (China), pp 440–445

  7. Zhu H, Gupta A, Rajeswaran A et al (2019) Robot collisions: dexterous manipulation with deep reinforcement learning: efficient, general, and low-cost. IEEE ICRA, Montreal, Canada, pp 3651–3657

  8. Ingrand F, Ghallab M (2017) Deliberation for autonomous robots: a survey. Artif Intell 247:10–44

    Article  MathSciNet  Google Scholar 

  9. Cafolla D, Wang M, Carbone G et al (2016) LARMbot: a new humanoid robot with parallel mechanisms. Springer, Cham, pp 275–283

    Google Scholar 

  10. Mahler J, Liang J, Niyaz S et al (2017) Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. arXiv preprint arXiv:1703.09312

  11. Zheng Y (2018) Real-time contact force distribution using a polytope hierarchy in the grasp wrench set. Robot Auton Syst 99:97–109

    Article  Google Scholar 

  12. Min JK, Ahn KH, Park HC et al (2019) A novel reactive-type joint torque sensor with high torsional stiffness for robot applications. Mechatronics 63:102265

    Article  Google Scholar 

  13. Eguíluz AG, Rañó I, Coleman SA et al (2019) Reliable robotic handovers through tactile sensing. Auton Robot 43(7):1–15

    Google Scholar 

  14. Luo S, Mou W, Althoefer K et al (2019) Shape recognition by combining proprioception and touch sensing. Auton Robot 43(4):993–1004

    Article  Google Scholar 

  15. Bohg J, Morales A, Asfour T et al (2013) Data-driven grasp synthesisa survey. IEEE Trans Robot 30(2):289–309

    Article  Google Scholar 

  16. Mateo CM, Gil P, Torres F (2016) 3D visual data-driven spatiotemporal deformations for non-rigid object grasping using robot hands. Sensors 16(5):640–665

    Article  Google Scholar 

  17. Seo J, Yim M, Kumar V (2016) A theory on grasping objects using effectors with curved contact surfaces and its application to whole-arm grasping. Int J Robot Res 35(9):1080–1102

    Article  Google Scholar 

  18. Fakhari A, Keshmiri M, Kao I et al (2016) Slippage control in soft finger grasping and manipulation. Adv Robot 30(2):97–108

    Article  Google Scholar 

  19. Shen X, Wang X, Tian M et al (2019) Modeling and sensorless force control of novel tendon-sheath artificial muscle based on hill muscle model. Mechatronics 62:102243

    Article  Google Scholar 

  20. Bender J, Mller M, Otaduy MA et al (2014) A survey on position-based simulation methods in computer graphics. Comput Graph Forum 33(6):228–251

    Article  Google Scholar 

  21. Steinemann D, Otaduy MA, Gross M (2008) Fast adaptive shape matching deformations. Proceedings of the 2008 ACM SIGGRAPH/Euro graphics symposium on computer animation, Dublin, Ireland, pp 87–94

  22. Hwang W, Lim SC (2017) Inferring interaction force from visual information without using physical force sensors. Sensors 17(11):2455–2470

    Article  Google Scholar 

  23. Wu J, Lu E, Kohli P et al (2017) Learning to see physics via visual de-animation. In: Advances in neural information processing systems, pp 152–163

  24. Li SQ, Zhang S, Fu Y et al (2018) The grasping force control for force sensor-less robot through point clouds mask segmentation. ICRAE, Guangzhou, China, pp 1–4

  25. Stachowsky M, Hummel T, Moussa M et al (2016) A slip detection and correction strategy for precision robot grasping. IEEE/ASME Trans Mechatron 21(5):2214–2226

    Article  Google Scholar 

  26. Pyo Y, Nakashima K, Kuwahata S et al (2015) Service robot system with an informationally structured environment. Robot Auton Syst 74:148–165

    Article  Google Scholar 

  27. Pham TH, Kyriazis N, Argyros AA et al (2017) Hand-object contact force estimation from markerless visual tracking. IEEE Trans Pattern Anal Mach Intell 40(12):2883–2896

    Article  Google Scholar 

  28. Pham TH, Kyriazis N, Argyros AA et al (2015) Towards force sensing from vision: observing hand-object interactions to infer manipulation forces. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, USA, pp 2810–2819

  29. Huang L, Yamada H, Ni T et al (2017) A masterCslave control method with gravity compensation for a hydraulic teleoperation construction robot. Adv Mech Eng 9(7):1–11

    Google Scholar 

  30. Wu Z, Song S, Khosla A et al (2015) 3d shapenets: a deep representation for volumetric shapes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, USA, pp 1912–1920

  31. Su H, Maji S, Kalogerakis E et al (2015) Multi-view convolutional neural networks for 3d shape recognition. In: Proceedings of the IEEE international conference on computer vision, Santiago, Chile, pp 945–953

  32. Tamada T, Ikarashi W, Yoneyama D et al (2014) High-speed bipedal robot running using high-speed visual feedback. In: IEEE-RAS international conference on humanoid robots, Madrid, Spain, pp 140–145

  33. Qi CR, Yi L, Su H et al (2017) Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: The conference on neural information processing systems, organized by the neural information processing systems, Long Beach, USA, pp 1–10

  34. Garcia-Garcia A, Gomez-Donoso F, Garcia-Rodriguez J et al (2016) Pointnet: a 3d convolutional neural network for real-time object class recognition. In: International joint conference on neural networks (IJCNN), Vancouver, Canada, pp 7815–1584

  35. Sanchez J Corrales (2018) Robotic manipulation and sensing of deformable objects in domestic and industrial application: a survey. Int J Robot Res 37(7):1–29

    Article  Google Scholar 

  36. Arriola-Rios Veronica EWyatt JL (2017) A multimodal model of object deformation under robotic pushing. IEEE Trans Cognit Dev Syst 9(2:153–169

    Article  Google Scholar 

  37. Kampouris C, Mariolis I, Peleka G et al (2016) Multi-sensorial and explorative recognition of garments and their material properties in unconstrained environment. In: ICRA, Stockholm, Sweden, pp 1656–1663

  38. Cretu AM et al (2012) Soft object deformation monitoring and learning for model-based robotic hand manipulation. IEEE Trans Syst Man Cybern B 42(3):740–753

    Article  Google Scholar 

  39. Hu Z, Sun P, Pan J (2018) Three-dimensional deformable object manipulation using fast online Gaussian process regression. IEEE Robot Autom Lett 3(2):979–986

    Article  Google Scholar 

  40. Yang B, Wang H, Chen W, et al (2016) Vision-based cutting control of deformable objects. In: IEEE international conference on real-time computing and robotics, Angkor Wat, Cambodia, pp 650–655

  41. He K, Gkioxari G, Dollár P et al (2017) Mask R-CNN. In: IEEE international conference on computer vision, Venice, Italy, pp 2980–2988

  42. Brie D, Bombardier V, Baeteman G et al (2016) Local surface sampling step estimation for extracting boundaries of planar point clouds. ISPRS J Photogramm Remote Sens 119:309–319

    Article  Google Scholar 

  43. Demarsin K, Vanderstraeten D, Volodine T et al (2007) Detection of closed sharp edges in point clouds using normal estimation and graph theory. Comput Aided Des 39(4):276–283

    Article  Google Scholar 

  44. Rusu RB, Cousins S (2011) 3d is here: point cloud library (pcl). In: IEEE international conference on robotics and automation, Miami, USA, pp 1–4

  45. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

Download references

Acknowledgements

We acknowledge the support received from the HUST & UBTECH Intelligent Service Robots Joint Lab and the National Nature Science Foundation of China (Grant No. 71771098).

Author information

Authors and Affiliations

  1. School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, 430074, China

    Shiqi Li, Shuai Zhang & Yan Fu

  2. HUST & UBTECH Intelligent Service Robots Joint Lab, Huazhong University of Science and Technology, Wuhan, 430074, China

    Shuai Zhang

  3. UBTECH Robotics CORP LTD, Shenzhen, 518000, China

    Youjun Xiong & Zheng Xie

Authors
  1. Shiqi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Youjun Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zheng Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuai Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, S., Zhang, S., Fu, Y. et al. Grasp2Hardness: fuzzy hardness inference of cylindrical objects for grasp force adjustment of force sensor-less robots. Intel Serv Robotics 14, 129–141 (2021). https://doi.org/10.1007/s11370-021-00362-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-021-00362-x

Keywords

Search

Navigation