Abstract
We consider a novel problem where multiple rigid convex polygonal objects rest in randomly placed positions and orientations on a planar surface visible from an overhead camera. The objective is to efficiently grasp and transport all objects into a bin using multi-object push-grasps, where multiple objects are pushed together to facilitate multi-object grasping. We provide necessary conditions for frictionless multi-object push-grasps and apply these to filter inadmissible grasps in a novel multi-object grasp planner. We find that our planner is 19 times faster than a Mujoco simulator baseline. We also propose a picking algorithm that uses both single- and multi-object grasps to pick objects. In physical grasping experiments comparing performance with a single-object picking baseline, we find that the frictionless multi-object grasping system achieves 13.6% higher grasp success and is 59.9% faster, from 212 PPH to 340 PPH. See https://sites.google.com/view/multi-object-grasping for videos and code.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Danielczuk, M., Mahler, J., Correa, C., Goldberg, K.: Linear push policies to increase grasp access for robot bin picking. In: CASE (2018). https://ieeexplore.ieee.org/document/8560406
Mahler, J., Goldberg, K.: Learning deep policies for robot bin picking by simulating robust grasping sequences. In: CoRL, vol. 78, pp. 515–524 (2017)
Matsumura, R., Domae, Y., Wan, W., Harada, K.: Learning based robotic bin-picking for potentially tangled objects. In: IROS (2019)
Huang, H., et al.: Mechanical search on shelves using LAX-RAY: lateral access X-RAY. In: IROS (2021). https://arxiv.org/abs/2011.11696
Ichnowski, J., Danielczuk, M., Xu, J., Satish, V., Goldberg, K.: GOMP: grasp-optimized motion planning for bin picking. In: ICRA (2020)
Ichnowski, J., Avigal, Y., Satish, V., Goldberg, K.: Deep learning can accelerate grasp-optimized motion planning. Sci. Robot. 5(48), eabd7710 (2020)
Yamada, T., Yamanaka, S., Yamada, M., Funahashi, Y., Yamamoto, H.: Grasp stability analysis of multiple planar objects. In: ROBIO (2009)
Yamada, T., Yamada, M., Yamamoto, H.: Stability analysis of multiple objects grasped by multifingered hands with revolute joints in 2D. In: IEEE International Conference on Mechatronics and Automation (2012)
Harada, K., Kaneko, M.: Kinematics and internal force in grasping multiple objects. In: IROS (1998)
Sakamoto, T., Wan, W., Nishi, T., Harada, K.: Efficient picking by considering simultaneous two-object grasping. In: IROS (2021)
Agboh, W.C., Ruprecht, D., Dogar, M.R.: Combining coarse and fine physics for manipulation using parallel-in-time integration. In: Asfour, T., Yoshida, E., Park, J., Christensen, H., Khatib, O. (eds.) ISRR 2019. Springer Proceedings in Advanced Robotics, vol. 20, pp. 725–740. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-95459-8_44
Agboh, W.C., Ruprecht, D., Dogar, M.R.: Parareal with a learned coarse model for robotic manipulation. Comput. Visual Sci. 23(8), 1–10 (2020)
Hasan, M., et al.: Human-like planning for reaching in cluttered environments. In: ICRA (2020)
Morrison, D., Corke, P., Leitner, J.: Learning robust, real-time, reactive robotic grasping. IJRR 39(2–3), 183–201 (2020)
Lou, X., Yang, Y., Choi, C.: Collision-aware target-driven object grasping in constrained environments. In: ICRA (2021)
Prattichizzo, D., Trinkle, J.C.: Grasping. In: Siciliano, B., Khatib, O. (eds.) Springer Handbook of Robotics, pp. 671–700. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-30301-5_29
Rodriguez, A., Mason, M.T., Ferry, S.: From caging to grasping. IJRR 31(7), 886–900 (2012)
Kehoe, B., Matsukawa, A., Candido, S., Kuffner, J., Goldberg, K.: Cloud-based robot grasping with the google object recognition engine. In: ICRA (2013)
Bohg, J., Morales, A., Asfour, T., Kragic, D.: Data-driven grasp synthesis-a survey. IEEE Trans. Rob. 30(2), 289–309 (2014)
Goldfeder, C., Allen, P.K.: Data-driven grasping. Auton. Robot. 31, 1–20 (2011)
Mahler, J., et al.: Dex-net 1.0: a cloud-based network of 3D objects for robust grasp planning using a multi-armed bandit model with correlated rewards. In: ICRA (2016)
Pauly, L., Agboh, W.C., Hogg, D.C., Fuentes, R.: O2a: one-shot observational learning with action vectors. Front. Robot. AI 8, 686368 (2021)
Bejjani, W., Agboh, W.C., Dogar, M.R., Leonetti, M.: Occlusion-aware search for object retrieval in clutter. In: IEEE IROS (2021)
Harada, K., Kaneko, M.: Enveloping grasp for multiple objects. In: ICRA (1998)
Harada, K., Kaneko, M., Tsujii, T.: Rolling-based manipulation for multiple objects. IEEE Trans. Robot. Autom. 16(5), 457–468 (2000)
Harada, K., Kaneko, M., Tsuji, T.: Active force closure for multiple objects. In: Advances in Robot Kinematics, pp. 155–164 (2000)
Yoshikawa, T., Watanabe, T., Daito, M.: Optimization of power grasps for multiple objects. In: ICRA (2001)
Yamada, T., Ooba, T., Yamamoto, T., Mimura, N., Funahashi, Y.: Grasp stability analysis of two objects in two dimensions. In: ICRA (2005)
Yamada, T., Mimura, N., Funahashi, Y.: Grasp stability analysis of two objects with both friction and frictionless contacts in two dimensions. In: IEEE International Symposium on Micro-NanoMechatronics and Human Science (2005)
Yamada, T., Yamamoto, H.: Static grasp stability analysis of multiple spatial objects. J. Control Sci. Eng. 3, 118–139 (2015)
Chen, T., Shenoy, A., Kolinko, A., Shah, S., Sun, Y.: Multi-object grasping - estimating the number of objects in a robotic grasp. In: IROS (2021)
Shenoy, A., Chen, T., Sun, Y.: Multi-object grasping - generating efficient robotic picking and transferring policy. In: CoRR (2021)
Dogar, M., Srinivasa, S.S.: A framework for push-grasping in clutter. In: RSS (2011)
Agboh, W.C., Dogar, M.R.: Real-time online re-planning for grasping under clutter and uncertainty. In: IEEE Humanoids, pp. 1–8 (2018)
Agboh, W.C., Dogar, M.R.: Pushing fast and slow: task-adaptive planning for non-prehensile manipulation under uncertainty. In: WAFR (2018)
Agboh, W.C., Dogar, M.R.: Robust physics-based manipulation by interleaving open and closed-loop execution. CoRR abs/2105.08325 (2021)
Goldberg, K.Y.: Orienting polygonal parts without sensors. Algorithmica 10, 201–225 (1993)
Acknowledgement
This research was performed at the AUTOLAB at UC Berkeley in affiliation with the Berkeley AI Research (BAIR) Lab, and the CITRIS “People and Robots” (CPAR) Initiative. The authors were supported in part by donations from Siemens. Mehmet Dogar was partially supported by an EPSRC Fellowship (EP/V052659) and Wisdom C. Agboh was supported by EPSRC Doctoral Prize Fellowship Award EP/T517860/1. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Sponsors.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary material 1 (mp4 7811 KB)
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Agboh, W.C., Ichnowski, J., Goldberg, K., Dogar, M.R. (2023). Multi-object Grasping in the Plane. In: Billard, A., Asfour, T., Khatib, O. (eds) Robotics Research. ISRR 2022. Springer Proceedings in Advanced Robotics, vol 27. Springer, Cham. https://doi.org/10.1007/978-3-031-25555-7_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-25555-7_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25554-0
Online ISBN: 978-3-031-25555-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)