Skip to main content
SpringerLink
Log in

Combined task and motion planning system for the service robot using hierarchical action decomposition

  • Original Research Paper
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

A service robot needs integration of symbolic reasoning for high-level task planning and geometric computation for low-level motion planning to provide services in an everyday human-living environment. For this purpose, one may develop individual modules for object recognition, knowledge inference, task planning, and motion planning, which requires a system that integrates them to provide services autonomously. In this paper, we propose a combined task-motion planning system implemented using existing open-source libraries. We implemented an action library module that specifies the relationship between the compound actions that are modeled in an AI planning language to enable abstract reasoning and the primitive actions that can be geometrically verified its feasibility. This serves as an interface between the two levels by providing a rule in which a compound action sequence obtained from task planning at the symbolic level is decomposed into a primitive action sequence capable of motion planning at the geometric level. In addition, we defined the relationship between the two types of actions in a hierarchical structure and added conditional clauses according to the task states, so that primitive actions that need to be additionally performed are automatically added to the action sequence during decomposition. This procedure enables the robot to successfully perform the task in response to an unintended change in the environment. We establish a task domain where a robot delivers an object to a human user in unexpected situations and verify the proposed method under dynamic simulation environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Aeronautiques C, Howe A, Knoblock C, McDermott ID, Ram A, Veloso M, Weld D, Sri DW, Barrett A, Christianson D et al (1998) Pddl| the planning domain definition language. Tech. rep., Technical Report

  2. Akbari A, Diab M, Rosell J (2020) Contingent task and motion planning under uncertainty for human-robot interactions. Appl Sci 10(5):1665

    Article  Google Scholar 

  3. Akbari A, Gillani M, Rosell J (2016) Reasoning-based evaluation of manipulation actions for efficient task planning. In: Robot 2015: Second Iberian Robotics Conference, Springer, pp 69–80

  4. Akbari A, Lagriffoul F, Rosell J (2019) Combined heuristic task and motion planning for bi-manual robots. Auton Robots 43(6):1575–1590

    Article  Google Scholar 

  5. Alford RW, Kuter U, Nau D (2009) Translating htns to pddl: a small amount of domain knowledge can go a long way. In: twenty-first international joint conference on artificial intelligence

  6. Awaad I, Kraetzschmar GK, Hertzberg J (2015) The role of functional affordances in socializing robots. Int J Soc Robot 7(4):421–438

    Article  Google Scholar 

  7. Beetz M, Beßler D, Haidu A, Pomarlan M, Bozcuoğlu A, Bartel G (2018) Know rob 2.0-a 2nd generation knowledge processing framework for cognition-enabled robotic agents. In: 2018 IEEE international conference on robotics and automation (ICRA), IEEE, 512–519

  8. Bidot J, Karlsson L, Lagriffoul F, Saffiotti A (2017) Geometric backtracking for combined task and motion planning in robotic systems. Artif Intell 247:229–265

    Article  MathSciNet  Google Scholar 

  9. Bonet B, Geffner H (2011) Planning under partial observability by classical replanning: theory and experiments

  10. Cambon S, Alami R, Gravot F (2009) A hybrid approach to intricate motion, manipulation and task planning. Int J Robot Res 28(1):104–126

    Article  Google Scholar 

  11. Cashmore M, Fox M, Long D, Magazzeni D, Ridder B, Carrera A, Palomeras N, Hurtos N, Carreras M (2015) Rosplan: planning in the robot operating system. Proc Int Conf Autom Plan Sched 25:333–341

    Google Scholar 

  12. Castillo LA, Fernández-Olivares J, García-Pérez Ó, Palao F (2006) Efficiently handling temporal knowledge in an htn planner. In: ICAPS, pp 63–72

  13. Chitta S, Sucan I, Cousins S (2012) Moveit![ros topics]. IEEE Robot Autom Mag 19(1):18–19

    Article  Google Scholar 

  14. Currie K, Tate A (1991) O-plan: the open planning architecture. Artif Intell 52(1):49–86

    Article  Google Scholar 

  15. Deng S, Xu X, Wu C, Chen K, Jia K (2021) 3d affordancenet: a benchmark for visual object affordance understanding. In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1778–1787

  16. Do TT, Nguyen A, Reid I (2018) Affordancenet: an end-to-end deep learning approach for object affordance detection. In: 2018 IEEE international conference on robotics and automation (ICRA), IEEE, pp 5882–5889

  17. Fan Y, Tomizuka M (2019) Efficient grasp planning and execution with multifingered hands by surface fitting. IEEE Robot Autom Lett 4(4):3995–4002

    Article  Google Scholar 

  18. Fikes RE, Nilsson NJ (1971) Strips: a new approach to the application of theorem proving to problem solving. Artif Intell 2(3–4):189–208

    Article  Google Scholar 

  19. Fox M, Long D (2003) Pddl2. 1: an extension to pddl for expressing temporal planning domains. J Artif Intell Res 20:61–124

    Article  Google Scholar 

  20. Jung ga Young Ik (2021) Dynamic 3d scene graph generation for robotic manipulation tasks. J Inst Control Robot Syst 16:313–318

    Google Scholar 

  21. Garrett CR, Chitnis R, Holladay R, Kim B, Silver T, Kaelbling LP, Lozano-Pérez T (2021) Integrated task and motion planning. Ann Rev Control Robot Autonom Syst 4:265–293

    Article  Google Scholar 

  22. Garrett CR, Lozano-Pérez T, Kaelbling LP (2015) Ffrob: an efficient heuristic for task and motion planning. In: Algorithmic Foundations of Robotics XI, Springer, pp 179–195

  23. Garrett CR, Lozano-Pérez T, Kaelbling LP (2018) Stripstream: integrating symbolic planners and blackbox samplers. arXiv preprint arXiv:1802.08705

  24. Ghallab M, Nau D, Traverso P (2004) Automated planning: theory and practice. Elsevier, Amsterdam

    MATH  Google Scholar 

  25. Helmert M (2006) The fast downward planning system. J Artif Intell Res 26:191–246

    Article  Google Scholar 

  26. Hoffmann J, Brafman R (2005) Contingent planning via heuristic forward search with implicit belief states. In: Proc. ICAPS, 2005

  27. Hoffmann J, Nebel B (2001) The ff planning system: fast plan generation through heuristic search. J Artif Intell Res 14:253–302

    Article  Google Scholar 

  28. Höller D, Behnke G, Bercher P, Biundo S, Fiorino H, Pellier D, Alford R (2020) Hddl: an extension to pddl for expressing hierarchical planning problems. Proc AAAI Conf Artif Intell 34:9883–9891

    Google Scholar 

  29. Hong-ryul Jung JJYM (2021) Efficient base repositioning for mobile manipulation based on inverse reachability. J Korea Robot Soc 16:313–318

    Article  Google Scholar 

  30. Kaelbling LP, Littman ML, Cassandra AR (1998) Planning and acting in partially observable stochastic domains. Artif Intell 101(1–2):99–134

    Article  MathSciNet  Google Scholar 

  31. Lagriffoul F, Dimitrov D, Saffiotti A, Karlsson L (2012) Constraint propagation on interval bounds for dealing with geometric backtracking. In: IEEE/RSJ international conference on intelligent robots and systems, IEEE, 957–964

  32. Latombe JC (2012) Robot motion planning, Springer Science & Business Media 124

  33. LaValle SM, Kuffner JJ, Donald B et al (2001) Rapidly-exploring random trees: progress and prospects. Algorithm Comput Robot New Dir 5:293–308

    MATH  Google Scholar 

  34. Lee S, Kim I (2019) Task and motion planning for grasping obstructed object in cluttered environment. J Korea Robot Soc 14(2):104–113

    Article  Google Scholar 

  35. Lemaignan S, Warnier M, Sisbot EA, Clodic A, Alami R (2017) Artificial cognition for social human-robot interaction: an implementation. Artif Intell 247:45–69

    Article  MathSciNet  Google Scholar 

  36. Miller AT, Allen PK (2004) Graspit! a versatile simulator for robotic grasping. IEEE Robot Autom Mag 11(4):110–122

    Article  Google Scholar 

  37. Nau DS, Au TC, Ilghami O, Kuter U, Murdock JW, Wu D, Yaman F (2003) Shop2: an htn planning system. J Artif Intell Res 20:379–404

    Article  Google Scholar 

  38. Pandey AK, Saut JP, Sidobre D, Alami R (2012) Towards planning human-robot interactive manipulation tasks: Task dependent and human oriented autonomous selection of grasp and placement. In: IEEE RAS & EMBS international conference on biomedical robotics and biomechatronics (BioRob), IEEE, pp 1371–1376

  39. Pednault EP (1989)Adl: exploring the middle ground between strips and the situation calculus. In: Proceedings of the first international conference on Principles of knowledge representation and reasoning. Morgan Kaufmann Publishers Inc, pp 324–332

  40. Pellier D, Fiorino H (2018) Pddl4j: a planning domain description library for java. J Exp Theor Artif Intell 30(1):143–176

    Article  Google Scholar 

  41. Plaku E, Hager GD (2010) Sampling-based motion and symbolic action planning with geometric and differential constraints. In: IEEE international conference on robotics and automation, IEEE, 5002–5008

  42. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: Unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788

  43. Rösmann C, Hoffmann F, Bertram T (2017) Integrated online trajectory planning and optimization in distinctive topologies. Robot Autonom Syst 88:142–153

    Article  Google Scholar 

  44. Rovida F, Crosby M, Holz D, Polydoros AS, Großmann B, Petrick R, Krüger V (2017)Skiros-a skill-based robot control platform on top of ros. In: Robot operating system (ROS). Springer, pp 121–160

  45. de Silva L, Pandey AK, Gharbi M, Alami R (2013) Towards combining htn planning and geometric task planning. In: RSS Workshop on Combined Robot Motion Planning and AI Planning for Practical Applications

  46. Srivastava S, Fang E, Riano L, Chitnis R, Russell S, Abbeel P (2014) Combined task and motion planning through an extensible planner-independent interface layer. In: IEEE international conference on robotics and automation (ICRA). IEEE, pp 639–646

  47. Su HR, Chen KY (2019) Design and implementation of a mobile robot with autonomous door opening ability. Int J Fuzzy Syst 21(1):333–342

    Article  Google Scholar 

  48. Sucan IA, Moll M, Kavraki LE (2012) The open motion planning library. IEEE Robot Autom Mag 19(4):72–82

    Article  Google Scholar 

  49. Tenorth M, Beetz M (2013) Knowrob: a knowledge processing infrastructure for cognition-enabled robots. Int J Robot Res 32(5):566–590

    Article  Google Scholar 

  50. Tsuchiya Y, Kiyokawa T, Ricardez GAG, Takamatsu J, Ogasawara T (2019) Pouring from deformable containers using dual-arm manipulation and tactile sensing. In: IEEE international conference on robotic computing (IRC). IEEE, pp 357–362

  51. Umbrico A, Cesta A, Cortellessa G, Orlandini A (2020) A holistic approach to behavior adaptation for socially assistive robots. Int J Soc Robot 12(3):617–637

    Article  Google Scholar 

  52. Urakami Y, Hodgkinson A, Carlin C, Leu R, Rigazio L, Abbeel P (2019) Doorgym: a scalable door opening environment and baseline agent. arXiv preprint arXiv:1908.01887

  53. Wan W, Harada K, Kanehiro F (2019) Preparatory manipulation planning using automatically determined single and dual arms. IEEE Transactions on Industrial Informatics

  54. Wells AM, Dantam NT, Shrivastava A, Kavraki LE (2019) Learning feasibility for task and motion planning in tabletop environments. IEEE Robot Autom Lett 4(2):1255–1262

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Technology Innovation Program and Industrial Strategic Technology Development Program (20018256, Development of service robot technologies for cleaning a table).

Author information

Authors and Affiliations

  1. Sungkyunkwan University, Suwon, South Korea

    Jeongmin Jeon, Hong-ryul Jung, Tuan Luong, Francisco Yumbla & Hyungpil Moon

Authors
  1. Jeongmin Jeon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hong-ryul Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tuan Luong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Francisco Yumbla
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hyungpil Moon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hyungpil Moon.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jeon, J., Jung, Hr., Luong, T. et al. Combined task and motion planning system for the service robot using hierarchical action decomposition. Intel Serv Robotics 15, 487–501 (2022). https://doi.org/10.1007/s11370-022-00437-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-022-00437-3

Keywords

Search

Navigation