Skip to main content
Log in

HERB: a home exploring robotic butler

Autonomous Robots Aims and scope Submit manuscript

Abstract

We describe the architecture, algorithms, and experiments with HERB, an autonomous mobile manipulator that performs useful manipulation tasks in the home. We present new algorithms for searching for objects, learning to navigate in cluttered dynamic indoor scenes, recognizing and registering objects accurately in high clutter using vision, manipulating doors and other constrained objects using caging grasps, grasp planning and execution in clutter, and manipulation on pose and torque constraint manifolds. We also present numerous severe real-world test results from the integration of these algorithms into a single mobile manipulator.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  • Beis, J. S., & Lowe, D. G. (1997). Shape indexing using approximate nearest-neighbour search in high-dimensional spaces. In Proc. IEEE conf. comp. vision patt. recog. (pp. 1000–1006).

  • Berenson, D., & Srinivasa, S. (2008). Grasp synthesis in cluttered environments for dexterous hands. In IEEE-RAS international conference on humanoid robots.

  • Berenson, D., Diankov, R., Nishiwaki, K., Kagami, S., & Kuffner, J. (2007). Grasp planning in complex scenes. In IEEE-RAS international conference on humanoid robots (Humanoids07).

  • Berenson, D., Srinivasa, S., Ferguson, D., Collet, A., & Kuffner, J. (2009a). Manipulation planning with workspace goal regions. In IEEE international conference on robotics and automation.

  • Berenson, D., Srinivasa, S., Ferguson, D., & Kuffner, J. (2009b). Manipulation planning on constraint manifolds. In IEEE international conference on robotics and automation.

  • Bradski, G., & Kaehler, A. (2008). Learning OpenCV: computer vision with the OpenCV library. New York: O’Reilly.

    Google Scholar 

  • Canine Companions (2009). Canine companions for independence. http://www.cci.org.

  • Collet, A., Berenson, D., Srinivasa, S. S., & Ferguson, D. (2009). Object recognition and full pose registration from a single image for robotic manipulation. In IEEE international conference on robotics and automation.

  • Diankov, R., & Kuffner, J. (2007). Randomized statistical path planning. In IEEE/RSJ international conference on intelligent robots and systems.

  • Diankov, R., & Kuffner, J. (2008). Openrave: A planning architecture for autonomous robotics. Tech. Rep. CMU-RI-TR-08-34, Robotics Institute, Carnegie Mellon University.

  • Diankov, R., Srinivasa, S., Ferguson, D., & Kuffner, J. (2008). Manipulation planning with caging grasps. In IEEE-RAS international conference on humanoid robots.

  • Eaton, J. W. (2002). GNU Octave Manual. Network Theory Limited.

  • Ekvall, S., Kragic, D., & Hoffmann, F. (2005). Object recognition and pose estimation using color cooccurrence histograms and geometric modeling. Image Vision Comput, 23(11), 943–955.

    Article  Google Scholar 

  • Exact Dynamics (2009). Arm: Assistive robotic manipulator. http://www.exactdynamics.nl.

  • Gallagher, G., Srinivasa, S., Bagnell, D., & Ferguson, D. (2009). An online approach to object detection, modeling and mapping for mobile robots. In Proc. int’l conf. robotics and automation.

  • Gordon, I., & Lowe, D. G. (2006). What and where: 3d object recognition with accurate pose. In Toward category-level object recognition (pp. 67–82).

  • Hahnel, D., Triebel, R., Burgard, W., & Thrun, S. (2003). Map building with mobile robots in dynamic environments. In Proc. int’l conf. robotics and automation.

  • Hollinger, G., Ferguson, D., Srinivasa, S., & Singh, S. (2009). Combining search and action for mobile robots. In Proc. int’l conf. robotics and automation.

  • IRobot (2009). Roomba. http://www.irobot.com.

  • Jain, A., & Kemp, C. C. (2008). Behaviors for robust door opening and doorway traversal with a force-sensing mobile manipulator. In Proceedings of the manipulation workshop in robotics science and systems.

  • Katz, D., & Brock, O. (2008). Manipulating articulated objects with interactive perception. In IEEE international conference on robotics and automation.

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60, 91–110.

    Article  Google Scholar 

  • Loy, G., & Zelinsky, A. (2003). Fast radial symmetry for detecting points of interest. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(8), 959–973.

    Article  Google Scholar 

  • Mendes, A., Bento, X., & Nunes, L. (2004). Multi-target detection and tracking with a laser scanner. In Intelligent vehicles symposium.

  • Mittrapiyanuruk, P., DeSouza, G. N., & Kak, A. C. (2004). Calculating the 3d-pose of rigid-objects using active appearance models. In IEEE international conference on robotics and automation (pp. 5147–5152). New York: IEEE.

    Google Scholar 

  • Monkey Helpers (2009). Helping hands: Monkey helpers for the disabled. http://www.monkeyhelpers.org.

  • Montemerlo, M., & Thrun, S. (2002). Conditional particle filters for simultaneous mobile robot localization and people-tracking (slap). In Proc. int’l conf. robotics and automation.

  • Muller, A., Kirsch, A., & Beetz, M. (2007). Transformational planning for everyday activity. In 17th international conference on automated planning and scheduling (pp. 248–255).

  • Nguyen, H., Anderson, C., Trevor, A., Jain, A., Xu, Z., & Kemp, C. (2008). El-e: an assistive robot that fetches objects from flat surfaces. In Proc. human robot interaction, the robotics helpers workshop.

  • Pereira, G. A. S., Kumar, V., & Campos, M. F. M. (2002). Decentralized algorithms for multirobot manipulation via caging. In Proceedings of the workshop on the algorithmic foundations of robotics.

  • Prats, M., Sanz, P. J., & del Pobil, A. P. (2008). A sensor-based approach for physical interaction based on hand, grasp and task frames. In Proceedings of the manipulation workshop in robotics science and systems.

  • Quigley, M., Gerkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., & Ng, A. (2009). (ros): an open-source robot operating system. In ICRA workshop on open source software in robotics.

  • Ray, C., Mondada, F., & Siegwart, R. (2001). What do people expect from robots? In Intelligent robots and systems, 2008 proceedings 2008 IEEE/RSJ international conference (pp. 3816–3821).

  • Saxena, A., Driemeyer, J., & Ng, A. (2008). Robotic grasping of novel objects using vision. The International Journal of Robotics Research, 27(2), 157–173.

    Article  Google Scholar 

  • Schulz, D., & Burgard, W. (2001). People tracking with a mobile robot using sample-based joint probabilistic data association filters. Robotics and Autonomous Systems, 34(2–3), 107–115.

    Article  MATH  Google Scholar 

  • Schulz, D., Burgard, W., Fox, D., & Cremers, A. (2003). People tracking with a mobile robot using sample-based joint probabilistic data association filters. The International Journal of Robotics Research, 22(2), 99–116.

    Article  Google Scholar 

  • Taylor, G., & Kleeman, L. (2003). Fusion of multimodal visual cues for modelbased object tracking. In Australasian conference on robotics and automation (ACRA2003), Brisbane, Australia.

  • The personal robotics project (2008). http://pr.willowgarage.com.

  • Vacchetti, L., Lepetit, V., & Fua, P. (2004). Stable real-time 3d tracking using online and offline information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(10), 1385–1391. doi:10.1109/TPAMI.2004.92.

    Article  Google Scholar 

  • Walter, J., & Arnrich, B. (2000). Gabor filters for object localization and robot grasping. In IEEE proceedings of the international conference on pattern recognition (p. 4124). Washington: IEEE Computer Society.

    Google Scholar 

  • Wang, C. C., & Thorpe, C. (2002). Simultaneous localization and mapping with detection and tracking of moving objects. In Proc. int’l conf. robotics and automation.

  • Zerofrog (2008). Libsiftfast. http://sourceforge.net/projects/libsift.

  • Zhang, J., Schmidt, R., & Knoll, A. (1999). Appearance-based visual learning in a neuro-fuzzy model for fine-positioning of manipulators. In IEEE international conference on robotics and automation (p. 1164).

  • Zickler, S., & Veloso, M. (2006). Detection and localization of multiple objects. In Humanoid robots, 2006 6th IEEE-RAS international conference (pp. 20–25). doi:10.1109/ICHR.2006.321358.

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Siddhartha S. Srinivasa.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Srinivasa, S.S., Ferguson, D., Helfrich, C.J. et al. HERB: a home exploring robotic butler. Auton Robot 28, 5–20 (2010). https://doi.org/10.1007/s10514-009-9160-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-009-9160-9

Navigation