Abstract
We describe the architecture, algorithms, and experiments with HERB, an autonomous mobile manipulator that performs useful manipulation tasks in the home. We present new algorithms for searching for objects, learning to navigate in cluttered dynamic indoor scenes, recognizing and registering objects accurately in high clutter using vision, manipulating doors and other constrained objects using caging grasps, grasp planning and execution in clutter, and manipulation on pose and torque constraint manifolds. We also present numerous severe real-world test results from the integration of these algorithms into a single mobile manipulator.
References
Beis, J. S., & Lowe, D. G. (1997). Shape indexing using approximate nearest-neighbour search in high-dimensional spaces. In Proc. IEEE conf. comp. vision patt. recog. (pp. 1000–1006).
Berenson, D., & Srinivasa, S. (2008). Grasp synthesis in cluttered environments for dexterous hands. In IEEE-RAS international conference on humanoid robots.
Berenson, D., Diankov, R., Nishiwaki, K., Kagami, S., & Kuffner, J. (2007). Grasp planning in complex scenes. In IEEE-RAS international conference on humanoid robots (Humanoids07).
Berenson, D., Srinivasa, S., Ferguson, D., Collet, A., & Kuffner, J. (2009a). Manipulation planning with workspace goal regions. In IEEE international conference on robotics and automation.
Berenson, D., Srinivasa, S., Ferguson, D., & Kuffner, J. (2009b). Manipulation planning on constraint manifolds. In IEEE international conference on robotics and automation.
Bradski, G., & Kaehler, A. (2008). Learning OpenCV: computer vision with the OpenCV library. New York: O’Reilly.
Canine Companions (2009). Canine companions for independence. http://www.cci.org.
Collet, A., Berenson, D., Srinivasa, S. S., & Ferguson, D. (2009). Object recognition and full pose registration from a single image for robotic manipulation. In IEEE international conference on robotics and automation.
Diankov, R., & Kuffner, J. (2007). Randomized statistical path planning. In IEEE/RSJ international conference on intelligent robots and systems.
Diankov, R., & Kuffner, J. (2008). Openrave: A planning architecture for autonomous robotics. Tech. Rep. CMU-RI-TR-08-34, Robotics Institute, Carnegie Mellon University.
Diankov, R., Srinivasa, S., Ferguson, D., & Kuffner, J. (2008). Manipulation planning with caging grasps. In IEEE-RAS international conference on humanoid robots.
Eaton, J. W. (2002). GNU Octave Manual. Network Theory Limited.
Ekvall, S., Kragic, D., & Hoffmann, F. (2005). Object recognition and pose estimation using color cooccurrence histograms and geometric modeling. Image Vision Comput, 23(11), 943–955.
Exact Dynamics (2009). Arm: Assistive robotic manipulator. http://www.exactdynamics.nl.
Gallagher, G., Srinivasa, S., Bagnell, D., & Ferguson, D. (2009). An online approach to object detection, modeling and mapping for mobile robots. In Proc. int’l conf. robotics and automation.
Gordon, I., & Lowe, D. G. (2006). What and where: 3d object recognition with accurate pose. In Toward category-level object recognition (pp. 67–82).
Hahnel, D., Triebel, R., Burgard, W., & Thrun, S. (2003). Map building with mobile robots in dynamic environments. In Proc. int’l conf. robotics and automation.
Hollinger, G., Ferguson, D., Srinivasa, S., & Singh, S. (2009). Combining search and action for mobile robots. In Proc. int’l conf. robotics and automation.
IRobot (2009). Roomba. http://www.irobot.com.
Jain, A., & Kemp, C. C. (2008). Behaviors for robust door opening and doorway traversal with a force-sensing mobile manipulator. In Proceedings of the manipulation workshop in robotics science and systems.
Katz, D., & Brock, O. (2008). Manipulating articulated objects with interactive perception. In IEEE international conference on robotics and automation.
Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60, 91–110.
Loy, G., & Zelinsky, A. (2003). Fast radial symmetry for detecting points of interest. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(8), 959–973.
Mendes, A., Bento, X., & Nunes, L. (2004). Multi-target detection and tracking with a laser scanner. In Intelligent vehicles symposium.
Mittrapiyanuruk, P., DeSouza, G. N., & Kak, A. C. (2004). Calculating the 3d-pose of rigid-objects using active appearance models. In IEEE international conference on robotics and automation (pp. 5147–5152). New York: IEEE.
Monkey Helpers (2009). Helping hands: Monkey helpers for the disabled. http://www.monkeyhelpers.org.
Montemerlo, M., & Thrun, S. (2002). Conditional particle filters for simultaneous mobile robot localization and people-tracking (slap). In Proc. int’l conf. robotics and automation.
Muller, A., Kirsch, A., & Beetz, M. (2007). Transformational planning for everyday activity. In 17th international conference on automated planning and scheduling (pp. 248–255).
Nguyen, H., Anderson, C., Trevor, A., Jain, A., Xu, Z., & Kemp, C. (2008). El-e: an assistive robot that fetches objects from flat surfaces. In Proc. human robot interaction, the robotics helpers workshop.
Pereira, G. A. S., Kumar, V., & Campos, M. F. M. (2002). Decentralized algorithms for multirobot manipulation via caging. In Proceedings of the workshop on the algorithmic foundations of robotics.
Prats, M., Sanz, P. J., & del Pobil, A. P. (2008). A sensor-based approach for physical interaction based on hand, grasp and task frames. In Proceedings of the manipulation workshop in robotics science and systems.
Quigley, M., Gerkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., & Ng, A. (2009). (ros): an open-source robot operating system. In ICRA workshop on open source software in robotics.
Ray, C., Mondada, F., & Siegwart, R. (2001). What do people expect from robots? In Intelligent robots and systems, 2008 proceedings 2008 IEEE/RSJ international conference (pp. 3816–3821).
Saxena, A., Driemeyer, J., & Ng, A. (2008). Robotic grasping of novel objects using vision. The International Journal of Robotics Research, 27(2), 157–173.
Schulz, D., & Burgard, W. (2001). People tracking with a mobile robot using sample-based joint probabilistic data association filters. Robotics and Autonomous Systems, 34(2–3), 107–115.
Schulz, D., Burgard, W., Fox, D., & Cremers, A. (2003). People tracking with a mobile robot using sample-based joint probabilistic data association filters. The International Journal of Robotics Research, 22(2), 99–116.
Taylor, G., & Kleeman, L. (2003). Fusion of multimodal visual cues for modelbased object tracking. In Australasian conference on robotics and automation (ACRA2003), Brisbane, Australia.
The personal robotics project (2008). http://pr.willowgarage.com.
Vacchetti, L., Lepetit, V., & Fua, P. (2004). Stable real-time 3d tracking using online and offline information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(10), 1385–1391. doi:10.1109/TPAMI.2004.92.
Walter, J., & Arnrich, B. (2000). Gabor filters for object localization and robot grasping. In IEEE proceedings of the international conference on pattern recognition (p. 4124). Washington: IEEE Computer Society.
Wang, C. C., & Thorpe, C. (2002). Simultaneous localization and mapping with detection and tracking of moving objects. In Proc. int’l conf. robotics and automation.
Zerofrog (2008). Libsiftfast. http://sourceforge.net/projects/libsift.
Zhang, J., Schmidt, R., & Knoll, A. (1999). Appearance-based visual learning in a neuro-fuzzy model for fine-positioning of manipulators. In IEEE international conference on robotics and automation (p. 1164).
Zickler, S., & Veloso, M. (2006). Detection and localization of multiple objects. In Humanoid robots, 2006 6th IEEE-RAS international conference (pp. 20–25). doi:10.1109/ICHR.2006.321358.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Srinivasa, S.S., Ferguson, D., Helfrich, C.J. et al. HERB: a home exploring robotic butler. Auton Robot 28, 5–20 (2010). https://doi.org/10.1007/s10514-009-9160-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10514-009-9160-9