Abstract
Manipulation tasks often require robots to recognize interactions between objects. For example, a robot may need to determine if it has grasped an object properly or if one object is resting on another in a stable manner. These interactions usually depend on the contacts between the objects, with different distributions of contacts affording different interactions. In this paper, we address the problem of learning to recognize interactions between objects based on contact distributions. We present a kernel-based approach for representing the estimated contact distributions. The kernel can be used for various interactions, and it allows the robot to employ a variety of kernel methods from machine learning. The approach was evaluated on blind grasping, lifting, and stacking tasks. Using 30 training samples and the proposed kernel, the robot already achieved classification accuracies of 71.9, 85.93, and 97.5% for the blind grasping, lifting and stacking tasks respectively. The kernel was also used to cluster interactions using spectral clustering. The clustering method successfully differentiated between different types of interactions, including placing, inserting, and pushing. The contact points were extracted using tactile sensors or 3D point cloud models of the objects. The robot could construct small towers of assorted blocks using the classifier for the stacking task.
Similar content being viewed by others
References
Abdo, N., Kretzschmar, H., Spinello, L., & Stachniss, C. (2013). Learning manipulation actions from a few demonstrations. In: International conference on robotics and automation (ICRA)
Amores, J. (2013). Multiple instance classification: Review, taxonomy and comparative study. Artificial Intelligence, 201, 81–105.
Bekiroglu, Y., Detry, R., & Kragic, D. (2011) Learning tactile characterizations of object- and pose-specific grasps. In: International conference on intelligent robots and systems (IROS)
Ben Amor, H., Kroemer, O., Hillenbrand, U., Neumann, G., & Peters, J. (2012). Generalization of human grasping for multi-fingered robot hands. In: International conference on intelligent robots and systems (IROS)
Bicchi, A., & Kumar, V. (2000). Robotic grasping and contact: A review. In: International conference on robotics and automation (ICRA)
Bohg, J., Johnson-Roberson, M., León, B., Felip, J., Gratal, X., Bergström, N., Kragic, D., & Morales, A. (2011). Mind the gap–robotic grasping under incomplete observation. In: International conference on robotics and automation (ICRA)
Bohg, J., Morales, A., Asfour, T., & Kragic, D. (2014). Data-driven grasp synthesis-a survey. IEEE Transactions on Robotics, 30, 289–309.
Boularias, A., Kroemer, O., & Peters, J. (2011). Learning robot grasping from 3d images with markov random fields. In: International conference on intelligent robot systems (IROS)
Chebotar, Y., Hausman, K., Kroemer, O., Sukhatme, G., & Schaal, S. (2016). Generalizing regrasping with supervised policy learning. In: International symposium on experimental robotics (ISER)
Chen, Z., Lii, N. Y., Wimboeck, T., Fan, S., Jin, M., Borst, C., & Liu, H. (2010). Experimental study on impedance control for the five-finger dexterous robot hand dlr-hit ii. In: International conference on intelligent robots and systems (IROS)
Dang, H., & Allen, P. K. (2012). Learning grasp stability. In: International conference on robotics and automation (ICRA)
Detry, R., Ek, C. H., Madry, M., Piater, J., & Kragic, D. (2012). Generalizing grasps across partly similar objects. In: International conference on robotics and automation (ICRA)
Eppner, C., & Brock, O. (2013). Grasping unknown objects by exploiting shape adaptability and environmental constraints. In: International conference on intelligent robots and systems (IROS)
Ferrari, C., & Canny, J. (1992). Planning optimal grasps. In: International conference on robotics and automation (ICRA) pp 2290–2295
Gibson, J. J. (1986). The ecological approach to visual perception. Hillsdale, NJ: Lawrence Erlbaum Associates.
Griffith, S., Sinapov, J., Sukhoy, V., & Stoytchev, A. (2012). A behavior-grounded approach to forming object categories: Separating containers from noncontainers. IEEE Transactions on Autonomous Mental Development, 4(1), 54–69.
Goldfeder, C., Ciocarlie, M., Dang, H., & Allen, P. K. (2009). The Columbia grasp database. In: International conference on robotics and automation (ICRA)
Hermans, T., Li, F, Rehg, J. M., & Bobick, A. F. (2013). Learning contact locations for pushing and orienting unknown objects. In: International conference on humanoid robots
Herzog, A., Pastor, P., Kalakrishnan, M., Righetti, L., Bohg, J., Asfour, T., et al. (2013). Learning of grasp selection based on shape-templates. Autonomous Robots, 36, 51–65.
Hofmann, T., Schölkopf, B., & Smola, A. J. (2008). Kernel methods in machine learning. Annals of Statistics
Jebara, T., Kondor, R., & Howard, A. (2004). Probability product kernels. Journal of Machine Learning Research (JMLR), 5, 819–844.
Jebara, T., Kondor, R., & Howard, A. (2004). Probability product kernels. Journal of Machine Learning Research (JMLR), 5, 819–844.
Jenssen, R., Principe, J. C., Erdogmus, D., & Eltoft, T. (2006). The cauchy-schwarz divergence and parzen windowing: Connections to graph theory and mercer kernels. Journal of the Franklin Institute, 343(6), 614–629.
Jentoft, L. P., Tenzer, Y., Vogt, D., Liu, J., Wood, R. J., Howe, R. D. (2013). Flexible, stretchable tactile arrays from mems barometers. In: International conference on advanced robotics (ICAR)
Jiang, Y., Lim, M., Zheng, C., & Saxena, A. (2012). Learning to place new objects in a scene. International Journal of Robotic Research (IJRR), 31(9), 1021–1043.
Kappler, D., Bohg, B., & Schaal, S. (2015). Leveraging big data for grasp planning. In: IEEE international conference on robotics and automation (ICRA)
Karayiannidis, Y., Smith, C., Vina, F. E., & Kragic, D. (2014). Online contact point estimation for uncalibrated tool use. In: International conference on robotics and automation (ICRA)
Kopicki, M., Detry, R., Adjigble, M., Stolkin, R., Leonardis, A., & Wyatt, J. L. (2015). One shot learning and generation of dexterous grasps for novel objects. The International Journal of Robotics Research (IJRR), 35, 959–976.
Kopicki, M. S., Zurek, S., Stolkin, R., Morwald, T., & Wyatt, J. L. (2011). Learning to predict how rigid objects behave under simple manipulation. In: International conference on robotics and automation (ICRA)
Kroemer, O., Ben Amor, H., Ewerton, M., & Peters, J. (2012a). Point cloud completion using extrusions. In: The international conference on humanoid robots
Kroemer, O., Daniel, C., Neumann, G., van Hoof, H., & Peters, J. (2015). Towards learning hierarchical skills for multi-phase manipulation tasks. In: International conference on robotics and automation (ICRA)
Kroemer, O., & Peters, J. (2014). Predicting object interactions from contact distributions. In: International conference on intelligent robots and systems
Kroemer, O., Ugur, E., Oztop, E., & Peters, J. (2012b). A kernel-based approach to direct action perception. In: International conference on robotics and automation (ICRA)
Kulick, J., Lang, T., Toussaint, M., & Lopes, M. (2013). Active Learning for Teaching a Robot Grounded Relational Symbols. In: International joint conference on artificial intelligence (IJCAI)
Laaksonen, J., Nikandrova, E., & Kyrki, V. (2012). Probabilistic sensor-based grasping. In: International conference on intelligent robots and systems (IROS)
Leischnig, S., Luettgen, S., Kroemer, O., & Peters, J. (2015). A comparison of contact distribution representations for learning to predict object interactions. In: International conference on humanoid robots
Lenz, I., Lee, H., & Saxena, A. (2013). Deep learning for detecting robotic grasps. In: Robotics: Science and systems (RSS)
Li, Q., Schürmann, C., Haschke, R., & Ritter, H. J. (2013) A control framework for tactile servoing. In: Robotics: Science and systems (R:SS)
Li, Z., & Sastry, S. S. (1988). Task-oriented optimal grasping by multifingered robot hands. Journal of Robotics and Automation, 4(1), 32–44.
Luxburg, U. (2007). A tutorial on spectral clustering. Statistics and Computing, 17(4), 395–416.
Madry, M., Bo, L., Kragic, D., & Fox, D. (2014). ST-HMP: Unsupervised spatio-temporal feature learning for tactile data. In: International conference on robotics and automation (ICRA)
Miller, A., & Allen, P. (1999). Examples of 3d grasp quality computations. In: International conference on robotics and automation (ICRA)
Miller, A., & Allen, P. (2004). Graspit!: A versatile simulator for robotic grasping. IEEE Robotics and Automation Magazine, 11, 110–122.
Molchanov, A., Kroemer, O., Su, Z., & Sukhatme, G. S. (2016). Contact localization on grasped objects using tactile sensing. In: International conference on intelligent robots and systems (IROS)
Montesano, L., Lopes, M., Bernardino, A., & Santos-Victor, J. (2007). Modeling affordances using bayesian networks. In: International conference on intelligent robot systems (IROS)
Ning, X., & Karypis, G. (2008). The set classification problem and solution methods. In: International conference on data mining workshops
Odhner, L. U., Jentoft, L. P., Claffee, M. R., Corson, N., Tenzer, Y., Ma, R. R., et al. (2014). A compliant, underactuated hand for robust manipulation. The International Journal of Robotics Research (IJRR), 33(5), 736–752.
Roa, M. A., & Suàrez, R. (2015). Grasp quality measures: Review and performance. Autonomous Robots, 38, 65–88.
Rosman, B., & Ramamoorthy, S. (2011). Learning spatial relationships between objects. The International Journal of Robotics Research, 30, 1328–1342.
Rusu, R. B., & Cousins, S. (2011). 3D is here: Point Cloud Library (PCL). In: International conference on robotics and automation (ICRA)
Sahin, E., Cakmak, M., Dogar, M. R., Ugur, E., & Ucoluk, G. (2007). To Afford or Not to Afford: A New Formalization of Affordances Toward Affordance-Based Robot Control. Adaptive Behavior, 4, 447–472.
Schölkopf, B., & Smola, A. J. (2001). Learning with kernels: Support vector machines, regularization, optimization, and beyond (1st ed.). Cambridge: The MIT Press.
Shi, J., & Malik, J. (2000). Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 888–905.
Sjoo, K., & Jensfelt, P. (2011). Learning spatial relations from functional simulation. In: International conference on intelligent robots and systems (IROS)
ten Pa, A., & Platt, R. (2015). Using geometry to detect grasp poses in 3d point clouds. In: International symposium on robotics research (ISRR)
Trinkle, J., & Paul, R. P. (1990). Planning for dextrous manipulation with sliding contacts. International Journal of Robotics Research, 9(3), 24–48.
Ugur, E., & Piater, J. (2015). Bottom-up learning of object categories, action effects and logical rules: From continuous manipulative exploration to symbolic planning. In: International conference on robotics and automation (ICRA), pp 2627–2633
Vedaldi, A., Gulshan, V., Varma, M., & Zisserman, A. (2009). Multiple kernels for object detection. In: International conference on computer vision (ICCV)
Veiga, F., van Hoof, H., Peters, J., & Hermans, T. (2015). Stabilizing novel objects by learning to predict tactile slip. In: International conference on intelligent robots and systems (IROS)
Will, P. M., & Grossman, D. D. (1975). An experimental system for computer controlled mechanical assembly. IEEE Transactions on Computers, 24(9), 879–888.
Author information
Authors and Affiliations
Corresponding author
Additional information
The research leading to these results has received funding from the European Community’s Seventh Framework Programme under Grant Agreements 610878 (3rdHand), 600716 (CoDyCo), and 610967 (TACMAN).
Rights and permissions
About this article
Cite this article
Kroemer, O., Leischnig, S., Luettgen, S. et al. A kernel-based approach to learning contact distributions for robot manipulation tasks. Auton Robot 42, 581–600 (2018). https://doi.org/10.1007/s10514-017-9651-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10514-017-9651-z