Skip to main content
SpringerLink
Log in

A Theory of Minimal 3D Point to 3D Plane Registration and Its Generalization

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Registration of 3D data is a key problem in many applications in computer vision, computer graphics and robotics. This paper provides a family of minimal solutions for the 3D-to-3D registration problem in which the 3D data are represented as points and planes. Such scenarios occur frequently when a 3D sensor provides 3D points and our goal is to register them to a 3D object represented by a set of planes. In order to compute the 6 degrees-of-freedom transformation between the sensor and the object, we need at least six points on three or more planes. We systematically investigate and develop pose estimation algorithms for several configurations, including all minimal configurations, that arise from the distribution of points on planes. We also identify the degenerate configurations in such registrations. The underlying algebraic equations used in many registration problems are the same and we show that many 2D-to-3D and 3D-to-3D pose estimation/registration algorithms involving points, lines, and planes can be mapped to the proposed framework. We validate our theory in simulations as well as in three real-world applications: registration of a robotic arm with an object using a contact sensor, registration of planar city models with 3D point clouds obtained using multi-view reconstruction, and registration between depth maps generated by a Kinect sensor.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. http://cmp.felk.cvut.cz/minimal/.

References

  • Besl, P. J., & McKay, N. D. (1992). A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256.

    Article  Google Scholar 

  • Chen, H. H. (1991). Pose determination from line-to-plane correspondences: existence condition and closed-form solutions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(6), 530–541.

    Article  Google Scholar 

  • Chen, Y., & Medioni, G. (1991). Object modeling by registration of multiple range images. In Proc. IEEE int’l conf. robotics automation (ICRA) (Vol. 3, pp. 2724–2729).

    Google Scholar 

  • Dhome, M., Richetin, M., Lapresté, J. T., & Rives, G. (1989). Determination of the attitude of 3D objects from a single perspective view. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(12), 1265–1278.

    Article  Google Scholar 

  • Drost, B., Ulrich, M., Navab, N., & Ilic, S. (2010). Model globally, match locally: efficient and robust 3D object recognition. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (pp. 998–1005).

    Google Scholar 

  • Enqvist, O., Josephson, K., & Kahl, F. (2009). Optimal correspondences from pairwise constraints. In Proc. IEEE int’l conf. computer vision (ICCV) (pp. 1295–1302).

    Google Scholar 

  • Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.

    Article  MathSciNet  Google Scholar 

  • Fitzgibbon, A. W. (2003). Robust registration of 2D and 3D point sets. Image and Vision Computing, 21(13–14), 1145–1153.

    Article  Google Scholar 

  • Furukawa, Y., & Ponce, J. (2010). Accurate, dense, and robust multi-view stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8), 1362–1376.

    Article  Google Scholar 

  • Gao, X. S., Hou, X. R., Tang, J., & Cheng, H. F. (2003). Complete solution classification for the perspective-three-point problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(8), 930–943.

    Article  Google Scholar 

  • Geyer, C., & Stewenius, H. (2007). A nine-point algorithm for estimating para-catadioptric fundamental matrices. In Proc. IEEE conf. computer vision and pattern recognition (CVPR).

    Google Scholar 

  • Grimson, W. E. L., & Lozano-Pérez, T. (1983). Model-based recognition and localization from sparse range or tactile data. MIT AI Lab, A. I. Memo, 738.

  • Haralick, R. M., Lee, C. N., Ottenberg, K., & Nolle, M. (1994). Review and analysis of solutions of the three point perspective pose estimation problem. International Journal of Computer Vision, 13(3), 331–356.

    Article  Google Scholar 

  • Horn, B. K. P. (1987). Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America. A, Online, 4(4), 629–642.

    Article  MathSciNet  Google Scholar 

  • Kukelova, Z., Bujnak, M., & Pajdla, T. (2008). Automatic generator of minimal problem solvers. In Proc. European conf. computer vision (ECCV) (pp. 302–315).

    Google Scholar 

  • Kukelova, Z., Bujnak, M., & Pajdla, T. (2010). Closed-form solutions to the minimal absolute pose problems with known vertical direction. In Proc. Asian conf. computer vision (ACCV) (pp. 216–229).

    Google Scholar 

  • Kukelova, Z., & Pajdla, T. (2007). A minimal solution to the autocalibration of radial distortion. In Proc. IEEE conf. computer vision and pattern recognition (CVPR).

    Google Scholar 

  • Li, H., & Hartley, R. (2005). A non-iterative method for correcting lens distortion from nine-point correspondences. In Proc. OMNIVIS.

    Google Scholar 

  • Li, H., & Hartley, R. (2007). The 3D-3D registration problem revisited. In Proc. IEEE int’l conf. computer vision (ICCV) (pp. 1–8).

    Google Scholar 

  • Naroditsky, O., Patterson, A., & Daniilidis, K. (2011). Automatic alignment of a camera with a line scan LIDAR system. In Proc. IEEE int’l conf. robotics automation (ICRA) (pp. 3429–3434).

    Google Scholar 

  • Naroditsky, O., Zhou, X. S., Gallier, J., Roumeliotis, S. I., & Daniilidis, K. (2012). Two efficient solutions for visual odometry using directional correspondence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(4), 818–824.

    Article  Google Scholar 

  • Nistér, D. (2003). An efficient solution to the five-point relative pose problem. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (pp. 195–202).

    Google Scholar 

  • Nistér, D. (2004). A minimal solution to the generalized 3-point pose problem. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (Vol. 1, pp. 560–567).

    Google Scholar 

  • Nistér, D., & Schaffalitzky, F. (2006). Four points in two or three calibrated views: theory and practice. International Journal of Computer Vision, 67(2), 211–231.

    Article  Google Scholar 

  • Olsson, C., Kahl, F., & Oskarsson, M. (2006). The registration problem revisited: optimal solutions from points, lines and planes. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (Vol. 1, pp. 1206–1213).

    Google Scholar 

  • Raguram, R., Frahm, J. M., & Pollefeys, M. (2008). A comparative analysis of RANSAC techniques leading to adaptive real-time random sample consensus. In Proc. European conf. computer vision (ECCV) (pp. 500–513).

    Google Scholar 

  • Ramalingam, S., & Lodha, S. K. (2003). Adaptive enhancement of 3D scenes using hierarchical registration of texture-mapped 3D models. In Proc. int’l conf. 3-D digital imaging and modeling (3DIM) (pp. 203–210).

    Google Scholar 

  • Ramalingam, S., Taguchi, Y., Marks, T. K., & Tuzel, O. (2010). P2Π: a minimal solution for registration of 3D points to 3D planes. In Proc. European conf. computer vision (ECCV) (Vol. 5, pp. 436–449).

    Google Scholar 

  • Rusinkiewicz, S., & Levoy, M. (2001). Efficient variants of the ICP algorithm. In Proc. int’l conf. 3-D digital imaging and modeling (3DIM) (pp. 145–152).

    Chapter  Google Scholar 

  • Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., & Blake, A. (2011). Real-time human pose recognition in parts from single depth images. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (pp. 1297–1304).

    Google Scholar 

  • Snavely, N., Seitz, S. M., & Szeliski, R. (2006). Photo tourism: exploring image collections in 3D. ACM Transactions on Graphics, 25(3), 835–846.

    Article  Google Scholar 

  • Stewenius, H., Nistér, D., Kahl, F., & Schaffalitzky, F. (2005a). A minimal solution for relative pose with unknown focal length. In Proc. IEEE conf. computer vision and pattern recognition (CVPR) (Vol. 2, pp. 789–794).

    Google Scholar 

  • Stewenius, H., Nistér, D., Oskarsson, M., & Astrom, K. (2005b). Solutions to minimal generalized relative pose problems. In Proc. OMNIVIS.

    Google Scholar 

  • Tu, P., Saxena, T., & Hartley, R. (1999). Recognizing objects using color-annotated adjacency graphs. In Lecture notes in computer science: shape, contour and grouping in computer vision (pp. 246–263).

    Chapter  Google Scholar 

Download references

Acknowledgements

Special thanks to Tim K. Marks and Oncel Tuzel in helping us in the conference submission. We would like to thank Jay Thornton, Keisuke Kojima, John Barnwell, and Haruhisa Okuda for their valuable feedback, help and support. We would also like to thank the anonymous reviewers and AC’s comments in ECCV 2010.

Author information

Authors and Affiliations

  1. Mitsubishi Electric Research Laboratories (MERL), Cambridge, MA, USA

    Srikumar Ramalingam & Yuichi Taguchi

Authors
  1. Srikumar Ramalingam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuichi Taguchi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Srikumar Ramalingam.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ramalingam, S., Taguchi, Y. A Theory of Minimal 3D Point to 3D Plane Registration and Its Generalization. Int J Comput Vis 102, 73–90 (2013). https://doi.org/10.1007/s11263-012-0576-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-012-0576-x

Keywords

Search

Navigation