Skip to main content
SpringerLink
Log in

Direct Image to Point Cloud Descriptors Matching for 6-DOF Camera Localization in Dense 3D Point Clouds

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2019)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11954))

Included in the following conference series:

Abstract

We propose a novel concept to directly match feature descriptors extracted from RGB images, with feature descriptors extracted from 3D point clouds. We use this concept to localize the position and orientation (pose) of the camera of a query image in dense point clouds. We generate a dataset of matching 2D and 3D descriptors, and use it to train a proposed Descriptor-Matcher algorithm. To localize a query image in a point cloud, we extract 2D key-points and descriptors from the query image. Then the Descriptor-Matcher is used to find the corresponding pairs 2D and 3D key-points by matching the 2D descriptors with the pre-extracted 3D descriptors of the point cloud. This information is used in a robust pose estimation algorithm to localize the query image in the 3D point cloud. Experiments demonstrate that directly matching 2D and 3D descriptors is not only a viable idea but can also be used for camera pose localization in dense 3D point clouds with high accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Kendall, A., Grimes, M., Cipolla, R.: PoseNet: a convolutional network for real-time 6-DOF camera relocalization. In: International Conference on Computer Vision, pp. 2938–2946. IEEE (2015)

    Google Scholar 

  2. Schönberger, J.L., Frahm, J.-M.: Structure-from-motion revisited. In: Conference on Computer Vision and Pattern Recognition. IEEE (2016)

    Google Scholar 

  3. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)

    Article  Google Scholar 

  4. Rusu, R.B., Cousins, S.: Point cloud library (PCL). In: International Conference on Robotics and Automation, pp. 1–4. IEEE (2011)

    Google Scholar 

  5. Harris, C.G., Stephens, M., et al.: A combined corner and edge detector. In: Alvey Vision Conference, vol. 15, pp. 10–5244. Citeseer (1988)

    Google Scholar 

  6. Lazebnik, S., Schmid, C., Ponce, J.: A sparse texture representation using local affine regions. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1265–1278 (2005)

    Article  Google Scholar 

  7. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  8. Chen, D.M., et al.: City-scale landmark identification on mobile devices. In: Conference on Computer Vision and Pattern Recognition, pp. 737–744. IEEE (2011)

    Google Scholar 

  9. Zamir, A.R., Shah, M.: Accurate image localization based on google maps street view. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 255–268. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15561-1_19

    Chapter  Google Scholar 

  10. Sattler, T., Leibe, B., Kobbelt, L.: Efficient & effective prioritized matching for large-scale image-based localization. IEEE Trans. Pattern Anal. Mach. Intell. 39(9), 1744–1756 (2017)

    Article  Google Scholar 

  11. Irschara, A., Zach, C., Frahm, J.-M., Bischof, H.: From structure-from-motion point clouds to fast location recognition. In: Conference on Computer Vision and Pattern Recognition, pp. 2599–2606. IEEE (2009)

    Google Scholar 

  12. Li, Y., Snavely, N., Huttenlocher, D.P.: Location recognition using prioritized feature matching. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 791–804. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15552-9_57

    Chapter  Google Scholar 

  13. Sattler, T., Havlena, M., Radenovic, F., Schindler, K., Pollefeys, M.: Hyperpoints and fine vocabularies for large-scale location recognition. In: International Conference on Computer Vision, pp. 2102–2110. IEEE (2015)

    Google Scholar 

  14. Kendall, A., Cipolla, R.: Geometric loss functions for camera pose regression with deep learning. In: Conference on Computer Vision and Pattern Recognition, pp. 5974–5983. IEEE (2017)

    Google Scholar 

  15. Walch, F., Hazirbas, C., Leal-Taixe, L., Sattler, T., Hilsenbeck, S., Cremers, D.: Image-based localization using LSTMs for structured feature correlation. In: International Conference on Computer Vision, pp. 627–637. IEEE (2017)

    Google Scholar 

  16. Brachmann, E., et al.: DSAC-differentiable RANSAC for camera localization. In: Conference on Computer Vision and Pattern Recognition, pp. 6684–6692. IEEE (2017)

    Google Scholar 

  17. Shotton, J., Glocker, B., Zach, C., Izadi, S., Criminisi, A., Fitzgibbon, A.: Scene coordinate regression forests for camera relocalization in RGB-D images. In: Conference on Computer Vision and Pattern Recognition, pp. 2930–2937. IEEE (2013)

    Google Scholar 

  18. Schönberger, J.L., Zheng, E., Frahm, J.-M., Pollefeys, M.: Pixelwise view selection for unstructured multi-view stereo. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 501–518. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_31

    Chapter  Google Scholar 

  19. Sattler, T., et al.: Benchmarking 6DOF outdoor visual localization in changing conditions. In: Conference on Computer Vision and Pattern Recognition, pp. 8601–8610. IEEE (2018)

    Google Scholar 

  20. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  Google Scholar 

  21. Gao, X.-S., Hou, X.-R., Tang, J., Cheng, H.-F.: Complete solution classification for the perspective-three-point problem. IEEE Trans. Pattern Anal. Mach. Intell. 25(8), 930–943 (2003)

    Article  Google Scholar 

  22. Torr, P.H., Zisserman, A.: MLESAC: a new robust estimator with application to estimating image geometry. Comput. Vis. Image Underst. 78(1), 138–156 (2000)

    Article  Google Scholar 

  23. Hane, C., Zach, C., Cohen, A., Angst, R., Pollefeys, M.: Joint 3D scene reconstruction and class segmentation. In: Conference on Computer Vision and Pattern Recognition, pp. 97–104. IEEE (2013)

    Google Scholar 

  24. Snavely, N., Seitz, S.M., Szeliski, R.: Modeling the world from internet photo collections. Int. J. Comput. Vision 80(2), 189–210 (2008)

    Article  Google Scholar 

  25. Valada, A., Radwan, N., Burgard, W.: Deep auxiliary learning for visual localization and odometry. In: International Conference on Robotics and Automation, pp. 6939–6946. IEEE (2018)

    Google Scholar 

Download references

Acknowledgments

This work was supported by the SIRF scholarship from the University of Western Australia (UWA) and by the Australian Research Council under Grant DP150100294.

Author information

Authors and Affiliations

  1. The University of Western Australia, Perth, Australia

    Uzair Nadeem, Mohammad A. A. K. Jalwana, Mohammed Bennamoun & Roberto Togneri

  2. Murdoch University, Perth, Australia

    Ferdous Sohel

Authors
  1. Uzair Nadeem
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad A. A. K. Jalwana
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammed Bennamoun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Roberto Togneri
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ferdous Sohel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Uzair Nadeem .

Editor information

Editors and Affiliations

  1. Australian National University, Canberra, ACT, Australia

    Tom Gedeon

  2. Murdoch University, Murdoch, WA, Australia

    Kok Wai Wong

  3. Kyungpook National University, Daegu, Korea (Republic of)

    Minho Lee

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nadeem, U., Jalwana, M.A.A.K., Bennamoun, M., Togneri, R., Sohel, F. (2019). Direct Image to Point Cloud Descriptors Matching for 6-DOF Camera Localization in Dense 3D Point Clouds. In: Gedeon, T., Wong, K., Lee, M. (eds) Neural Information Processing. ICONIP 2019. Lecture Notes in Computer Science(), vol 11954. Springer, Cham. https://doi.org/10.1007/978-3-030-36711-4_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-36711-4_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-36710-7

  • Online ISBN: 978-3-030-36711-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation