Abstract
Images now come in different forms – color, near-infrared, depth, etc. – due to the development of special and powerful cameras in computer vision and computational photography. Their cross-modal correspondence establishment is however left behind. We address this challenging dense matching problem considering structure variation possibly existing in these image sets and introduce new model and solution. Our main contribution includes designing the descriptor named robust selective normalized cross correlation (RSNCC) to establish dense pixel correspondence in input images and proposing its mathematical parameterization to make optimization tractable. A computationally robust framework including global and local matching phases is also established. We build a multi-modal dataset including natural images with labeled sparse correspondence. Our method will benefit image and vision applications that require accurate image alignment.
Chapter PDF
Similar content being viewed by others
References
Agrawal, A.K., Raskar, R., Nayar, S.K., Li, Y.: Removing photography artifacts using gradient projection and flash-exposure sampling. ToG 24(3), 828–835 (2005)
Andronache, A., von Siebenthal, M., Székely, G., Cattin, P.C.: Non-rigid registration of multi-modal images using both mutual information and cross-correlation. Medical Image Analysis 12(1), 3–15 (2008)
Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. CVIU 63(1), 75–104 (1996)
Brown, M., Susstrunk, S.: Multi-spectral sift for scene category recognition. In: CVPR, pp. 177–184 (2011)
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J.(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)
Bruhn, A., Weickert, J.: Towards ultimate motion estimation: Combining highest accuracy with real-time performance. In: ICCV, pp. 749–755 (2005)
Chui, H., Rangarajan, A.: A new point matching algorithm for non-rigid registration. Computer Vision and Image Understanding 89(2), 114–141 (2003)
Fattal, R., Lischinski, D., Werman, M.: Gradient domain high dynamic range compression. ToG 21(3), 249–256 (2002)
Firmenichy, D., Brown, M., Süsstrunk, S.: Multispectral interest points for rgb-nir image registration. In: ICIP, pp. 181–184 (2011)
Han, J., Pauwels, E.J., de Zeeuw, P.M.: Visible and infrared image registration in man-made environments employing hybrid visual features. Pattern Recognition Letters 34(1), 42–51 (2013)
Heo, Y.S., Lee, K.M., Lee, S.U.: Robust stereo matching using adaptive normalized cross-correlation. PAMI 33(4), 807–822 (2011)
Hermosillo, G., Chefd’Hotel, C., Faugeras, O.D.: Variational methods for multimodal image matching. IJCV 50(3), 329–343 (2002)
Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell. 17(1-3), 185–203 (1981)
Hrkać, T., Kalafatić, Z., Krapac, J.: Infrared-visual image registration based on corners and hausdorff distance. In: Ersbøll, B.K., Pedersen, K.S. (eds.) SCIA 2007. LNCS, vol. 4522, pp. 383–392. Springer, Heidelberg (2007)
Irani, M., Anandan, P.: Robust multi-sensor image alignment. In: ICCV, pp. 959–966 (1998)
Jian, B., Vemuri, B.C.: Robust point set registration using gaussian mixture models. PAMI 33(8), 1633–1645 (2011)
Kolár, R., Kubecka, L., Jan, J.: Registration and fusion of the autofluorescent and infrared retinal images. International Journal of Biomedical Imaging (2008)
Krishnan, D., Fergus, R.: Dark flash photography. ToG 28(3) (2009)
Liu, C., Yuen, J., Torralba, A., Sivic, J., Freeman, W.T.: Sift flow: Dense correspondence across different scenes. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 28–42. Springer, Heidelberg (2008)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–110 (2004)
Palos, G., Betrouni, N., Coulanges, M., Vermandel, M., Devlaminck, V., Rousseau, J.: Multimodal matching by maximisation of mutual information and optical flow technique. In: IEEE International Conference on Engineering in Medicine and Biology Society, pp. 1679–1682 (2004)
Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M.F., Hoppe, H., Toyama, K.: Digital photography with flash and no-flash image pairs. ToG 23(3), 664–672 (2004)
Pluim, J.P.W., Maintz, J.B.A., Viergever, M.A.: Mutual information based registration of medical images: A survey. IEEE Transaction on Medical Imaging 22(8), 986–1004 (2003)
Sen, P., Kalantari, N.K., Yaesoubi, M., Darabi, S., Goldman, D.B., Shechtman, E.: Robust patch-based hdr reconstruction of dynamic scenes. ToG 31(6), 203 (2012)
Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: CVPR, pp. 2432–2439 (2010)
Sun, J., Kang, S.B., Xu, Z., Tang, X., Shum, H.Y.: Flash cut: Foreground extraction with flash and no-flash image pairs. In: CVPR (2007)
Sun, J., Zheng, N., Shum, H.Y.: Stereo matching using belief propagation. PAMI 25(7), 787–800 (2003)
Szeliski, R.: Image alignment and stitching: A tutorial. Foundations and Trends in Computer Graphics Vision 2(1), 1–104 (2006)
Tsin, Y., Kanade, T.: A correlation-based approach to robust point set registration. In: Pajdla, T., Matas, J.(G.) (eds.) ECCV 2004. LNCS, vol. 3023, pp. 558–569. Springer, Heidelberg (2004)
Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM Journal on Imaging Sciences 1(3), 248–272 (2008)
Wedel, A., Rabe, C., Vaudrey, T., Brox, T., Franke, U., Cremers, D.: Efficient dense scene flow from sparse or dense stereo data. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 739–751. Springer, Heidelberg (2008)
Werlberger, M., Pock, T., Bischof, H.: Motion estimation with non-local total variation regularization. In: CVPR, pp. 2464–2471 (2010)
Xiong, Z., Zhang, Y.: A critical review of image registration methods. International Journal of Image and Data Fusion 1(2), 137–158 (2010)
Xu, L., Jia, J., Matsushita, Y.: Motion detail preserving optical flow estimation. PAMI 34(9), 1744–1757 (2012)
Yan, Q., Shen, X., Xu, L., Zhuo, S., Zhang, X., Shen, L., Jia, J.: Cross-field joint image restoration via scale map. In: ICCV (2013)
Yang, J., Blum, R.S., Williams, J.P., Sun, Y., Xu, C.: Non-rigid image registration using geometric features and local salient region features. In: CVPR, pp. 825–832 (2006)
Yi, Z., Soatto, S.: Nonrigid registration combining global and local statistics. In: CVPR (2009)
Yuan, L., Sun, J., Quan, L., Shum, H.Y.: Image deblurring with blurred/noisy image pairs. ToG 26(3) (2007)
Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L1 optical flow. Pattern Recognition, 214–223 (2007)
Zhang, Z., Jiang, Y., Tsui, H.: Consistent multi-modal non-rigid registration based on a variational approach. Pattern Recognition Letters 27(7), 715–725 (2006)
Zimmer, H., Bruhn, A., Weickert, J., Valgaerts, L., Salgado, A., Rosenhahn, B., Seidel, H.-P.: Complementary optic flow. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds.) EMMCVPR 2009. LNCS, vol. 5681, pp. 207–220. Springer, Heidelberg (2009)
Zitová, B., Flusser, J.: Image registration methods: a survey. Image and Vision Computing 21(11), 977–1000 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Shen, X., Xu, L., Zhang, Q., Jia, J. (2014). Multi-modal and Multi-spectral Registration for Natural Images. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8692. Springer, Cham. https://doi.org/10.1007/978-3-319-10593-2_21
Download citation
DOI: https://doi.org/10.1007/978-3-319-10593-2_21
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10592-5
Online ISBN: 978-3-319-10593-2
eBook Packages: Computer ScienceComputer Science (R0)