Skip to main content
Log in

Stereo Matching with Nonlinear Diffusion

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

One of the central problems in stereo matching (and other image registration tasks) is the selection of optimal window sizes for comparing image regions. This paper addresses this problem with some novel algorithms based on iteratively diffusing support at different disparity hypotheses, and locally controlling the amount of diffusion based on the current quality of the disparity estimate. It also develops a novel Bayesian estimation technique, which significantly outperforms techniques based on area-based matching (SSD) and regular diffusion. We provide experimental results on both synthetic and real stereo image pairs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Arnold, R.D. 1983. Automated stereo perception. Technical Report AIM-351, Artificial Intelligence Laboratory, Stanford University.

  • Baker, H.H. 1980. Edge based stereo correlation. In Image Understanding Workshop, L.S. Baumann (Ed.), Science Applications International Corporation, pp. 168–175.

  • Barnard, S.T. 1989. Stochastic stereo matching over scale. International Journal of Computer Vision, 3(1):17–32.

    Google Scholar 

  • Barnard, S.T. and Fischler, M.A. 1982. Computational stereo. ACM Computing Surveys, 14(4):553–572.

    Google Scholar 

  • Belhumeur, P.N. and Mumford, D. 1992. A Bayesian treatment of the stereo correspondence problem using half-occluded regions. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'92), Champaign-Urbana, IL, IEEE Computer Society Press, pp. 506–512.

    Google Scholar 

  • Besag, J. 1986. On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society, B-48(3):259–302.

    Google Scholar 

  • Black, M.J. and Rangarajan, A. 1996. On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. International Journal of Computer Vision, 19(1):57– 91.

    Google Scholar 

  • Blake, A. and Zisserman, A. 1987.Visual Reconstruction.MITPress: Cambridge, MA.

    Google Scholar 

  • Bolles, R.C., Baker, H.H., and Marimont, D.H. 1987. Epipolarplane image analysis: An approach to determining structure from motion. International Journal of Computer Vision, 1:7– 55.

    Google Scholar 

  • Cover, T.M. and Thomas, J.A. 1991. Elements of Information Theory. John Wiley & Sons: New York.

    Google Scholar 

  • Cox, I.J. 1994. A maximum likelihood n-camera stereo algorithm. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'94), Seattle, WA, IEEE Computer Society Press, pp. 733–739.

    Google Scholar 

  • Dhond, U.R. and Aggarwal, J.K. 1989. Structure from stereo—A review. IEEE Transactions on Systems, Man, and Cybernetics, 19(6):1489–1510.

    Google Scholar 

  • Fua, P. 1993. A parallel stereo algorithm that produces dense depth maps and preserves image features. Machine Vision and Applications, 6:35–49.

    Google Scholar 

  • Geiger, D. and Girosi, F. 1991. Mean field theory for surface reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(5):401–412.

    Google Scholar 

  • Geiger, D., Ladendorf, B., and Yuille, A. 1992. Occlusions and binocular stereo. In Second European Conference on Computer Vision (ECCV'92), Santa Margherita Ligure, Italy, LNCS 588, Springer-Verlag, pp. 425–433.

  • Geman, S. and Geman, D. 1984. Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6):721– 741.

    Google Scholar 

  • Grimson, W.E.L. 1985. Computational experiments with a feature based stereo algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 7(1):17–34.

    Google Scholar 

  • Hoff, W. and Ahuja, N. 1989. Surfaces from stereo: Integrating feature matching, disparity estimation, and contour detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(2):121–136.

    Google Scholar 

  • Huber, P.J. 1981. Robust Statistics. John Wiley & Sons: New York, NY.

    Google Scholar 

  • Intille, S.S. and Bobick, A.F. 1994. Disparity-space images and large occlusion stereo. In Third European Conference on Computer Vision (ECCV';94), Stockholm, Sweden, LNCS 801, Springer-Verlag, vol. 2, pp. 179–186.

    Google Scholar 

  • Jenkin, M.R.M., Jepson, A.D., and Tsotsos, J.K. 1991. Techniques for disparity measurement. CVGIP: Image Understanding, 53(1):14–30.

    Google Scholar 

  • Jones, D.G. and Malik, J. 1992. A computational framework for determining stereo correspondence from a set of linear spatial filters. In Second European Conference on Computer Vision (ECCV'92), Santa Margherita Ligure, Italy, LNCS 588, Springer-Verlag, pp. 395–410.

  • Kanade, T. 1994. Development of a video-rate stereo machine. In Image Understanding Workshop, Monterey, CA, Morgan Kaufmann Publishers, pp. 549–557.

    Google Scholar 

  • Kanade, T. and Okutomi, M. 1994. A stereo matching algorithm with an adaptive window: Theory and experiment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(9):920– 932.

    Google Scholar 

  • Kang, S.B., Webb, J., Zitnick, L., and Kanade, T. 1995. A multibaseline stereo system with active illumination and real-time image acquisition. In Fifth International Conference on Computer Vision (ICCV’95), MIT, Cambridge, MA, IEEE Computer Society Press, pp. 88–93.

    Google Scholar 

  • Lucas, B.D. and Kanade, T. 1981. An iterative image registration technique with an application in stereo vision. In Seventh International Joint Conference on Artificial Intelligence (IJCAI-81), Vancouver, pp. 674–679.

  • Marr, D. and Poggio, T. 1976. Cooperative computation of stereo disparity. Science, 194:283–287.

    Google Scholar 

  • Marr, D.C. and Poggio, T. 1979. A computational theory of human stereo vision. Proceedings of the Royal Society of London, B204:301–328.

    Google Scholar 

  • Marroquin, J., Mitter, S., and Poggio, T. 1987. Probabilistic solution of ill-posed problems in computational vision. Journal of the American Statistical Association, 82(397):76– 89.

    Google Scholar 

  • Matthies, L., Szeliski, R., and Kanade, T. 1989. Kalman filter-based algorithms for estimating depth from image sequences. International Journal of Computer Vision, 3:209–236.

    Google Scholar 

  • Nordström, N. 1990. Biased anisotropic diffusion—A unified regularization and diffusion approach to edge detection. In First European Conference on Computer Vision (ECCV'90), Antibes, France, LNCS 427, Springer-Verlag, pp. 18–27.

  • Ohta, Y. and Kanade, T. 1985. Stereo by intra-and interscanline search using dynamic programming. IEEE Transactions on Pattern Analysis and Machine Intelligence, 7(2):139– 154.

    Google Scholar 

  • Okutomi, M. and Kanade, T. 1992. A locally adaptive window for signal matching. International Journal of Computer Vision, 7(2):143–162.

    Google Scholar 

  • Okutomi, M. and Kanade, T. 1993. A multiple-baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(4):353–363.

    Google Scholar 

  • Olsen, S.I. 1990. Stereo correspondence by surface reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(3):309–314.

    Google Scholar 

  • Parisi, G. 1988. Statistical Field Theory. Addison-Wesley.

  • Perona, P. and Malik, J. 1990. Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7):629–639.

    Google Scholar 

  • Peterson, C. and Söderberg, B. 1989. A new method of mapping optimization problems onto neural networks. International Journal of Neural Systems, 1(1):3.

    Google Scholar 

  • Pollard, S.B., Mayhew, J.E.W., and Frisby, J.P. 1985. PMF: A stereo correspondence algorithm using a disparity gradient limit. Perception, 14:449–470.

    Google Scholar 

  • Prazdny, K. 1985. Detection of binocular disparities. Biological Cybernetics, 52(2):93–99.

    Google Scholar 

  • Proesmans, M., VanGool, L.J., Pauwels, E., and Oosterlinck, A. 1994. Determination of optical flow and its discontinuities using nonlinear diffusion. In Third European Conference on Computer Vision (ECCV'94), Stockholm, Sweden, LNCS 801, Springer-Verlag, vol. 2, pp. 295–304.

    Google Scholar 

  • Quam, L.H. 1984. Hierarchicalwarp stereo. In Image Understanding Workshop, New Orleans, Louisiana, Science Applications International Corporation, pp. 149–155.

    Google Scholar 

  • Ryan, T.W., Gray, R.T., and Hunt, B.R. 1980. Prediction of correlation errors in stereo-pair images. Optical Engineering, 19(3):312– 322.

    Google Scholar 

  • Scharstein, D. 1994. Matching images by comparing their gradient fields. In 12th International Conference on Pattern Recognition (ICPR’94), Jerusalem, Israel, vol. 1, pp. 572–575.

    Google Scholar 

  • Seitz, P. 1989. Using local orientation information as image primitive for robust object recognition. In SPIE Visual Communications and Image Processing IV, vol. 1199, pp. 1630–1639.

    Google Scholar 

  • Shah, J. 1993. A nonlinear diffusion model for discontinuous disparity and half-occlusion in stereo. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'93), New York, NY, IEEE Computer Society Press, pp. 34–40.

    Google Scholar 

  • Stewart, C.V., Flatland, R.Y., and Bubna, K. 1996. Geometric constraints and stereo disparity computation. International Journal of Computer Vision, 20(3):143–168.

    Google Scholar 

  • Szeliski, R. 1989. Bayesian Modeling of Uncertainty in Low-Level Vision. Kluwer Academic Publishers: Boston, MA.

    Google Scholar 

  • Szeliski, R. and Hinton, G. 1985. Solving random-dot stereograms using the heat equation. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’85), San Francisco, CA, IEEE Computer Society Press, pp. 284– 288.

    Google Scholar 

  • Szeliski, R. and Golland, P. 1998. Stereo matching with transparency and matting. In Sixth International Conference on Computer Vision (ICCV'98), Bombay, India, IEEE Computer Society Press.

    Google Scholar 

  • Terzopoulos, D. 1986. Regularization of inverse visual problems involving discontinuities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(4):413–424.

    Google Scholar 

  • Tian, Q. and Huhns, M.N. 1986. Algorithms for subpixel registration. Computer Vision, Graphics, and Image Processing, 35:220– 233.

    Google Scholar 

  • Witkin, A., Terzopoulos, D., and Kass, M. 1987. Signal matching through scale space. International Journal of Computer Vision, 1:133–144.

    Google Scholar 

  • Yang, Y., Yuille, A., and Lu, J. 1993. Local, global, and multilevel stereo matching. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'93), New York, NY, IEEE Computer Society Press, pp. 274–279.

    Google Scholar 

  • Yuille, A.L. and Poggio, T. 1984. A generalized ordering constraint for stereo correspondence. A.I. Memo 777, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA.

    Google Scholar 

  • Zabih, R. and Woodfill, J. 1994. Non-parametric local transforms for computing visual correspondence. In Third European Conference on Computer Vision (ECCV'94), Stockholm, Sweden, LNCS 801, Springer-Verlag, vol. 2, pp. 151– 158.

    Google Scholar 

  • Zerubia, J. and Chepalla, R. 1993. Mean field annealing using compound Gauss-Markov random fields for edge detection and image estimation. IEEE Transactions on Neural Networks, 4(4).

  • Zhang, J., Modestino, J.W., and Langan, D.A. 1994. Maximumlikelihood parameter estimation for unsupervised stochastic model-based image segmentation. IEEE Transactions on Image Processing, 3(4):404–420.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Scharstein, D., Szeliski, R. Stereo Matching with Nonlinear Diffusion. International Journal of Computer Vision 28, 155–174 (1998). https://doi.org/10.1023/A:1008015117424

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008015117424

Navigation