Skip to main content
SpringerLink
Log in

Observing Shape from Defocused Images

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Accommodation cues are measurable properties of an image that are associated with a change in the geometry of the imaging device. To what extent can three-dimensional shape be reconstructed using accommodation cues alone? This question is fundamental to the problem of reconstructing shape from focus (SFF) and shape from defocus (SFD) for applications in inspection, microscopy, image restoration and visualization. We address it by studying the “observability” of accommodation cues in an analytical framework that reveals under what conditions shape can be reconstructed from defocused images. We do so in three steps: (1) we characterize the observability of any surface in the presence of a controlled radiance (“weak observability”), (2) we conjecture the existence of a radiance that allows distinguishing any two surfaces (“sufficient excitation”) and (3) we show that in the absence of any prior knowledge on the radiance, two surfaces can be distinguished up to the degree of resolution determined by the complexity of the radiance (“strong observability”). We formulate the problem of reconstructing the shape and radiance of a scene as the minimization of the information divergence between blurred images, and propose an algorithm that is provably convergent and guarantees that the solution is admissible, in the sense of corresponding to a positive radiance and imaging kernel.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Asada, N., Fujiwara, H., and Matsuyama, T. 1998. Seeing behind the scene: Analysis of photometric properties of occluding edges by reversed projection blurring model. IEEE Trans. Pattern Analysis and Machine Intelligence, 20:155–167.

    Google Scholar 

  • Chan, T.F. and Wong, C.K. 1998. Total variation blind deconvolution. IEEE Transactions on Image Processing, 7(3):370–375.

    Google Scholar 

  • Chaudhuri, S. and Rajagopalan, A.N. 1999. Depth from Defocus: A Real Aperture Imaging Approach. Springer-Verlag: Berlin.

    Google Scholar 

  • Cover, T.M. and Thomas, J.A. 1991. Elements of Information Theory.Wiley Interscience.

  • Csiszár, I. 1991. Why least squares and maximum entropy? An axiomatic approach to inverse problems. Ann. of Stat., 19:2033– 2066.

    Google Scholar 

  • Ens, J. and Lawrence, P. 1993. An investigation of methods for determining depth from focus. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(2):97–108.

    Google Scholar 

  • Farid, H. and Simoncelli, E.P. 1998. Range estimation by optical differentiation. Journal of the Optical Society of America A, 15(7):1777–1786.

    Google Scholar 

  • Favaro, P. and Soatto, S. 2000. Shape and radiance estimation from the information divergence of blurred images. In Proc. European Conference on Computer Vision, 1:755–768.

    Google Scholar 

  • Friedlander, G. and Joshi, M. 1998. Introduction to the Theory of Distributions. Cambridge University Press: Cambridge.

    Google Scholar 

  • Girod, B. and Scherock, S. 1989. Depth from focus of structured light. In SPIE, pp. 209–215.

  • Gokstorp, M. 1994. Computing depth from out-of-focus blur using a local frequency representation. In International Conference on Pattern Recognition, Vol. A, pp. 153–158.

    Google Scholar 

  • Harikumar, G. and Bresler, Y. 1999. Perfect blind restoration of images blurred by multiple filters: Theory and efficient algorithms. IEEE Transactions on Image Processing, 8(2):202–219.

    Google Scholar 

  • Hopkins, H.H. 1955. The frequency response of a defocused optical system. Proc. R. Soc. London Ser. A, 231:91–103.

    Google Scholar 

  • Hwang, T.L., Clark, J.J., and Yuille, A.L. 1989. A depth recovery algorithm using defocus information. In Computer Vision and Pattern Recognition, pp. 476–482.

  • Kalifa, J., Mallat, S., and Rouge, B. 1998. Image deconvolution in mirror wavelet bases. In Proc. International Conference on Image Processing, p. 98.

  • Kundur, D. and Hatzinakos, D. 1998. A novel blind deconvolution scheme for image restoration using recursive filtering. IEEE Transactions on Signal Processing, 46(2):375–390.

    Google Scholar 

  • Luemberger, D. 1968. Optimization by Vector Space Methods.Wiley: New York.

    Google Scholar 

  • Marshall, J.A., Burbeck, C.A., Ariely, D., Rolland, J.P., and Martin, K.E. 1996. Occlusion edge blur: A cue to relative visual depth. Journal of the Optical Society of America A, 13(4):681–688.

    Google Scholar 

  • Mennucci, A. and Soatto, S. 1999. The accommodation cue, part 1: Modeling. Essrl Technical Report 99-001, Washington University.

  • Murase, H. and Nayar, S. 1995. Visual learning and recognition or 3d object from appearance. Intl. J. Computer Vision, 14(1):5–24.

    Google Scholar 

  • Nair, H.N. and Stewart, C.V. 1992. Robust focus ranging. In Computer Vision and Pattern Recognition, pp. 309–314.

  • Namba, M. and Ishida, Y. 1998. Wavelet transform domain blind deconvolution. Signal Processing, 68(1):119–124.

    Google Scholar 

  • Nayar, S.K. and Nakagawa, Y. 1990. Shape from focus: An effective approach for rough surfaces. In IEEE International Conference on Robotics and Automation, pp. 218–225.

  • Nayar, S.K., Watanabe, M., and Noguchi, M. 1995. Real-time focus range sensor. In Proc. International Conference on Computer Vision, pp. 995–1001.

  • Neelamani, R., Choi, H., and Baraniuk, R. 1999. Wavelet-domain regularized deconvolution for ill-conditioned systems. In Proc.International Conference on Image Processing, pp. 58–72.

  • Noguchi, M. and Nayar, S.K. 1994. Microscopic shape from focus using active illumination. In International Conference on Pattern Recognition, pp. 147–152.

  • Pentland, A. 1987. A new sense for depth of field. IEEE Trans.Pattern Anal. Mach. Intell., 9:523–531.

    Google Scholar 

  • Pentland, A., Darrell, T., Turk, M., and Huang, W. 1989. A simple, real-time range camera. In Computer Vision and Pattern Recognition, pp. 256–261.

  • Pentland, A., Scherock, S., Darrell, T., and Girod, B. 1994. Simple range cameras based on focal error. Journal of the Optical Society of America A, 11(11):2925–2934.

    Google Scholar 

  • Rajagopalan, A.N. and Chaudhuri, S. 1995. A block shift-variant blur model for recovering depth from defocused images. In Proc. International Conference on Image Processing, pp. 636– 639.

  • Rajagopalan, A.N. and Chaudhuri, S. 1997. Optimal selection of camera parameters for recovery of depth from defocused images.In Computer Vision and Pattern Recognition, pp. 219– 224.

  • Rajagopalan, A.N. and Chaudhuri, S. 1998. Optimal recovery of depth from defocused images using an mrf model. In Proc. International Conference on Computer Vision, pp. 1047–1052.

  • Schechner, Y.Y. and Kiryati, N. 1999. The optimal axial interval in estimating depth from defocus. In IEEE Proc. International Conference on Computer Vision, Vol. II, pp. 843–848.

    Google Scholar 

  • Schechner, Y.Y., Kiryati, N., and Basri, R. 1998. Separation of transparent layers using focus. In IEEE Proc. International Conference on Computer Vision, pp. 1061–1066.

  • Scherock, S. 1981. Depth from focus of structured light. In Technical Report-167, Media-Lab, MIT.

  • Schneider, G., Heit, B., Honig, J., and Bremont, J. 1994. Monocular depth perception by evaluation of the blur in defocused images.In Proc. International Conference on Image Processing, Vol. 2, pp. 116–119.

    Google Scholar 

  • Simoncelli, E.P. and Farid, H. 1996. Direct differential range estimation using optical masks. In European Conference on Computer Vision, Vol. II, pp. 82–93.

    Google Scholar 

  • Snyder, D.L., Schulz, T.J., and O'Sullivan, J.A. 1992. Deblurring subject to nonnegativity constraints. IEEE Transactions on Signal Processing, 40(5):1142–1150.

    Google Scholar 

  • Soatto, S. and Favaro, P. 2000. A geometric approach to blind deconvolution with application to shape from defocus. Proc. IEEE Computer Vision and Pattern Recognition, 2:10–17.

    Google Scholar 

  • Subbarao, M. and Surya, G. 1994. Depth from defocus: A spatial domain approach. International Journal of Computer Vision, 13(3):271–294.

    Google Scholar 

  • Subbarao, M. and Wei, T.C. 1992. Depth from defocus and rapid autofocusing: A practical approach. In Computer Vision and Pattern Recognition, pp. 773–776

  • Taylor, M. 1996. Partial Differential Equations (volume i: Basic Theory). Springer Verlag: Berlin.

    Google Scholar 

  • Watanabe, M. and Nayar, S.K. 1996a. Minimal operator set for passive depth from defocus. In Computer Vision and Pattern Recognition, pp. 431–438.

  • Watanabe, M. and Nayar, S.K. 1996b. Telecentric optics for computational vision. In European Conference on Computer Vision, Vol. II, pp. 439–445.

    Google Scholar 

  • Xiong, Y. and Shafer, S.A. 1995. Moment filters for high precision computation of focus and stereo. In Proc. of International Conference on Intelligent Robots and Systems, pp. 108–113.

  • Ziou, D. 1998. Passive depth from defocus using a spatial domain approach. In Proc. International Conference on Computer Vision, pp. 799–804.

Download references

Author information

Authors and Affiliations

  1. Electrical Engineering Department, Washington University in St. Louis, One Brookings Drive, St. Louis, MO, 63130

    Paolo Favaro

  2. Scuola Normale Superiore, Piazza Dei Cavalieri 7, Pisa, 56100, Italy

    Andrea Mennucci

  3. Computer Science Department, University of California, Los Angeles, 405 Hilgard Avenue, Los Angeles, CA, 90095

    Stefano Soatto

Authors
  1. Paolo Favaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrea Mennucci
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefano Soatto
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Favaro, P., Mennucci, A. & Soatto, S. Observing Shape from Defocused Images. International Journal of Computer Vision 52, 25–43 (2003). https://doi.org/10.1023/A:1022366408068

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1022366408068

Search

Navigation