Abstract
Specular reflections present difficulties for many areas of computer vision such as stereo and segmentation. To separate specular and diffuse reflection components, previous approaches generally require accurate segmentation, regionally uniform reflectance or structured lighting. To overcome these limiting assumptions, we propose a method based on color analysis and multibaseline stereo that simultaneously estimates the separation and the true depth of specular reflections. First, pixels with a specular component are detected by a novel form of color histogram differencing that utilizes the epipolar constraint. This process uses relevant data from all the stereo images for robustness, and addresses the problem of color occlusions. Based on the Lambertian model of diffuse reflectance, stereo correspondence is then employed to compute for specular pixels their corresponding diffuse components in other views. The results of color-based detection aid the stereo correspondence, which determines both separation and true depth of specular pixels. Our approach integrates color analysis and multibaseline stereo in a synergistic manner to yield accurate separation and depth, as demonstrated by our results on synthetic and real image sequences.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This work was performed while the second author was visiting Microsoft Research, Asia.
References
D.N. Bhat and S.K. Nayar. Binocular stereo in the presence of specular reflection. In ARPA, pages II:1305–1315, 1994.
D.N. Bhat and S.K. Nayar. Stereo in the presence of specular reflection. In ICCV, pages 1086–1092, 1995.
A.F. Bobick and S.S. Intille. Large occlusion stereo. IJCV, 33(3):1–20, Sept. 1999.
P.E. Debevec and J. Malik. Recovering high dynamic range radiance maps from photographs. Computer Graphics (SIGGRAPH), 31:369–378, 1997.
H. Jin, A. Yezzi, and S. Soatto. Variational multiframe stereo in the presence of specular reflections. Technical Report TR01-0017, UCLA, 2001.
S.B. Kang, R. Szeliski, and J. Chai. Handling occlusions in dense multi-view stereo. In CVPR, pages 103–110, Dec. 2001.
G.J. Klinker, S.A. Shafer, and T. Kanade. A physical approach to color image understanding. IJCV, 4(1):7–38, Jan. 1990.
L. Lam and C.Y. Suen. Application of majority voting to pattern recognition: An analysis of its behaviour and performance. IEEE Trans. on Systems, Man, and Cyberbetics, 27(5):553–568, 1997.
S.W. Lee and R. Bajcsy. Detection of specularity using color and multiple views. Image and Vision Computing, 10:643–653, 1992.
D.C. Marr and T. Poggio. A computational theory of human stereo vision. In Lucia M. Vaina, editor, From the Retina to the Neocortex: Selected Papers of David Marr, pages 263–290. Birkhäuser, Boston, MA, 1991.
Y. Nakamura, T. Matsura, K. Satoh, and Y. Ohta. Occlusion detectable stereo-occlusion patterns in camera matrix. In CVPR, pages 371–378, 1996.
S.K. Nayar, X. Fang, and T.E. Boult. Removal of specularities using color and polarization. In CVPR, pages 583–590, 1993.
M. Okutomi and T. Kanade. A multiple baseline stereo. IEEE PAMI, 15, 1993.
Y. Sato and K. Ikeuchi. Temporal-color space analysis of reflection. J. of the Opt. Soc. of America A, 11, 1994.
S. Shafer. Using color to separate reflection components. Color Research and Applications, 10, 1985.
F. Tong and B. V. Funt. Specularity removal for shape from shading. In Proc. Vision Interface, pages 98–103, 1988.
L.B. Wolff and T.E. Boult. Constraining object features using a polarization rflectance model. IEEE PAMI, 13, 1991.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lin, S., Li, Y., Kang, S.B., Tong, X., Shum, HY. (2002). Diffuse-Specular Separation and Depth Recovery from Image Sequences. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds) Computer Vision — ECCV 2002. ECCV 2002. Lecture Notes in Computer Science, vol 2352. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-47977-5_14
Download citation
DOI: https://doi.org/10.1007/3-540-47977-5_14
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43746-8
Online ISBN: 978-3-540-47977-2
eBook Packages: Springer Book Archive