Abstract
This paper proposes an adaptive color-guided auto-regressive (AR) model for high quality depth recovery from low quality measurements captured by depth cameras. We formulate the depth recovery task into a minimization of AR prediction errors subject to measurement consistency. The AR predictor for each pixel is constructed according to both the local correlation in the initial depth map and the nonlocal similarity in the accompanied high quality color image. Experimental results show that our method outperforms existing state-of-the-art schemes, and is versatile for both mainstream depth sensors: ToF camera and Kinect.
Chapter PDF
Similar content being viewed by others
References
Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV 47(1), 7–42 (2002)
Prasad, T., Hartmann, K., Weihs, W., Ghobadi, S., Sluiter, A.: First steps in enhancing 3D vision technique using 2D/3D sensors. In: Computer Vision Winter Workshop (2006)
Linarth, A., Penne, J., Liu, B., Jesorsky, O., Kompe, R.: Fast fusion of range and video sensor data. In: Advanced Microsystems for Automotive Applications, pp. 119–134. Springer (2007)
Lindner, M., Kolb, A., Hartmann, K.: Data-fusion of PMD-based distance-information and high-resolution RGB-images. In: Int. Symp. on Sig., Circ. & Syst., vol. 1, pp. 1–4. IEEE (2007)
Guomundsson, S., Larsen, R., Aanaes, H., Pardas, M., Casas, J.: ToF imaging in smart room environments towards improved people tracking. In: CVPR Workshops (2008)
Diebel, J., Thrun, S.: An application of markov random fields to range sensing. Advances in Neural Information Processing Systems 18, 291 (2006)
Lu, J., Min, D., Pahwa, R., Do, M.: A revisit to MRF-based depth map super-resolution and enhancement. In: ICASSP (2011)
Zhu, J., Wang, L., Gao, J., Yang, R.: Spatial-temporal fusion for high accuracy depth maps using dynamic MRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(5), 899–909 (2010)
Yang, Q., Yang, R., Davis, J., Nister, D.: Spatial-depth super resolution for range images. In: CVPR (2007)
Dolson, J., Baek, J., Plagemann, C., Thrun, S.: Upsampling range data in dynamic environments. In: CVPR (2010)
Huhle, B., Schairer, T., Jenke, P., Straßer, W.: Fusion of range and color images for denoising and resolution enhancement with a non-local filter. CVIU 114(12), 1336–1345 (2010)
Min, D., Lu, J., Do, M.: Depth video enhancement based on joint global mode filtering. IEEE TIP 21(3), 1176–1190 (2011)
Huhle, B., Schairer, T., Jenke, P., Straßer, W.: Robust non-local denoising of colored depth data. In: CVPR Workshops (2008)
Park, J., Kim, H., Tai, Y., Brown, M., Kweon, I.: High quality depth map upsampling for 3D-ToF cameras. In: ICCV (2011)
Lai, K., Bo, L., Ren, X., Fox, D.: A large-scale hierarchical multi-view RGB-D object dataset. In: ICRA (2011)
Matyunin, S., Vatolin, D., Berdnikov, Y., Smirnov, M.: Temporal filtering for depth maps generated by Kinect depth camera. In: 3DTV Conference (2011)
Camplani, M., Salgado, L., Grupo de Tratamiento de Imágenes, E.T.S.I de Telecomunicación: Efficient spatio-temporal hole filling strategy for Kinect depth maps. In: SPIE (2012)
Berdnikov, Y., Vatolin, D.: Real-time depth map occlusion filling and scene background restoration for projected-pattern based depth cameras. In: Graphic Conf., IETP (2011)
Kolb, A., Barth, E., Koch, R., Larsen, R.: Time-of-flight cameras in computer graphics. In: Computer Graphics Forum, vol. 29, pp. 141–159. Wiley Online Library (2010)
Zhang, X., Wu, X.: Image interpolation by adaptive 2-D autoregressive modeling and soft-decision estimation. IEEE TIP 17(6), 887–896 (2008)
Zhang, Y., Zhao, D., Ji, X., Wang, R., Gao, W.: A spatio-temporal auto regressive model for frame rate upconversion. IEEE TCSVT 19(9), 1289–1301 (2009)
Middlebury datasets, http://vision.middlebury.edu/stereo/data/
He, K., Sun, J., Tang, X.: Guided Image Filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 1–14. Springer, Heidelberg (2010)
Open source computer vision (OpenCV), http://opencv.org
Paliy, D., Foi, A., Bilcu, R., Katkovnik, V.: Denoising and interpolation of noisy bayer data with adaptive cross-color filters. In: Proc. of SPIE VCIP (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yang, J., Ye, X., Li, K., Hou, C. (2012). Depth Recovery Using an Adaptive Color-Guided Auto-Regressive Model. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds) Computer Vision – ECCV 2012. ECCV 2012. Lecture Notes in Computer Science, vol 7576. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33715-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-33715-4_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33714-7
Online ISBN: 978-3-642-33715-4
eBook Packages: Computer ScienceComputer Science (R0)