Abstract
Reflection removal from photographs is an important task in computational photography, but also for computer vision tasks that involve imaging through windows and similar settings. Traditionally, the problem is approached as a single reflection removal problem under very controlled scenarios. In this paper we aim to generalize the reflection removal to real-world scenarios with more complicated light interactions. To this end, we propose a simple yet efficient learning framework for supervised image reflection separation with a polarization-guided ray-tracing model and loss function design. Instead of a conventional image sensor, we use a polarization sensor that instantaneously captures four linearly polarized photos of the scene in the same image. Through a combination of a new polarization-guided image formation model and a novel supervised learning framework for the interpretation of a ray-tracing image formation model, a general method is obtained to tackle general image reflection removal problems. We demonstrate our method with extensive experiments on both real and synthetic data and demonstrate the unprecedented quality of image reconstructions.
S. Qiu—Jointly first author.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Fan, Q., Yang, J., Hua, G., Chen, B., Wipf, D.: A generic deep architecture for single image reflection removal and image smoothing. In: CVPR (2017)
Fan, Q., Yang, J., Wipf, D., Chen, B., Tong, X.: Image smoothing via unsupervised learning. TOG 37, 1–4 (2019)
Gandelsman, Y., Shocher, A., Irani, M.: Double-dip: unsupervised image decomposition via coupled deep-image-priors. In: CVPR (2019)
Zhang, X., Ng, R., Chen, Q.: Single image reflection separation with perceptual losses. In: CVPR (2018)
Levin, A., Weiss, Y.: User assisted separation of reflections from a single image using a sparsity prior. PAMI 29, 1647–1654 (2007)
Li, Y., Brown, M.S.: Exploiting reflection change for automatic reflection removal. In: ICCV (2013)
Levin, A., Zomet, A., Weiss, Y.: Learning to perceive transparency from the statistics of natural scenes. In: NeurIPS (2003)
Levin, A., Zomet, A., Weiss, Y.: Separating reflections from a single image using local features. In: CVPR (2004)
Li, Y., Brown, M.S.: Single image layer separation using relative smoothness. In: CVPR (2014)
Shih, Y., Krishnan, D., Durand, F., Freeman, W.T.: Reflection removal using ghosting cues. In: CVPR (2015)
Arvanitopoulos, N., Achanta, R., Susstrunk, S.: Single image reflection suppression. In: CVPR (2017)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR (2015)
Xu, L., Lu, C., Xu, Y., Jia, J.: Image smoothing via L0 gradient minimization. TOG 30, 11 (2011)
Wan, R., Shi, B., Hwee, T.A., Kot, A.C.: Depth of field guided reflection removal. In: ICIP(2016)
Yang, J., Gong, D., Liu, L., Shi, Q.: Seeing deeply and bidirectionally: a deep learning approach for single image reflection removal. In: ECCV (2018)
Jin, M., Süsstrunk, S., Favaro, P.: Learning to see through reflections. In: ICCP (2018)
Wan, R., Shi, B., Duan, L.Y., Tan, A.H., Kot, A.C.: CRRN: multi-scale guided concurrent reflection removal network. In: CVPR (2018)
Wen, Q., Tan, Y., Qin, J., Liu, W., Han, G., He, S.: Single image reflection removal beyond linearity. In: CVPR (2019)
Wei, K., Yang, J., Fu, Y., David, W., Huang, H.: Single image reflection removal exploiting misaligned training data and network enhancements. In: CVPR (2019)
Xue, T., Rubinstein, M., Liu, C., Freeman, W.T.: A computational approach for obstruction-free photography. TOG 34, 1–11 (2015)
Guo, X., Cao, X., Ma, Y.: Robust separation of reflection from multiple images. In: CVPR (2014)
Han, B.J., Sim, J.Y.: Reflection removal using low-rank matrix completion. In: CVPR (2017)
Lyu, Y., Cui, Z., Li, S., Pollefeys, M., Shi, B.: Reflection separation using a pair of unpolarized and polarized images. In: NeurIPS (2019)
Wieschollek, P., Gallo, O., Gu, J., Kautz, J.: Separating reflection and transmission images in the wild. In: ECCV (2018)
Kong, N., Tai, Y., Shin, J.S.: A physically-based approach to reflection separation: from physical modeling to constrained optimization. PAMI 36, 209–221 (2014)
Farid, H., Adelson, E.H.: Separating reflections and lighting using independent components analysis. In: CVPR (1999)
Miyazaki, D., Ikeuchi, K.: Inverse polarization raytracing: estimating surface shapes of transparent objects. In: CVPR (2005)
Qiu, S., Fu, Q., Wang, C., Heidrich, W.: Polarization demosaicking for monochrome and color polarization focal plane arrays. In: Vision, Modeling and Visualization (2019)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, R., Qiu, S., Zang, G., Heidrich, W. (2020). Reflection Separation via Multi-bounce Polarization State Tracing. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12358. Springer, Cham. https://doi.org/10.1007/978-3-030-58601-0_46
Download citation
DOI: https://doi.org/10.1007/978-3-030-58601-0_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58600-3
Online ISBN: 978-3-030-58601-0
eBook Packages: Computer ScienceComputer Science (R0)