Minhao Fan
ABSTRACT
Retinex model is widely adopted in various low-light image enhancement tasks. The basic idea of the Retinex theory is to decompose images into reflectance and illumination. The ill-posed decomposition is usually handled by hand-crafted constraints and priors. With the recently emerging deep-learning based approaches as tools, in this paper, we integrate the idea of Retinex decomposition and semantic information awareness. Based on the observation that various objects and backgrounds have different material, reflection and perspective attributes, regions of a single low-light image may require different adjustment and enhancement regarding contrast, illumination and noise. We propose an enhancement pipeline with three parts that effectively utilize the semantic layer information. Specifically, we extract the segmentation, reflectance as well as illumination layers, and concurrently enhance every separate region, i.e. sky, ground and objects for outdoor scenes. Extensive experiments on both synthetic data and real world images demonstrate the superiority of our method over current state-of-the-art low-light enhancement algorithms.
Supplemental Material
- M. Abdullah-Al-Wadud, M. H. Kabir, M. A. Akber Dewan, and O. Chae. 2007. A Dynamic Histogram Equalization for Image Contrast Enhancement. IEEE Trans. on Consumer Electronics, Vol. 53, 2 (May 2007), 593--600.Google Scholar
Digital Library
- T. Arici, S. Dikbas, and Y. Altunbasak. 2009. A histogram modification framework and its application for image contrast enhancement. IEEE Trans. on Image Processing, Vol. 18, 9 (2009), 1921--1935.Google ScholarDigital Library
- G. Brostow, J. Shotton, J. Fauqueur, and R. Cipolla. 2008. Segmentation and Recognition Using Structure from Motion Point Clouds. In Proc. IEEE European Conf. Computer Vision. 44--57.Google Scholar
- J. Cai, S. Gu, and L. Zhang. 2018. Learning a deep single image contrast enhancer from multi-exposure images. IEEE Trans. on Image Processing, Vol. 27, 4 (April 2018), 2049--2062.Google ScholarCross Ref
- J. Cai, W. Zuo, and L. Zhang. 2019. Extreme Channel Prior Embedded Network for Dynamic Scene Deblurring. arXiv, Vol. abs/1903.00763 (2019). arxiv: 1903.00763Google Scholar
- C. Chen, Q. Chen, J. Xu, and V. Koltun. 2018. Learning to See in the Dark. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition.Google Scholar
- M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. 2016. The Cityscapes Dataset for Semantic Urban Scene Understanding. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition.Google Scholar
- K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian. 2007. Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering. IEEE Trans. on Image Processing, Vol. 16, 8 (Aug 2007), 2080--2095.Google ScholarCross Ref
- X. Fu, Y. Sun, M. LiWang, Y. Huang, X. Zhang, and X. Ding. 2014. A novel retinex based approach for image enhancement with illumination adjustment. In Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing. 1190--1194.Google Scholar
- X. Fu, D. Zeng, Y. Huang, Y. Liao, X. Ding, and J. Paisley. 2016a. A fusion-based enhancing method for weakly illuminated images. Signal Processing, Vol. 129 (2016), 82 -- 96.Google ScholarDigital Library
- X. Fu, D. Zeng, Y. Huang, X. P. Zhang, and X. Ding. 2016b. A Weighted Variational Model for Simultaneous Reflectance and Illumination Estimation. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition. 2782--2790.Google Scholar
- X. Guo, Y. Li, and H. Ling. 2017. LIME: Low-Light Image Enhancement via Illumination Map Estimation. IEEE Trans. on Image Processing, Vol. 26, 2 (Feb 2017), 982--993.Google ScholarDigital Library
- H. Ibrahim and N. S. Pik Kong. 2007. Brightness Preserving Dynamic Histogram Equalization for Image Contrast Enhancement. IEEE Trans. on Consumer Electronics, Vol. 53, 4 (Nov 2007), 1752--1758.Google ScholarDigital Library
- Y. Jiang, X. Gong, D. Liu, Y. Cheng, C. Fang, X. Shen, J. Yang, P. Zhou, and Z. Wang. 2019. EnlightenGAN: Deep Light Enhancement without Paired Supervision. arXiv e-prints (June 2019), arXiv:1906.06972. arxiv: 1906.06972Google Scholar
- D. Jobson, Z. Rahman, and G. Woodell. 1997 a. A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans. on Image Processing, Vol. 6, 7 (Jul 1997), 965--976.Google ScholarDigital Library
- D. Jobson, Z. Rahman, and G. Woodell. 1997 b. Properties and performance of a center/surround retinex. IEEE Trans. on Image Processing, Vol. 6, 3 (Mar 1997), 451--462.Google ScholarDigital Library
- G. Kim, D. Kwon, and J. Kwon. 2019. Low-Lightgan: Low-Light Enhancement Via Advanced Generative Adversarial Network With Task-Driven Training. In Proc. IEEE Int'l Conf. Image Processing. 2811--2815.Google Scholar
- E. Land. 1977. The retinex theory of color vision. Sci. Amer (1977), 108--128.Google Scholar
- C. Lee, J. Kim, C. Lee, and C. Kim. 2014a. Optimized Brightness Compensation and Contrast Enhancement for Transmissive Liquid Crystal Displays. IEEE Trans. on Circuits and Systems for Video Technology, Vol. 24, 4 (April 2014), 576--590.Google Scholar
- C. Lee, C. Lee, and C. Kim. 2013a. Contrast Enhancement Based on Layered Difference Representation of 2D Histograms. IEEE Trans. on Image Processing, Vol. 22, 12 (Dec 2013), 5372--5384.Google ScholarDigital Library
- C. Lee, J. Shih, C. Lien, and C. Han. 2013b. Adaptive multiscale retinex for image contrast enhancement. In Signal-Image Technology & Internet-Based Systems (SITIS), 2013 International Conference on. IEEE, 43--50.Google Scholar
- J. Lee, C. Lee, J. Sim, and C. Kim. 2014b. Depth-guided adaptive contrast enhancement using 2D histograms. In Proc. IEEE Int'l Conf. Image Processing. 4527--4531.Google Scholar
- L. Li, R. Wang, W. Wang, and W. Gao. 2015. A low-light image enhancement method for both denoising and contrast enlarging. In Proc. IEEE Int'l Conf. Image Processing. 3730--3734.Google Scholar
- M. Li, J. Liu, W. Yang, X. Sun, and Z. Guo. 2018. Structure-Revealing Low-Light Image Enhancement Via Robust Retinex Model. IEEE Trans. on Image Processing, Vol. 27, 6 (June 2018), 2828--2841.Google ScholarCross Ref
- Y. Loh and C. Chan. 2019. Getting to know low-light images with the Exclusively Dark dataset. Computer Vision and Image Understanding, Vol. 178 (2019), 30 -- 42.Google ScholarCross Ref
- K. Lore, A. Akintayo, and S. Sarkar. 2017. LLNet: A deep autoencoder approach to natural low-light image enhancement. Pattern Recognition, Vol. 61 (2017), 650--662.Google ScholarDigital Library
- F. Lv, F. Lu, J. Wu, and C. Lim. 2018. MBLLEN: Low-light Image/Video Enhancement Using CNNs. In British Machine Vision Conference.Google Scholar
- B. Mildenhall, J. Barron, J. Chen, D. Sharlet, R. Ng, and R. Carroll. 2018. Burst Denoising With Kernel Prediction Networks. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition. 2502--2510.Google Scholar
- A. Mittal, R. Soundararajan, and A. Bovik. 2013. Making a "Completely Blind" Image Quality Analyzer. IEEE Signal Process. Lett., Vol. 20, 3 (2013), 209--212.Google ScholarCross Ref
- K. Nakai, Y. Hoshi, and A. Taguchi. 2013. Color image contrast enhacement method based on differential intensity/saturation gray-levels histograms. In International Symposium on Intelligent Signal Processing and Communications Systems. 445--449.Google Scholar
- S. Pizer, R. Johnston, J. Ericksen, B. Yankaskas, and K. Muller. 1990. Contrast-limited adaptive histogram equalization: speed and effectiveness. In Proc. of Conf. on Visualization in Biomedical Computing. 337--345.Google Scholar
- W. Ren, S. Liu, L. Ma, Q. Xu, X. Xu, X. Cao, J. Du, and M. Yang. 2019. Low-Light Image Enhancement via a Deep Hybrid Network. IEEE Trans. on Image Processing, Vol. 28, 9 (Sep. 2019), 4364--4375.Google ScholarCross Ref
- X. Ren, M. Li, W. Cheng, and J. Liu. 201 Joint Enhancement and Denoising Method via Sequential Decomposition. In IEEE Int'l Symposium on Circuits and Systems. 1--5.Google Scholar
- Y. Ren, Z. Ying, T. Li, and G. Li. 2019. LECARM: Low-Light Image Enhancement Using the Camera Response ModelIEEE Trans. Circuits Syst. Video Techn., Vol. 29, 4 (2019), 968--981.Google ScholarCross Ref
- L. Shen, Z. Yue, F. Feng, Q. Chen, S. Liu, and J. Ma. 2017. MSR-net:Low-light Image Enhancement Using Deep Convolutional Network. ArXiv e-prints (November 2017). arxiv: cs.CV/1711.02488Google Scholar
- L. Tao, C. Zhu, G. Xiang, Y. Li, H. Jia, and X. Xie. 2017. LLCNN: A convolutional neural network for low-light image enhancement. In Proc. IEEE Visual Communication and Image Processing. 1--4.Google Scholar
- L. Wang, L. Xiao, H. Liu, and Z. Wei. 2014. Variational Bayesian Method for Retinex. IEEE Trans. on Image Processing, Vol. 23, 8 (Aug 2014), 3381--3396.Google ScholarCross Ref
- R. Wang, Q. Zhang, C. Fu, X. Shen, W. Zheng, and J. Jia. 2019. Underexposed Photo Enhancement Using Deep Illumination Estimation. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition.Google Scholar
- S. Wang, J. Zheng, H. M. Hu, and B. Li. 2013. Naturalness Preserved Enhancement Algorithm for Non-Uniform Illumination Images. IEEE Trans. on Image Processing, Vol. 22, 9 (Sept 2013), 3538--3548.Google ScholarDigital Library
- W. Wang, C. Wei, W. Yang, and J. Liu. 2018. GLADNet: Low-Light Enhancement Network with Global Awareness. In Automatic Face & Gesture Recognition (FG 2018), 2018 13th IEEE International Conference. IEEE, 751--755.Google Scholar
- Y. Wang, Y. Cao, Z. Zha, J. Zhang, Z. Xiong, W. Zhang, and F. Wu. 2019. Progressive Retinex: Mutually Reinforced Illumination-Noise Perception Network for Low Light Image Enhancement. In ACM Trans. Multimedia.Google Scholar
- Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Trans. on Image Processing, Vol. 13, 4 (April 2004), 600--612.Google ScholarDigital Library
- C. Wei, W. Wang, W. Yang, and J. Liu. 2018. Deep Retinex Decomposition for Low-Light Enhancement. In British Machine Vision Conference.Google Scholar
- X. Wu, X. Liu, K. Hiramatsu, and K. Kashino. 2017. Contrast-accumulated histogram equalization for image enhancement. In Proc. IEEE Int'l Conf. Image Processing. 3190--3194.Google Scholar
- W. Yang, S. Wang, Y. Fang, Y. Wu, and J. Liu. 2020. From Fidelity to Perceptual Quality: A Semi-Supervised Approach for Low-Light Image Enhancement. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition.Google Scholar
- Z. Ying, G. Li, and W. Gao. 2017. A Bio-Inspired Multi-Exposure Fusion Framework for Low-light Image Enhancement. ArXiv e-prints (November 2017). arxiv: cs.CV/1711.00591Google Scholar
- Z. Ying, G. Li, Y. Ren, R. Wang, and W. Wang. 2017. A New Image Contrast Enhancement Algorithm Using Exposure Fusion Framework. In Proc. Int'l Conf. Computer Analysis of Images and Patterns. 36--46.Google Scholar
- Y. Yuan, W. Yang, W. Ren, J. Liu, W. Scheirer, and Z. Wang. 2019. UG2+ Track 2: A Collective Benchmark Effort for Evaluating and Advancing Image Understanding in Poor Visibility Environments. arXiv e-prints (Apr 2019), arXiv:1904.04474.Google Scholar
- X. Zhang, P. Shen, L. Luo, L. Zhang, and J. Song. 2012. Enhancement and noise reduction of very low light level images. In Proc. IEEE Int'l Conf. Pattern Recognition. 2034--2037.Google Scholar
- Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu. 2018. Residual Dense Network for Image Super-Resolution. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition.Google Scholar
- Y. Zhang, J. Zhang, and X. Guo. 2019. Kindling the Darkness: A Practical Low-light Image Enhancer. In Proc. ACM Int'l Conf. Multimedia.Google Scholar
Index Terms
- Integrating Semantic Segmentation and Retinex Model for Low-Light Image Enhancement
Recommendations
Low-light image enhancement using inverted image normalized by atmospheric light
Highlights- This paper presents a novel low-light image enhancement algorithm using the inverted image normalized by atmospheric light.
- A mathematical connection between the Retinex theory and the atmospheric scattering model by normalizing an ...
AbstractImages captured under low-light conditions are characterized by problems, including low brightness, low contrast, and color distortion. Low-light image enhancement, aims to enhance visual effects for the benefit of subsequent image processing and ...
A Low-light Image Enhancement Method Balancing Brightness Boosting and Noise Smoothing
CSAI '22: Proceedings of the 2022 6th International Conference on Computer Science and Artificial IntelligenceImages captured in low-light or uneven lighting environments need to use low-light image enhancement techniques to boost brightness and contrast for high-quality improved results. Some existing low-light image enhancement methods over-enhance the image ...
Semi-supervised Zero-Shot Learning in Retinex decomposition-based Low-Light Image Enhancement
ICCAI '23: Proceedings of the 2023 9th International Conference on Computing and Artificial IntelligenceThe images recorded by people are often subject to multiple degradation challenges due to the shooting environment and shooting equipment. In low-light scenes, images acquired by non-professional equipment are usually accompanied by a large amount of ...
Comments