Srimanta Mandal
ABSTRACT
In practice, images can contain different amounts of noise for different color channels, which is not acknowledged by existing super-resolution approaches. In this paper, we propose to super-resolve noisy color images by considering the color channels jointly. Noise statistics are blindly estimated from the input low-resolution image and are used to assign different weights to different color channels in the data cost. Implicit low-rank structure of visual data is enforced via nuclear norm minimization in association with adaptive weights, which is added as a regularization term to the cost. Additionally, multi-scale details of the image are added to the model through another regularization term that involves projection onto PCA basis, which is constructed using similar patches extracted across different scales of the input image. The results demonstrate the super-resolving capability of the approach in real scenarios.
- Abdelrahman Abdelhamed, Stephen Lin, and Michael S. Brown. 2018. A High-Quality Denoising Dataset for Smartphone Cameras. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1692--1700.Google Scholar
- M. Aharon, M. Elad, and A. Bruckstein. Nov. 2006. K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation. IEEE Transactions on Signal Processing 54, 11 (Nov. 2006), 4311--4322.Google ScholarDigital Library
- Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. 2011. Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Found. Trends Mach. Learn. 3, 1 (Jan. 2011), 1--122. https://doi.org/10.1561/2200000016Google ScholarDigital Library
- G. Chen, F. Zhu, and P. A. Heng. 2015. An Efficient Statistical Method for Image Noise Level Estimation. In IEEE International Conference on Computer Vision (ICCV). 477--485. https://doi.org/10.1109/ICCV.2015.62Google ScholarDigital Library
- K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian. 2007. Color Image Denoising via Sparse 3D Collaborative Filtering with Grouping Constraint in Luminance-Chrominance Space. In IEEE International Conference on Image Processing (ICIP), Vol. 1. I-313--I-316. https://doi.org/10.1109/ICIP.2007.4378954Google Scholar
- Chao Dong, ChenChange Loy, Kaiming He, and Xiaoou Tang. 2014. Learning a Deep Convolutional Network for Image Super-Resolution. In Computer Vision -ECCV 2014. Lecture Notes in Computer Science, Vol. 8692. Springer International Publishing, 184--199. https://doi.org/10.1007/978-3-319-10593-2_13Google ScholarCross Ref
- Chao Dong, Chen Change Loy, and Xiaoou Tang. 2016. Accelerating the Super-Resolution Convolutional Neural Network. In Computer Vision - ECCV 2016, Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (Eds.). Springer International Publishing, Cham, 391--407.Google Scholar
- W. Dong, L. Zhang, G. Shi, and X. Li. 2013. Nonlocally Centralized Sparse Representation for Image Restoration. IEEE Transactions on Image Processing 22, 4 (April 2013), 1620--1630. https://doi.org/10.1109/TIP.2012.2235847Google ScholarDigital Library
- Weisheng Dong, Lei Zhang, Guangming Shi, and Xiaolin Wu. 2011. Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization. IEEE Transactions on Image Processing 20, 7 (Jul. 2011), 1838--1857. https://doi.org/10.1109/TIP.2011.2108306Google ScholarDigital Library
- W.T. Freeman, T.R. Jones, and E.C. Pasztor. 2002. Example-based super-resolution. IEEE, Computer Graphics and Applications 22, 2 (mar/apr 2002), 56--65. https://doi.org/10.1109/38.988747Google ScholarDigital Library
- D. Glasner, S. Bagon, and M. Irani. 2009. Super-resolution from a single image. In IEEE International Conference on Computer Vision (ICCV). 349--356. https://doi.org/10.1109/ICCV.2009.5459271Google Scholar
- Shuhang Gu, Qi Xie, Deyu Meng, Wangmeng Zuo, Xiangchu Feng, and Lei Zhang. 2017. Weighted Nuclear Norm Minimization and Its Applications to Low Level Vision. International Journal of Computer Vision 121, 2 (01 Jan 2017), 183--208. https://doi.org/10.1007/s11263-016-0930-5Google ScholarDigital Library
- K. He, X. Zhang, S. Ren, and J. Sun. 2016. Deep Residual Learning for Image Recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 770--778. https://doi.org/10.1109/CVPR.2016.90Google Scholar
- J. B. Huang, A. Singh, and N. Ahuja. 2015. Single image super-resolution from transformed self-exemplars. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5197--5206. https://doi.org/10.1109/CVPR.2015.7299156Google ScholarCross Ref
- G. Jeon and E. Dubois. 2013. Demosaicking of Noisy Bayer-Sampled Color Images With Least-Squares Luma-Chroma Demultiplexing and Noise Level Estimation. IEEE Transactions on Image Processing 22, 1 (Jan 2013), 146--156. https://doi.org/10.1109/TIP.2012.2214041Google ScholarDigital Library
- Atsunori Kanemura, Shin ichi Maeda, and Shin Ishii. 2009. Superresolution with compound Markov random fields via the variational {EM} algorithm. Neural Networks 22, 7 (2009), 1025--1034. https://doi.org/10.1016/j.neunet.2008.12.005Google ScholarDigital Library
- Hakki Can Karaimer and Michael S. Brown. 2016. A Software Platform for Manipulating the Camera Imaging Pipeline. In Computer Vision - ECCV, Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (Eds.). Springer International Publishing, Cham, 429--444.Google Scholar
- J. Kim, J. K. Lee, and K. M. Lee. 2016. Accurate Image Super-Resolution Using Very Deep Convolutional Networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1646--1654. https://doi.org/10.1109/CVPR.2016.182Google Scholar
- J. Kim, J. K. Lee, and K. M. Lee. 2016. Deeply-Recursive Convolutional Network for Image Super-Resolution. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1637--1645. https://doi.org/10.1109/CVPR.2016.181Google Scholar
- Marc Lebrun, Miguel Colom, and Jean-Michel Morel. 2015. The Noise Clinic: a Blind Image Denoising Algorithm. Image Processing On Line 5 (2015), 1--54. https://doi.org/10.5201/ipol.2015.125Google ScholarCross Ref
- C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi. 2017. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 105--114. https://doi.org/10.1109/CVPR.2017.19Google Scholar
- B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee. 2017. Enhanced Deep Residual Networks for Single Image Super-Resolution. In IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 1132--1140. https://doi.org/10.1109/CVPRW.2017.151Google Scholar
- C. Liu, R. Szeliski, S. Bing Kang, C. L. Zitnick, and W. T. Freeman. 2008. Automatic Estimation and Removal of Noise from a Single Image. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 2 (Feb 2008), 299--314. https://doi.org/10.1109/TPAMI.2007.1176Google ScholarDigital Library
- J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman. 2009. Non-local sparse models for image restoration. In IEEE 12th International Conference on Computer Vision. 2272--2279. https://doi.org/10.1109/ICCV.2009.5459452Google Scholar
- S. Mandal, A. Bhavsar, and A.K. Sao. 2014. Super-resolving a Single Intensity/Range Image via Non-local Means and Sparse Representation. In Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), 2014. 1--8. https://doi.org/10.1145/2683483.2683541Google ScholarDigital Library
- S. Mandal, A. Bhavsar, and A. K. Sao. 2017. Depth Map Restoration From Undersampled Data. IEEE Transactions on Image Processing 26, 1 (Jan 2017), 119--134. https://doi.org/10.1109/TIP.2016.2621410Google ScholarDigital Library
- Srimanta Mandal, Arnav Bhavsar, and Anil Kumar Sao. 2017. Noise adaptive super-resolution from single image via non-local mean and sparse representation. Signal Processing 132 (2017), 134--149. https://doi.org/10.1016/j.sigpro.2016.09.017Google ScholarCross Ref
- Srimanta Mandal and A. N. Rajagopalan. 2018. Single Noisy Image Super Resolution by Minimizing Nuclear Norm in Virtual Sparse Domain. In Computer Vision, Pattern Recognition, Image Processing, and Graphics, Renu Rameshan, Chetan Arora, and Sumantra Dutta Roy (Eds.). Springer Singapore, Singapore, 163--176.Google Scholar
- Srimanta Mandal and Anil Kumar Sao. 2016. Employing structural and statistical information to learn dictionary(s) for single image super-resolution in sparse domain. Signal Processing: Image Communication 48 (2016), 63--80. https://doi.org/10.1016/j.image.2016.08.006Google ScholarCross Ref
- Xiao-Jiao Mao, Chunhua Shen, and Yu-Bin Yang. 2016. Image Restoration Using Very Deep Convolutional Encoder-decoder Networks with Symmetric Skip Connections. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS'16). Curran Associates Inc., USA, 2810--2818. http://dl.acm.org/citation.cfm?id=3157382.3157412Google ScholarDigital Library
- Antonio Marquina and Stanley J. Osher. 2008. Image Super-Resolution by TV-Regularization and Bregman Iteration. Journal of Scientific Computing 37 (2008), 367--382. Issue 3. https://doi.org/10.1007/s10915-008-9214-8Google ScholarDigital Library
- S. Nam, Y. Hwang, Y. Matsushita, and S. J. Kim. 2016. A Holistic Approach to Cross-Channel Image Noise Modeling and Its Application to Image Denoising. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1683--1691. https://doi.org/10.1109/CVPR.2016.186Google Scholar
- Sung Cheol Park, Min Kyu Park, and Moon Gi Kang. 2003. Super-resolution image reconstruction: a technical overview. IEEE, Signal Processing Magazine 20, 3 (may 2003), 21--36. https://doi.org/10.1109/MSP.2003.1203207Google Scholar
- T. Peleg and M. Elad. 2014. A Statistical Prediction Model Based on Sparse Representations for Single Image Super-Resolution. IEEE Transactions on Image Processing 23, 6 (June 2014), 2569--2582. https://doi.org/10.1109/TIP.2014.2305844Google ScholarDigital Library
- T. Plötz and S. Roth. 2017. Benchmarking Denoising Algorithms with Real Photographs. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2750--2759. https://doi.org/10.1109/CVPR.2017.294Google Scholar
- R. Rubinstein, A.M. Bruckstein, and M. Elad. 2010. Dictionaries for Sparse Representation Modeling. Proc. IEEE 98, 6 (june 2010), 1045--1057. https://doi.org/10.1109/JPROC.2010.2040551Google ScholarCross Ref
- W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang. 2016. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1874--1883. https://doi.org/10.1109/CVPR.2016.207Google Scholar
- Assaf Shocher, Nadav Cohen, and Michal Irani. 2018. "Zero-Shot" Super-Resolution Using Deep Internal Learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 3118--3126.Google ScholarCross Ref
- A. Singh, F. Porikli, and N. Ahuja. 2014. Super-resolving Noisy Images. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2846--2853. https://doi.org/10.1109/CVPR.2014.364Google ScholarDigital Library
- Henry Stark and Peyma Oskoui. 1989. High-resolution image recovery from image-plane arrays, using convex projections. J. Opt. Soc. Am. A 6, 11 (Nov. 1989), 1715--1726.Google ScholarCross Ref
- Radu Timofte, Vincent De Smet, and Luc Van Gool. 2015. A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution. In Computer Vision - ACCV 2014. Lecture Notes in Computer Science, Vol. 9006. Springer International Publishing, 111--126. https://doi.org/10.1007/978-3-319-16817-3_8Google Scholar
- S. Vishnukumar, Madhu S. Nair, and M. Wilscy. 2014. Edge preserving single image super-resolution with improved visual quality. Signal Processing 105, 0 (2014), 283--297. https://doi.org/10.1016/j.sigpro.2014.05.033Google ScholarCross Ref
- J. Xu, L. Zhang, D. Zhang, and X. Feng. 2017. Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising. In IEEE International Conference on Computer Vision (ICCV). 1105--1113. https://doi.org/10.1109/ICCV.2017.125Google Scholar
- Chih-Yuan Yang, Jia-Bin Huang, and Ming-Hsuan Yang. 2011. Exploiting Self-similarities for Single Frame Super-Resolution. In Computer Vision - ACCV 2010, Ron Kimmel, Reinhard Klette, and Akihiro Sugimoto (Eds.). Lecture Notes in Computer Science, Vol. 6494. Springer Berlin Heidelberg, 497--510. https://doi.org/10.1007/978-3-642-19318-7_39Google Scholar
- Jianchao Yang, J. Wright, T. Huang, and Yi Ma. Jun. 2008. Image super-resolution as sparse representation of raw image patches. In IEEE Conference on Computer Vision and Pattern Recognition. 1--8. https://doi.org/10.1109/CVPR.2008.4587647Google Scholar
- Jianchao Yang, J. Wright, T.S. Huang, and Yi Ma. Nov. 2010. Image Super-Resolution Via Sparse Representation. IEEE Transactions on Image Processing 19, 11 (Nov. 2010), 2861--2873. https://doi.org/10.1109/TIP.2010.2050625Google ScholarDigital Library
- Roman Zeyde, Michael Elad, and Matan Protter. 2012. On Single Image Scale-Up Using Sparse-Representations. In Curves and Surfaces. Vol. 6920. Springer, 711--730. https://doi.org/10.1007/978-3-642-27413-8_47Google ScholarDigital Library
- Xin Zhang, Edmund Y. Lam, EdX. Wu, and Kenneth K.Y. Wong. 2008. Application of Tikhonov Regularization to Super-Resolution Reconstruction of Brain MRI Images. In Medical Imaging and Informatics, Xiaohong Gao, Henning Müller, MartinJ. Loomes, Richard Comley, and Shuqian Luo (Eds.). Lecture Notes in Computer Science, Vol. 4987. Springer Berlin Heidelberg, 51--56. https://doi.org/10.1007/978-3-540-79490-5_8Google ScholarDigital Library
Index Terms
- Color Image Super Resolution in Real Noise
Recommendations
Single image super-resolution based on nonlocal sparse and low-rank regularization
PRICAI'16: Proceedings of the 14th Pacific Rim International Conference on Trends in Artificial IntelligenceImage super resolution (SR) is an active research topic to obtain an high resolution (HR) image from the low resolution (LR) observation. Many results of existing methods may be corrupted by some artifacts. In this paper, we propose an SR reconstruction ...
Colorization for single image super resolution
ECCV'10: Proceedings of the 11th European conference on Computer vision: Part VIThis paper introduces a new procedure to handle color in single image super resolution (SR). Most existing SR techniques focus primarily on enforcing image priors or synthesizing image details; less attention is paid to the final color assignment. As a ...
Real Color Image Enhanced by Illumination - Reflectance Model and Wavelet Transformation
ITCS '09: Proceedings of the 2009 International Conference on Information Technology and Computer Science - Volume 01Because the greatly most algorithms of real color image enhancement did not relate to the spectral sensitivity of most human vision, the color restoration functions of some real color image enhancement algorithms are greatly at random and not proved , ...
Comments