Zeev Farbman
Dani Lischinski
Richard Szeliski
ABSTRACT
Many recent computational photography techniques decompose an image into a piecewise smooth base layer, containing large scale variations in intensity, and a residual detail layer capturing the smaller scale details in the image. In many of these applications, it is important to control the spatial scale of the extracted details, and it is often desirable to manipulate details at multiple scales, while avoiding visual artifacts.
In this paper we introduce a new way to construct edge-preserving multi-scale image decompositions. We show that current basedetail decomposition techniques, based on the bilateral filter, are limited in their ability to extract detail at arbitrary scales. Instead, we advocate the use of an alternative edge-preserving smoothing operator, based on the weighted least squares optimization framework, which is particularly well suited for progressive coarsening of images and for multi-scale detail extraction. After describing this operator, we show how to use it to construct edge-preserving multi-scale decompositions, and compare it to the bilateral filter, as well as to other schemes. Finally, we demonstrate the effectiveness of our edge-preserving decompositions in the context of LDR and HDR tone mapping, detail enhancement, and other applications.
Supplemental Material
- Alvarez, L., Lions, P.-L., and Morel, J.-M. 1992. Image selective smoothing and edge detection by nonlinear diffusion. ii. SIAM Journal on Numerical Analysis 29, 3 (June), 845--866. Google Scholar
Digital Library
- Aubert, G., and Kornprobst, P. 2006. Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, vol. 147 of Applied Mathematical Sciences. Springer. Google ScholarDigital Library
- Bae, S., Paris, S., and Durand, F. 2006. Two-scale tone management for photographic look. ACM Trans. Graph. 25, 3 (July), 654--662. Google ScholarDigital Library
- Barash, D. 2002. A fundamental relationship between bilateral filtering, adaptive smoothing, and the nonlinear diffusion equation. IEEE Trans. Pattern Anal. Mach. Intell. 24, 6, 844--847. Google ScholarDigital Library
- Black, M. J., Sapiro, G., Marimont, D. H., and Heeger, D. 1998. Robust anisotropic diffusion. IEEE Trans. Image Proc. 7, 3 (Mar.), 421--432. Google ScholarDigital Library
- Buades, A., Coll, B., and Morel, J. M. 2006. The staircasing effect in neighborhood filters and its solution. IEEE Transactions on Image Processing 15, 6, 1499--1505. Google ScholarDigital Library
- Buatois, L., Caumon, G., and Levy, B. 2007. Concurrent number cruncher: An efficient sparse linear solver on the GPU. In High Performance Computation Conference (HPCC), Springer. Google ScholarDigital Library
- Burt, P., and Adelson, E. H. 1983. The Laplacian pyramid as a compact image code. IEEE Trans. Comm. 31, 532--540.Google ScholarCross Ref
- Chen, J., Paris, S., and Durand, F. 2007. Real-time edge-aware image processing with the bilateral grid. ACM Trans. Graph. 26, 3 (July), Article 103. Google ScholarDigital Library
- Choudhury, P., and Tumblin, J. 2003. The trilateral filter for high contrast images and meshes. In Proc. EGSR 2003, Eurographics, 186--196. Google ScholarDigital Library
- Comaniciu, D., and Meer, P. 2002. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 5, 603--619. Google ScholarDigital Library
- DeCarlo, D., and Santella, A. 2002. Stylization and abstraction of photographs. ACM Trans. Graph. 21, 3 (July), 769--776. Google ScholarDigital Library
- Durand, F., and Dorsey, J. 2002. Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans. Graph. 21, 3 (July), 257--266. Google ScholarDigital Library
- Eisemann, E., and Durand, F. 2004. Flash photography enhancement via intrinsic relighting. ACM Trans. Graph. 23, 3 (August), 673--678. Google ScholarDigital Library
- Elad, M. 2002. On the bilateral filter and ways to improve it. IEEE Trans. Image Proc. 11, 10, 1141--1151. Google ScholarDigital Library
- Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. ACM Trans. Graph. 21, 3 (July), 249--256. Google ScholarDigital Library
- Fattal, R., Agrawala, M., and Rusinkiewicz, S. 2007. Multiscale shape and detail enhancement from multi-light image collections. ACM Trans. Graph. 26, 3 (July), Article 51. Google ScholarDigital Library
- Jobson, D. J., Rahman, Z., and Woodell, G. A. 1997. A multi-scale Retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans. Image Proc. 6, 7 (July), 965--976. Google ScholarDigital Library
- Khan, E. A., Reinhard, E., Fleming, R. W., and Bülthoff, H. H. 2006. Image-based material editing. ACM Trans. Graph. 25, 3 (July), 654--663. Google ScholarDigital Library
- Lagendijk, R. L., Biemond, J., and Boekee, D. E. 1988. Regularized iterative image restoration with ringing reduction. IEEE Trans. Acoustics, Speech, and Signal Proc., Speech, Signal Proc. 36, 12 (December), 1874--1888.Google Scholar
- Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. ACM Trans. Graph. 23, 3 (August), 689--694. Google ScholarDigital Library
- Levin, A., Fergus, R., Durand, F., and Freeman, W. T. 2007. Image and depth from a conventional camera with a coded aperture. ACM Trans. Graph. 26, 3 (July), Article 70. Google ScholarDigital Library
- Li, Y., Sharan, L., and Adelson, E. H. 2005. Compressing and companding high dynamic range images with subband architectures. ACM Trans. Graph. 24, 3 (July), 836--844. Google ScholarDigital Library
- Lischinski, D., Farbman, Z., Uyttendaele, M., and Szeliski, R. 2006. Interactive local adjustment of tonal values. ACM Trans. Graph. 25, 3 (July), 646--653. Google ScholarDigital Library
- Mantiuk, R., Myszkowski, K., and Seidel, H.-P. 2006. A perceptual framework for contrast processing of high dynamic range images. ACM Trans. Appl. Percept. 3, 3, 286--308. Google ScholarDigital Library
- Mrázek, P., Weickert, J., and Bruhn, A. 2006. On robust estimation and smoothing with spatial and tonal kernels. In Geometric Properties from Incomplete Data, R. Klette, R. Kozera, L. Noakes, and J. Weickert, Eds. Springer, Dordrecht, 335--352.Google Scholar
- Nordstrom, K. N. 1989. Biased anisotropic diffusion --- a unified regularization and diffusion approach to edge detection. Tech. Rep. UCB/CSD-89-514, EECS Department, University of California, Berkeley, May. Google ScholarDigital Library
- Oh, B. M., Chen, M., Dorsey, J., and Durand, F. 2001. Image-based modeling and photo editing. In Proc. ACM SIGGRAPH 2001, ACM, E. Fiume, Ed., 433--442. Google ScholarDigital Library
- Oppenheim, A. V., and Schafer, R. W. 1989. Discrete-Time Signal Processing. Prentice Hall. Google ScholarDigital Library
- Paris, S., and Durand, F. 2006. A fast approximation of the bilateral filter using a signal processing approach. In Proc. ECCV '06, IV: 568--580. Google ScholarDigital Library
- Paris, S. 2007. A gentle introduction to bilateral filtering and its applications. In ACM SIGGRAPH 2007 courses, Course 13. Google ScholarDigital Library
- Pattanaik, S. N., Ferwerda, J. A., Fairchild, M. D., and Greenberg, D. P. 1998. A multiscale model of adaptation and spatial vision for realistic image display. In Proc. ACM SIGGRAPH 98, M. Cohen, Ed., 287--298. Google ScholarDigital Library
- Perona, P., and Malik, J. 1990. Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Machine Intell. 12, 7 (July), 629--639. Google ScholarDigital Library
- Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., and Toyama, K. 2004. Digital photography with flash and no-flash image pairs. ACM Trans. Graph. 23, 3 (August), 664--672. Google ScholarDigital Library
- Reinhard, E., Stark, M., Shirley, P., and Ferwerda, J. 2002. Photographic tone reproduction for digital images. ACM Trans. Graph. 21, 3 (July), 267--276. Google ScholarDigital Library
- Saad, Y. 2003. Iterative Methods for Sparse Linear Systems, second ed. SIAM. Google ScholarDigital Library
- Scales, J. A., and Gersztenkorn, A. 1988. Robust methods in inverse theory. Inverse Problems 4, 1071--1091.Google ScholarCross Ref
- Scherzer, O., and Weickert, J. 2000. Relations between regularization and diffusion filtering. Journal of Mathematical Imaging and Vision 12, 1 (February), 43--63. Google ScholarDigital Library
- Schlick, C. 1994. Quantization techniques for visualization of high dynamic range pictures. In Photorealistic Rendering Techniques, Springer-Verlag, P. Shirley, G. Sakas, and S. Müller, Eds., 7--20.Google Scholar
- Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proc. ICCV '98, IEEE Computer Society, 839--846. Google ScholarDigital Library
- Tumblin, J., and Turk, G. 1999. LCIS: A boundary hierarchy for detail-preserving contrast reduction. In Proc. ACM SIGGRAPH 99, A. Rockwood, Ed., ACM, 83--90. Google ScholarDigital Library
- Weiss, B. 2006. Fast median and bilateral filtering. ACM Trans. Graph. 25, 3 (July), 519--526. Google ScholarDigital Library
- Winnemöller, H., Olsen, S. C., and Gooch, B. 2006. Realtime video abstraction. ACM Trans. Graph. 25, 3 (July), 1221--1226. Google ScholarDigital Library
- Zervakis, M. E. 1990. Nonlinear image restoration techniques. PhD thesis, Univ. Toronto, Toronto, ON, Canada.Google Scholar
Index Terms
- Edge-preserving decompositions for multi-scale tone and detail manipulation
Recommendations
Edge-preserving decompositions for multi-scale tone and detail manipulation
Many recent computational photography techniques decompose an image into a piecewise smooth base layer, containing large scale variations in intensity, and a residual detail layer capturing the smaller scale details in the image. In many of these ...
Photographic tone reproduction for digital images
A classic photographic task is the mapping of the potentially high dynamic range of real world luminances to the low dynamic range of the photographic print. This tone reproduction problem is also faced by computer graphics practitioners who map digital ...
Fast bilateral filtering for the display of high-dynamic-range images
We present a new technique for the display of high-dynamic-range images, which reduces the contrast while preserving detail. It is based on a two-scale decomposition of the image into a base layer, encoding large-scale variations, and a detail layer. ...
Comments