skip to main content
research-article

Revealing and modifying non-local variations in a single image

Published:02 November 2015Publication History
Skip Abstract Section

Abstract

We present an algorithm for automatically detecting and visualizing small non-local variations between repeating structures in a single image. Our method allows to automatically correct these variations, thus producing an 'idealized' version of the image in which the resemblance between recurring structures is stronger. Alternatively, it can be used to magnify these variations, thus producing an exaggerated image which highlights the various variations that are difficult to spot in the input image. We formulate the estimation of deviations from perfect recurrence as a general optimization problem, and demonstrate it in the particular cases of geometric deformations and color variations.

References

  1. Barnes, C., Shechtman, E., Goldman, D. B., and Finkelstein, A. 2010. The generalized patchmatch correspondence algorithm. In ECCV. Springer, 29--43. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Barnsley, M. F., and Sloan, A. D., 1990. Methods and apparatus for image compression by iterated function system. US Patent 4,941,193.Google ScholarGoogle Scholar
  3. Boiman, O., and Irani, M. 2007. Detecting irregularities in images and in video. International Journal of Computer Vision 74, 1, 17--31. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. 2004. High accuracy optical flow estimation based on a theory for warping. In ECCV. Springer, 25--36.Google ScholarGoogle Scholar
  5. Buades, A., Coll, B., and Morel, J.-M. 2005. A non-local algorithm for image denoising. In CVPR, vol. 2, IEEE, 60--65. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Clerc, M., and Mallat, S. 2002. The texture gradient equation for recovering shape from texture. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 4, 536--549. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Criminisi, A., Pérez, P., and Toyama, K. 2004. Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing 13, 9, 1200--1212. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Dabov, K., Foi, A., Katkovnik, V., and Egiazarian, K. 2007. Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Transactions on Image Processing 16, 8, 2080--2095. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Danielyan, A., Katkovnik, V., and Egiazarian, K. 2012. Bm3d frames and variational image deblurring. IEEE Transactions on Image Processing 21, 4, 1715--1728. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Dissemination of it for the promotion of materials science (doitpoms). http://www.doitpoms.ac.uk/.Google ScholarGoogle Scholar
  11. Efros, A. A., and Freeman, W. T. 2001. Image quilting for texture synthesis and transfer. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques, ACM, 341--346. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Eisenacher, C., Lefebvre, S., and Stamminger, M. 2008. Texture synthesis from photographs. In Computer Graphics Forum, vol. 27, Wiley Online Library, 419--428.Google ScholarGoogle Scholar
  13. Freedman, G., and Fattal, R. 2011. Image and video upscaling from local self-examples. ACM Transactions on Graphics (TOG) 30, 2, 12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Glasner, D., Bagon, S., and Irani, M. 2009. Superresolution from a single image. In ICCV, IEEE, 349--356.Google ScholarGoogle Scholar
  15. Goferman, S., Zelnik-Manor, L., and Tal, A. 2012. Context-aware saliency detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 34, 10, 1915--1926. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Hays, J., Leordeanu, M., Efros, A. A., and Liu, Y. 2006. Discovering texture regularity as a higher-order correspondence problem. In Europian Conference on Computer Vision (ECCV). Springer, 522--535. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Jojic, N., Frey, B. J., and Kannan, A. 2003. Epitomic analysis of appearance and shape. In ICCV, IEEE, 34--41. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Kim, V. G., Lipman, Y., and Funkhouser, T. A. 2012. Symmetry-guided texture synthesis and manipulation. ACM Transactions on Graphics (TOG) 31, 3, 22. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Lebrun, M., Colom, M., and Morel, J. 2014. The noise clinic: A universal blind denoising algorithm. In ICIP, IEEE, 2674--2678.Google ScholarGoogle Scholar
  20. Liu, Y., Lin, W.-C., and Hays, J. 2004. Near-regular texture analysis and manipulation. In ACM Transactions on Graphics (TOG), vol. 23, ACM, 368--376. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Liu, C., Torralba, A., Freeman, W. T., Durand, F., and Adelson, E. H. 2005. Motion magnification. ACM Transactions on Graphics (TOG) 24, 3, 519--526. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Liu, X., Jiang, L., Wong, T.-T., and Fu, C.-W. 2012. Statistical invariance for texture synthesis. IEEE Transactions on Visualization and Computer Graphics 18, 11, 1836--1848. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Liu, C. 2009. Beyond pixels: Exploring new representations and applications for motion analysis. PhD thesis, Citeseer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Malik, J., and Rosenholtz, R. 1997. Computing local surface orientation and shape from texture for curved surfaces. International Journal of Computer Vision 23, 2, 149--168. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Michaeli, T., and Irani, M. 2013. Nonparametric blind superresolution. In ICCV, IEEE, 945--952. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Michaeli, T., and Irani, M. 2014. Blind deblurring using internal patch recurrence. In ECCV.Google ScholarGoogle Scholar
  27. Mishne, G., and Cohen, I. 2013. Multiscale anomaly detection using diffusion maps. IEEE Journal of Selected Topics in Signal Processing 7, 1, 111--123.Google ScholarGoogle ScholarCross RefCross Ref
  28. Park, M., Brocklehurst, K., Collins, R. T., and Liu, Y. 2009. Deformed lattice detection in real-world images using mean-shift belief propagation. IEEE Transactions on Pattern Analysis and Machine Intelligence 31, 10, 1804--1816. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Seo, H. J., and Milanfar, P. 2009. Static and space-time visual saliency detection by self-resemblance. Journal of vision 9, 12, 15.Google ScholarGoogle ScholarCross RefCross Ref
  30. Shankar, N., and Zhong, Z. 2005. Defect detection on semi-conductor wafer surfaces. Microelectronic Engineering 77, 3, 337--346. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Simakov, D., Caspi, Y., Shechtman, E., and Irani, M. 2008. Summarizing visual data using bidirectional similarity. In CVPR, IEEE.Google ScholarGoogle Scholar
  32. Wadhwa, N., Rubinstein, M., Durand, F., and Freeman, W. T. 2013. Phase-based video motion processing. ACM Trans. Graph.. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Wadhwa, N., Dekel, T., Wei, D., Durand, F., and Freeman, W. T. 2015. Deviation magnification: Revealing geometric deviation in a single image. In SIGGRAPH Asia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Wu, H., Rubinstein, M., Shih, E., Guttag, J. V., Durand, F., and Freeman, W. T. 2012. Eulerian video magnification for revealing subtle changes in the world. ACM Trans. Graph.. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Zontak, M., and Cohen, I. 2010. Defect detection in patterned wafers using anisotropic kernels. Machine Vision and Applications 21, 2, 129--141. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Revealing and modifying non-local variations in a single image

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Graphics
        ACM Transactions on Graphics  Volume 34, Issue 6
        November 2015
        944 pages
        ISSN:0730-0301
        EISSN:1557-7368
        DOI:10.1145/2816795
        Issue’s Table of Contents

        Copyright © 2015 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 2 November 2015
        Published in tog Volume 34, Issue 6

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader