Abstract
We present an interactive system for fragment-based image completion which exploits information about the approximate 3D structure in a scene in order to estimate and apply perspective corrections when copying a source fragment to a target position. Even though implicit 3D information is used, the interaction is strictly 2D, which makes the user interface very simple and intuitive. We propose different interaction metaphors in our system for providing 3D information interactively. Our search and matching procedure is done in the Fourier domain, and hence it is very fast and it allows us to use large fragments and multiple source images with high resolution while still obtaining interactive response times. Our image completion technique also takes user-specified structure information into account where we generalize the concept of feature curves to arbitrary sets of feature pixels. We demonstrate our technique on a number of difficult completion tasks.
Similar content being viewed by others
References
Arya, S., Mount, D.M.: Approximate nearest neighbor queries in fixed dimensions. In: SODA ’93: Proceedings of the fourth annual ACM-SIAM Symposium on Discrete algorithms, pp. 271–280. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA (1993)
Ashikhmin, M.: Synthesizing natural textures. In: SI3D ’01: Proceedings of the 2001 symposium on Interactive 3D graphics, pp. 217–226. ACM Press, New York, NY, USA (2001)
Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., Verdera, J.: Filling-in by joint interpolation of vector fields and gray levels. In: IEEE Transactions on Image Processing, vol. 10, pp. 1200–1211 (2001)
Barrett, W.A., Cheney, A.S.: Object-based image editing. In: SIGGRAPH ’02: Proceedings of the 29th annual conference on Computer graphics and interactive techniques, pp. 777–784 (2002)
Bertalmio, M., Bertozzi, A., Sapiro, G.: Navier-stokes, fluid dynamics, and image and video inpainting. In: Proc. of Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 355–362 (2001)
Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: SIGGRAPH ’00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pp. 417–424. ACM Press/Addison-Wesley Publishing Co., New York, NY, USA (2000)
Bertalmio, M., Vese, L.A., Sapiro, G., Osher, S.: Simultaneous structure and texture image inpainting. In: CVPR (2), pp. 707–712 (2003)
Boehm, W., Prautzsch, H.: Geometric Concepts for Geometric Design. A K Peters, Ltd. (1994)
Criminisi, A., Pérez, P., Toyama, K.: Object removal by exemplar-based inpainting. In: CVPR (2), pp. 721–728 (2003)
Datar, M., Immorlica, N., Indyk, P., Mirrokni, V.S.: Locality-sensitive hashing scheme based on p-stable distributions. In: SCG ’04: Proceedings of the twentieth annual symposium on Computational geometry, pp. 253–262. ACM Press, New York, NY, USA (2004)
Debevec, P.E., Taylor, C.J., Malik, J.: Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach. In: SIGGRAPH ’96: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pp. 11–20 (1996)
Drori, I., Cohen-Or, D., Yeshurun, H.: Fragment-based image completion. ACM Transactions on Graphics 22(3), 303–312 (2003)
Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: SIGGRAPH ’01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pp. 341–346. ACM Press, New York, NY, USA (2001)
Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: ICCV ’99: Proceedings of the International Conference on Computer Vision-Volume 2, p. 1033 (1999)
Frigo, M., Johnson, S.G.: The design and implementation of FFTW3. Proceedings of the IEEE 93(2), 216–231 (2005)
Harrison, P.: A non-hierarchical procedure for re-synthesis of complex textures. In: International Conference in Central Europe on Computer Graphics and Visualization, pp. 190–197 (2001)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2003)
Hel-Or, Y., Hel-Or, H.: Real time pattern matching using projection kernels. In: ICCV, pp. 1486–1493 (2003)
Horry, Y., Anjyo, K.I., Arai, K.: Tour into the picture: using a spidery mesh interface to make animation from a single image. In: SIGGRAPH ’97: Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pp. 225–232 (1997)
Jia, J., Tang, C.K.: Image repairing: Robust image synthesis by adaptive nd tensor voting. In: CVPR (1), pp. 643–650 (2003)
Kasson, J.M., Plouffe, W.: An analysis of selected computer interchange color spaces. ACM Trans. Graph. 11(4), 373–405 (1992)
Komodakis, N., Tziritas, G.: Image completion using global optimization. In: CVPR06, pp. I: 442–452 (2006)
Kwatra, V., Essa, I., Bobick, A., Kwatra, N.: Texture optimization for example-based synthesis. ACM Trans. Graph. 24(3), 795–802 (2005)
Kwatra, V., Schoedl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: Image and video synthesis using graph cuts. ACM Transactions on Graphics, SIGGRAPH 2003 22(3), 277–286 (2003)
Liang, L., Liu, C., Xu, Y.Q., Guo, B., Shum, H.Y.: Real-time texture synthesis by patch-based sampling. ACM Trans. Graph. 20(3), 127–150 (2001)
Liu, Y., Lin, W.C., Hays, J.: Near-regular texture analysis and manipulation. ACM Trans. Graph. 23(3), 368–376 (2004)
Oh, B.M., Chen, M., Dorsey, J., Durand, F.: Image-based modeling and photo editing. In: SIGGRAPH ’01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pp. 433–442. ACM Press, New York, NY, USA (2001)
Oliveira, M.M., Bowen, B., McKenna, R., Chang, Y.S.: Fast digital image inpainting. In: Proceedings of the International Conference on Visualization, Imaging and Image Processing, pp. 261–266 (2001)
Pavić, D., Schönefeld, V., Kobbelt, L.: Accompanying video. http://www.rwth-graphics.de/downloads/
Pérez, P., Gangnet, M., Blake, A.: Patchworks: Example-based region tiling for image editing. Tech. Rep. MSR-TR-2004-04, Microsoft Research (2004)
Sun, J., Yuan, L., Jia, J., Shum, H.Y.: Image completion with structure propagation. ACM Trans. Graph. 24(3), 861–868 (2005)
Wei, L.Y., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: SIGGRAPH ’00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pp. 479–488. ACM Press/Addison-Wesley Publishing Co., New York, NY, USA (2000)
Wilczkowiak, M., Brostow, G.J., Tordoff, B., Cipolla, R.: Hole filling through photomontage. In: 16th British Machine Vision Conference 2005 - BMVC’2005, Oxford, United Kingdom, pp. 492–501 (2005)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pavić, D., Schönefeld, V. & Kobbelt, L. Interactive image completion with perspective correction. Visual Comput 22, 671–681 (2006). https://doi.org/10.1007/s00371-006-0050-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-006-0050-2