Mingtian Zhao
Song-Chun Zhu
ABSTRACT
We present an interactive abstract painting system named Sisley. Sisley works upon the psychological principle [Berlyne 1971] that abstract arts are often characterized by their greater perceptual ambiguities than photographs, which tend to invoke moderate mental efforts of the audience for interpretation, accompanied with subtle aesthetic pleasures. Given an input photograph, Sisley decomposes it into a hierarchy/tree of its constituent image components (e.g., regions, objects of different categories) with interactive guidance from the user, then automatically generates corresponding abstract painting images, with increased ambiguities of both the scene and individual objects at desired levels. Sisley consists of three major working parts: (1) an interactive image parser executing the tasks of segmentation, labeling, and hierarchical organization, (2) a painterly rendering engine with abstract operators for transferring the image appearance, and (3) a numerical ambiguity computation and control module of servomechanism. With the help of Sisley, even an amateur user can create abstract paintings from photographs easily in minutes. We have evaluated the rendering results of Sisley using human experiments, and verified that they have similar abstract effects to original abstract paintings by artists.
- Berlyne, D. E. 1971. Aesthetics and Psychobiology. Appleton-Century-Crofts, Inc.Google Scholar
- Boykov, Y., and Kolmogorov, V. 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26, 9, 1124--1137. Google ScholarDigital Library
- Bridson, R. 2007. Fast poisson disk sampling in arbitrary dimensions. In SIGGRAPH 2007 Sketches, 22. Google ScholarDigital Library
- Collomosse, J. P., and Hall, P. M. 2003. Cubist style rendering of photographs. IEEE Trans. Vis. Comput. Graph. 9, 4, 443--453. Google ScholarDigital Library
- Collomosse, J. P., and Hall, P. M. 2006. Video motion analysis for the synthesis of dynamic cues and futurist art. Graphical Models, 5, 402--414. Google ScholarDigital Library
- Cooke, H. L. 1978. Painting Techniques of the Masters. Watson Guptill / Pitman Publishing.Google Scholar
- Cover, T. M., and Thomas, J. A. 2006. Elements of Information Theory, 2nd Edition. Wiley-Interscience. Google ScholarDigital Library
- DeCarlo, D., and Santella, A. 2002. Stylization and abstraction of photographs. In Proceedings of SIGGRAPH 2002, 769--776. Google ScholarDigital Library
- Deussen, O., Hiller, S., van Overveld, C., and Strothotte, T. 2000. Floating points: A method for computing stipple drawings. Comput. Graph. Forum 19, 3, 40--51.Google ScholarCross Ref
- Duda, R. O., Hart, P. E., and Stork, D. G. 2000. Pattern Classification, 2nd Edition. Wiley-Interscience. Google ScholarDigital Library
- Durand, F. 2002. An invitation to discuss computer depiction. In Proceedings of NPAR 2002, 111--124. Google ScholarDigital Library
- Finkelstein, A., and Range, M. 1998. Image mosaics. In Proceedings of the 7th International Conference on Electronic Publishing, held jointly with the 4th International Conference on Raster Imaging and Digital Typography (EP/RIDT '98), 11--22. Google ScholarDigital Library
- Funch, B. S. 1997. The Psychology of Art Appreciation. Museum Tusculanum Press.Google Scholar
- Gal, R., Sorkine, O., Popa, T., Sheffer, A., and Cohen-Or, D. 2007. 3D collage: Expressive non-realistic modeling. In Proceedings of NPAR 2007, 7--14. Google ScholarDigital Library
- Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. 2004. Bayesian Data Analysis, 2nd Edition. Chapman & Hall.Google Scholar
- Gooch, B., and Gooch, A. A. 2001. Non-Photorealistic Rendering. A K Peters, Ltd. Google ScholarDigital Library
- Guo, C.-E., Zhu, S.-C., and Wu, Y. N. 2007. Primal sketch: Integrating structure and texture. Comput. Vis. Image Understand. 106, 1, 5--19. Google ScholarDigital Library
- Haeberli, P. 1990. Paint by numbers: Abstract image representations. In Computer Graphics (Proceedings of SIGGRAPH '90), 207--214. Google ScholarDigital Library
- Hertzmann, A., and Perlin, K. 2000. Painterly rendering for video and interaction. In Proceedings of NPAR 2000, 7--12. Google ScholarDigital Library
- Hertzmann, A. 1998. Painterly rendering with curved brush strokes of multiple sizes. In Proceedings of SIGGRAPH '98, 453--460. Google ScholarDigital Library
- Jaynes, E. T. 1957. Information theory and statistical mechanics. Phys. Rev. 106, 4, 620--630.Google ScholarCross Ref
- Kaplan, C. S., and Salesin, D. H. 2004. Dihedral Escherization. In Proceedings of Graphics Interface 2004, 255--262. Google ScholarDigital Library
- Kersten, D. 1987. Predictability and redundancy of natural images. J. Opt. Soc. Am. A 4, 12, 2395--2400.Google ScholarCross Ref
- Klein, A. W., Sloan, P.-P. J., Colburn, R. A., Finkelstein, A., and Cohen, M. F. 2001. Video cubism. Tech. Rep. MSR-TR-2001-45, Microsoft Research.Google Scholar
- Klein, A. W., Grant, T., Finkelstein, A., and Cohen, M. F. 2002. Video mosaics. In Proceedings of NPAR 2002, 21--28. Google ScholarDigital Library
- Lee, S., Olsen, S. C., and Gooch, B. 2006. Interactive 3D fluid jet painting. In Proceedings of NPAR 2006, 97--104. Google ScholarDigital Library
- Litwinowicz, P. 1997. Processing images and video for an impressionist effect. In Proceedings of SIGGRAPH '97, 407--414. Google ScholarDigital Library
- Lombaert, H., Sun, Y., Grady, L., and Xu, C. 2005. A multilevel banded graph cuts method for fast image segmentation. In Proceedings of ICCV 2005, Volume 1, 259--265. Google ScholarDigital Library
- Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 2, 91--110. Google ScholarDigital Library
- Marr, D. 1982. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. W. H. Freeman. Google ScholarDigital Library
- Meier, B. J. 1996. Painterly rendering for animation. In Proceedings of SIGGRAPH '96, 477--484. Google ScholarDigital Library
- Mi, X., DeCarlo, D., and Stone, M. 2009. Abstraction of 2D shapes in terms of parts. In Proceedings of NPAR 2009, 15--24. Google ScholarDigital Library
- Montgomery, D. C. 2000. Design and Analysis of Experiments, 5th Edition. Wiley.Google Scholar
- Morel, J.-M., Alvarez, L., Galerne, B., and Gousseau, Y., 2006. Texture synthesis by abstract painting technique. http://www.cmla.ens-cachan.fr/membres/morel.html.Google Scholar
- Mureika, J. R., Dyer, C. C., and Cupchik, G. C. 2005. On multifractal structure in non-representational art. Phys. Rev. E 72, 046101.Google ScholarCross Ref
- Orchard, J., and Kaplan, C. S. 2008. Cut-out image mosaics. In Proceedings of NPAR 2008, 79--87. Google ScholarDigital Library
- Orzan, A., Bousseau, A., Barla, P., and Thollot, J. 2007. Structure-preserving manipulation of photographs. In Proceedings of NPAR 2007, 103--110. Google ScholarDigital Library
- Raskar, R., Ilie, A., and Yu, J. 2004. Image fusion for context enhancement and video surrealism. In Proceedings of NPAR 2004, 85--152. Google ScholarDigital Library
- Rigau, J., Feixas, M., and Sbert, M. 2008. Informational aesthetics measures. IEEE Comput. Graph. and Appl. 28, 2, 24--34. Google ScholarDigital Library
- Ruderman, D. L. 1994. The statistics of natural images. Network: Computation in Neural Systems 5, 4, 517--548.Google ScholarCross Ref
- Smith, K., Liu, Y., and Klein, A. 2005. Animosaics. In Proceedings of SCA 2005, 201--208. Google ScholarDigital Library
- Stork, D. G. 2009. Computer vision and computer graphics analysis of paintings and drawings: An introduction to the literature. In Proceedings of the 13th International Conference on Computer Analysis of Images and Patterns (CAIP '09), 9--24. Google ScholarDigital Library
- Strothotte, T., and Schlechtweg, S. 2002. Non-Photorealistic Computer Graphics: Modeling, Rendering and Animation. Morgan Kaufmann. Google ScholarDigital Library
- Taylor, R. P., Micolich, A. P., and Jonas, D. 1999. Fractal analysis of Pollock's drip paintings. Nature 399, 422.Google ScholarCross Ref
- Torralba, A., Fergus, R., and Weiss, Y. 2008. Small codes and large databases for recognition. In Proceedings of CVPR 2008, 1--8.Google Scholar
- Tu, Z., Chen, X., Yuille, A. L., and Zhu, S.-C. 2005. Image parsing: Unifying segmentation, detection, and recognition. Int. J. Comput. Vis. 63, 2, 113--140. Google ScholarDigital Library
- van de Sande, K., Gevers, T., and Snoek, C. 2010. Evaluating color descriptors for object and scene recognition. IEEE Trans. Pattern Anal. Mach. Intell., in press. Google ScholarDigital Library
- von Helmholtz, H. 1866. Physiological Optics. Transl. JP Southhall, 1911. Washington, DC: Opt. Soc. Am. (From German).Google Scholar
- Wallraven, C., Fleming, R., Cunningham, D., Rigau, J., Feixas, M., and Sbert, M. 2009. Categorizing art: Comparing humans and computers. Computers & Graphics 33, 4, 484--495. Google ScholarDigital Library
- Winnemöller, H., Olsen, S. C., and Gooch, B. 2006. Realtime video abstraction. ACM Trans. Graph. 25, 3, 1221--1226. Google ScholarDigital Library
- Xu, J., and Kaplan, C. S. 2008. Artistic thresholding. In Proceedings of NPAR 2008, 39--47. Google ScholarDigital Library
- Yao, B., Yang, X., and Zhu, S.-C. 2007. Introduction to a large-scale general purpose ground truth database: Methodology, annotation tool and benchmarks. In Proceedings of the International Conferences on Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR '07), 169--183. Google ScholarDigital Library
- Yedidia, J. S., Freeman, W. T., and Weiss, Y., 2001. Understanding belief propagation and its generalizations. IJCAI 2001 Distinguished Lecture Track.Google Scholar
- Yevin, I. 2006. Ambiguity in art. Complexus 2006, 3, 74--83.Google ScholarCross Ref
- Zeng, K., Zhao, M., Xiong, C., and Zhu, S.-C. 2009. From image parsing to painterly rendering. ACM Trans. Graph. 29, 1, 2:1--2:11. Google ScholarDigital Library
Index Terms
- Sisley the abstract painter
Recommendations
Portrait painting using active templates
NPAR '11: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Non-Photorealistic Animation and RenderingPortraiture plays a substantial role in traditional painting, yet it has not been studied in depth in painterly rendering research. The difficulty in rendering human portraits is due to our acute visual perception to the structure of human face. To ...
Abstract painting with interactive control of perceptual entropy
This article presents a framework for generating abstract art from photographs. The aesthetics of abstract art is largely attributed to its greater perceptual ambiguity than photographs. According to psychological theories [Berlyne 1971], the ambiguity ...
Using artificial intelligence techniques to emulate the creativity of a portrait painter
EVA '16: Proceedings of the conference on Electronic Visualisation and the ArtsWe present three new machine learning based artificial intelligence (AI) techniques, which we have added to our parameterised computational painterly rendering framework and show their benefits in computational creativity and non-photorealistic ...
Comments