Mikhail Bessmeltsev
Abstract
Artists routinely use gesture drawings to communicate ideated character poses for storyboarding and other digital media. During subsequent posing of the 3D character models, they use these drawing as a reference, and perform the posing itself using 3D interfaces which require time and expert 3D knowledge to operate. We propose the first method for automatically posing 3D characters directly using gesture drawings as an input, sidestepping the manual 3D posing step. We observe that artists are skilled at quickly and effectively conveying poses using such drawings, and design them to facilitate a single perceptually consistent pose interpretation by viewers. Our algorithm leverages perceptual cues to parse the drawings and recover the artist-intended poses. It takes as input a vector-format rough gesture drawing and a rigged 3D character model, and plausibly poses the character to conform to the depicted pose. No other input is required. Our contribution is two-fold: we first analyze and formulate the pose cues encoded in gesture drawings; we then employ these cues to compute a plausible image space projection of the conveyed pose and to imbue it with depth. Our framework is designed to robustly overcome errors and inaccuracies frequent in typical gesture drawings. We exhibit a wide variety of character models posed by our method created from gesture drawings of complex poses, including poses with occlusions and foreshortening. We validate our approach via result comparisons to artist-posed models generated from the same reference drawings, via studies that confirm that our results agree with viewer perception, and via comparison to algorithmic alternatives.
Supplemental Material
Available for Download
Supplemental file.
- Attneave, F., and Frost, R. 1969. The determination of perceived tridimensional orientation by minimum criteria. Perception & Psychophysics 6, 6, 391--396.Google Scholar
Cross Ref
- Bai, Y., Kaufman, M. D., Liu, C. K., and Popovic, J. 2016. Artistic-dynamics for 2d animation. ACM Trans. Graph. 35, 4. Google ScholarDigital Library
- Bessmeltsev, M., Chang, W., Vining, N., Sheffer, A., and Singh, K. 2015. Modeling character canvases from cartoon drawings. Transactions on Graphics 34, 5. Google ScholarDigital Library
- Blair, P. 1994. Cartoon Animation. Walter Foster Publishing.Google Scholar
- Bregler, C., Malik, J., and Pullen, K. 2004. Twist based acquisition and tracking of animal and human kinematics. Int. J. Comput. Vision 56, 3, 179--194. Google ScholarDigital Library
- Buchanan, P., Mukundan, R., and Doggett, M. 2013. Automatic single-view character model reconstruction. In Proc. Symp. Sketch-Based Interfaces and Modeling, 5--14. Google ScholarDigital Library
- Chen, Y., Kim, T., and Cipolla, R. 2011. Silhouette-based object phenotype recognition using 3d shape priors. In IEEE International Conference on Computer Vision, ICCV, 25--32. Google ScholarDigital Library
- Cherlin, J. J., Samavati, F., Sousa, M. C., and Jorge, J. a. 2005. Sketch-based modeling with few strokes. Proc. 21st spring conference on Computer graphics 1, 212, 137. Google ScholarDigital Library
- Choi, M. G., Yang, K., Igarashi, T., Mitani, J., and Lee, J. 2012. Retrieval and visualization of human motion data via stick figures. Computer Graphics Forum 31, 2057--2065. Google ScholarDigital Library
- Cordier, F., Seo, H., Park, J., and Noh, J. Y. 2011. Sketching of mirror-symmetric shapes. IEEE Trans. Visualization and Computer Graphics 17, 11. Google ScholarDigital Library
- Davis, J., Agrawala, M., Chuang, E., Popović, Z., and Salesin, D. 2003. A Sketching Interface for Articulated Figure Animation. Proc. Symposium on Computer Animation, 320--328. Google ScholarDigital Library
- de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Transactions on Graphics 27, 1. Google ScholarDigital Library
- Eissen, K., and Steur, R. 2011. Sketching: The Basics. Bis Publishers.Google Scholar
- Favreau, J.-D., Lafarge, F., and Bousseau, A. 2016. Fidelity vs. simplicity: a global approach to line drawing vectorization. ACM Trans. Graph (Proc. SIGGRAPH 2016). Google ScholarDigital Library
- Gall, J., Stoll, C., De Aguiar, E., Theobalt, C., Rosenhahn, B., and Seidel, H. P. 2009. Motion capture using joint skeleton tracking and surface estimation. IEEE Computer Vision and Pattern Recognition Workshops, 1746--1753.Google Scholar
- Gall, J., Rosenhahn, B., Brox, T., and Seidel, H. P. 2010. Optimization and filtering for human motion capture : AAA multi-layer framework. International Journal of Computer Vision 87, 1-2, 75--92. Google ScholarDigital Library
- Gingold, Y., Igarashi, T., and Zorin, D. 2009. Structured annotations for 2D-to-3D modeling. ACM Trans. Graph. 28, 5. Google ScholarDigital Library
- Grochow, K., Martin, S. L., Hertzmann, A., and Popović, Z. 2004. Style-based inverse kinematics. ACM Trans. Graph. 23, 3 (Aug.), 522--531. Google ScholarDigital Library
- Guay, M., Cani, M.-p., and Ronfard, R. 2013. The Line of Action : an Intuitive Interface for Expressive Character Posing. ACM Trans. on Graphics, 6, 8. Google ScholarDigital Library
- Hahn, F., Mutzel, F., Coros, S., Thomaszewski, B., Nitti, M., Gross, M., and Sumner, R. W. 2015. Sketch abstractions for character posing. In Proc. Symp. Computer Animation, 185--191. Google ScholarDigital Library
- Hale, R., and Coyle, T. 1991. Master Class in Figure Drawing. Watson-Guptill.Google Scholar
- Hampton, M. 2009. Figure Drawing: Design and Invention. Figuredrawing.info.Google Scholar
- Hecker, R., and Perlin, K. 1992. Controlling 3d objects by sketching 2d views. Proc. SPIE 1828, 46--48.Google Scholar
- Hess, R., and Field, D. 1999. Integration of contours: new insights. Trends in Cognitive Sciences 3, 12 (Dec.), 480--486.Google ScholarCross Ref
- Hogarth, B. 1996. Dynamic Figure Drawing. Watson-Guptill.Google Scholar
- Ionescu, C., Papava, D., Olaru, V., and Sminchisescu, C. 2014. Human3.6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE Trans. Pattern Analysis & Machine Intelligence 36, 7, 1325--1339. Google ScholarDigital Library
- Jain, E., Sheikh, Y., Mahler, M., and Hodgins, J. 2012. Three-dimensional proxies for hand-drawn characters. ACM Trans. on Graphics 31, 1, 1--16. Google ScholarDigital Library
- Karpenko, O. A., and Hughes, J. F. 2006. SmoothSketch: 3D free-form shapes from complex sketches. ACM Trans. on Graphics 1, 212, 589--598. Google ScholarDigital Library
- Koffka, K. 1955. Principles of Gestalt Psychology. Routledge & K. Paul.Google Scholar
- Koller, D., and Friedman, N. 2009. Probabilistic Graphical Models: Principles and Techniques. MIT Press. Google ScholarDigital Library
- Kraevoy, V., Sheffer, A., and van de Panne, M. 2009. Modeling from contour drawings. In Proc. Symposium on Sketch-Based Interfaces and Modeling, 37--44. Google ScholarDigital Library
- Leland, N. 2006. The New Creative Artist. F+W Media.Google Scholar
- Lin, J., Igarashi, T., Mitani, J., and Saul, G. 2010. A sketching interface for sitting-pose design. In Proc. Sketch-Based Interfaces and Modeling Symposium, 111--118. Google ScholarDigital Library
- Mao, C., Qin, S. F., and Wright, D. K. 2005. A sketch-based gesture interface for rough 3D stick figure animation. Proc. Sketch Based Interfaces and Modeling.Google Scholar
- Nicolades, K. 1975. The Natural Way to Draw. Houghton Mifflin.Google Scholar
- Sapp, B., Toshev, A., and Taskar, B. 2010. Cascaded models for articulated pose estimation. Lecture Notes in Computer Science 6312, 406--420. Google ScholarDigital Library
- Schmidt, R., Khan, A., Kurtenbach, G., and Singh, K. 2009. On expert performance in 3D curve-drawing tasks. Proc. Symposium on Sketch-Based Interfaces and Modeling 1, 133. Google ScholarDigital Library
- Shimony, S. E. 1994. Finding maps for belief networks is np-hard. Artificial Intelligence 68, 2, 399 -- 410. Google ScholarDigital Library
- Shtof, A., Agathos, A., Gingold, Y., Shamir, A., and Cohen-Or, D. 2013. Geosemantic snapping for sketch-based modeling. Computer Graphics Forum 32, 2, 245--253.Google ScholarCross Ref
- Tekin, B., Sun, X., Wang, Lepetit, V., and Fua, P. 2015. Predicting people's 3d poses from short sequences. In arXiv preprint arXiv:1504.08200.Google Scholar
- Thiery, J.-M., Guy, E., and Boubekeur, T. 2013. Spheremeshes: Shape approximation using spherical quadric error metrics. ACM Transaction on Graphics 32, 6. Google ScholarDigital Library
- Wei, X., and Chai, J. 2011. Intuitive interactive human-character posing with millions of example poses. IEEE Comput. Graph. Appl. 31, 4, 78--88. Google ScholarDigital Library
- Xu, B., Chang, W., Sheffer, A., Bousseau, A., McCrae, J., and Singh, K. 2014. True2form: 3d curve networks from 2d sketches via selective regularization. ACM Transactions on Graphics 33, 4. Google ScholarDigital Library
- Zhao, J., and Badler, N. I. 1994. Inverse kinematics positioning using nonlinear programming for highly articulated figures. ACM Transactions on Graphics 13, 4, 313--336. Google ScholarDigital Library
Index Terms
- Gesture3D: posing 3D characters via gesture drawings
Recommendations
Sketch2Pose: estimating a 3D character pose from a bitmap sketch
Artists frequently capture character poses via raster sketches, then use these drawings as a reference while posing a 3D character in a specialized 3D software --- a time-consuming process, requiring specialized 3D training and mental effort. We tackle ...
Modeling Character Canvases from Cartoon Drawings
We introduce a novel technique for the construction of a 3D character proxy, or canvas, directly from a 2D cartoon drawing and a user-provided correspondingly posed 3D skeleton. Our choice of input is motivated by the observation that traditional ...
Enhancement of 3D character animations through the use of automatically generated guidelines inspired by traditional art concepts
SIGGRAPH '16: ACM SIGGRAPH 2016 PostersEffective communication through character animation depends on the recognition of the performed body expressions. The creation of the right body postures is crucial for character animation in the context of animated films and games, as it allows for ...
Comments