Abstract
We demonstrate a method to compute three-dimensional (3D) motion fields on a face. Twelve synchronized and calibrated cameras are positioned around a talking person, and observe its head in motion. We represent the head as a deformable mesh, which is fitted in a global optimization step to silhouette-contour and multi-camera stereo data derived from all images. The non-rigid displacement of the mesh from frame to frame, the 3D motion field, is determined from the spatiotemporal derivatives in all the images. We integrate these cues over time, thus producing an animated representation of the talking head. Our ability to estimate 3D motion fields points to a new framework for the study of action. The 3D motion fields can serve as an intermediate representation, which can be analyzed using geometrical and statistical tools for the purpose of extracting representations of generic human actions.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download to read the full chapter text
Chapter PDF
References
Aggarwal, J. and Cai, Q. (1999). Human motion analysis: A review. Computer Vision and Image Understanding, 73 (3): 428–440.
Bascle, B. and Blake, A. (1998). Separability of pose and expression in facial tracing and animation. In ICCV98, pages 323–328.
Black, M. and Yacoob, Y. (1997). Recognizing facial expressions in image sequences using local parameterized models of image motion. IJCV, 25 (1): 23–48.
Bottema, O. and Roth, B. (1979). Theoretical Kinematics. North-Holland.
Davis, L., Borovikov, E., Cutler, R., Harwood, D., and Horprasert, T. (1999). Multi-perspective analysis of human action. In Proc. of Third ANIMATED HEADS: Prom 3D Motion Fields to Action Descriptions 11 International Workshop on Cooperative Distributed Vision,Kyoto, Japan.
Essa, I. and Pentland, A. (1997). Coding, analysis, interpretation, and recognition of facial expressions. IEEE Trans. PAMI, 19 (7): 757–763.
Fermüller, C., Pless, R., and Aloimonos, Y. (2000). The Ouchi illusion as an artifact of biased flow estimation. Vision Research, 40: 77–96.
Fua, P. and Miccio, C. (1999). Animated heads from ordinary images: A least-squares approach. Computer Vision and Image Understanding, 75 (3): 247–259.
Gavrila, D. (1999). The visual analysis of human movement: A survey. Computer Vision and Image Understanding, 73 (1): 82–98.
Guenter, B., Grimm, C., Wood, D., Malvar, H., and Pighin, F. (1998). Making faces. In Proc. of SIGGRAPH, pages 55–66.
Horn, B. K. P. (1986). Robot Vision. McGraw Hill, New York. Laurentini, A. (1994). The visual hull concept for silhouette-based image understanding. IEEE Trans. PAMI, 16 (2): 150–162.
Malassiotis, S. and Strintzis, M. (1997). Model-based joint motion and structure estimation from stereo images. Computer Vision and Image Understanding, 65 (1): 79–94.
Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., and Salesin, D. (1998). Synthesizing realistic facial expressions from photographs.
Reynard, D., Wildenberg, A., Blake, A., and Marchant, J. (1996). Learning dynamics of complex motions from image sequences. In ECCV96, pages 1: 357–368.
Spies, H., Jaehne, B., and Barron, J. (2000). Dense range flow from depth and intensity data. In ICPR00.
Terzopoulos, D. and Waters, K. (1993). Analysis and synthesis of facial image sequences using physical and anatomical models. IEEE Trans. PAMI, 15 (6): 569–579.
Vedula, S., Baker, S., Rander, P., Collins, R., and Kanade, T. (1999). Three-dimensional scene flow. In ICCV99, pages 722–729.
Vedula, S., Baker, S., Seitz, S., and Kanade, T. (2000). Shape and motion carving in 6d. In CVPR00, pages II: 592–598.
Vetter, T. and Blanz, V. (1998). Estimating coloured 3-d face models from single images: An example-based approach. In ECCV98, pages 499–513.
Zhang, Y. and Kambhamettu, C. (2000). Integrated 3d scene flow and structure recovery from multiview image sequences. In CVPR00, pages 11: 674–681.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer Science+Business Media New York
About this chapter
Cite this chapter
Neumann, J., Fermüller, C., Aloimonos, Y. (2001). Animated Heads: From 3D Motion Fields to Action Descriptions. In: Magnenat-Thalmann, N., Thalmann, D. (eds) Deformable Avatars. IFIP — The International Federation for Information Processing, vol 68. Springer, Boston, MA. https://doi.org/10.1007/978-0-306-47002-8_1
Download citation
DOI: https://doi.org/10.1007/978-0-306-47002-8_1
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4757-4930-4
Online ISBN: 978-0-306-47002-8
eBook Packages: Springer Book Archive