Xiaolin Wei
Abstract
This paper presents a video-based motion modeling technique for capturing physically realistic human motion from monocular video sequences. We formulate the video-based motion modeling process in an image-based keyframe animation framework. The system first computes camera parameters, human skeletal size, and a small number of 3D key poses from video and then uses 2D image measurements at intermediate frames to automatically calculate the "in between" poses. During reconstruction, we leverage Newtonian physics, contact constraints, and 2D image measurements to simultaneously reconstruct full-body poses, joint torques, and contact forces. We have demonstrated the power and effectiveness of our system by generating a wide variety of physically realistic human actions from uncalibrated monocular video sequences such as sports video footage.
Supplemental Material
Available for Download
- Agarwal, A., and Triggs, B. 2006. Recovering 3D human pose from monocular images. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI). 28(1):44--58. Google Scholar
Digital Library
- Bazaraa, M. S., Sherali, H. D., and Shetty, C. M. 1993. Nonlinear Programming: Theory and Algorithms. John Wiley and Sons Ltd. 2nd Edition.Google Scholar
- Bregler, C., Malik, J., and Pullen, K. 2004. Twist Based Acquisition and Tracking of Animal and Human Kinematics. International Journal of Computer Vision. 56(3):179--194. Google ScholarDigital Library
- Brubaker, M. A., and Fleet, D. J. 2008. The Kneed Walker for human pose tracking. In Proceedings of IEEE CVPR. 1--8.Google Scholar
- Chai, J., and Hodgins, J. 2005. Performance Animation from Low-dimensional Control Signals. In ACM Transactions on Graphics. 24(3):686--696. Google ScholarDigital Library
- Chen, Y.-L., and Chai, J. 2009. Simultaneous Reconstruction of 3D Human Skeleton and Motion from Monocular Video Sequences. Proceedings of The Ninth Asian Conference on Computer Vision.Google Scholar
- Cohen, M. F. 1992. Interactive Spacetime Control for Animation. In Proceedings of ACM SIGGRAPH 1992. 293--302. Google ScholarDigital Library
- Comaniciu, D., and Meer, P. 2002. Mean Shift: A Robust Approach Toward Feature Space Analysis. In IEEE Trans. Pattern Analysis and Machine Intelligence. 24(5):603--619. Google ScholarDigital Library
- Cowley, A., and Taylor, C. J., 2001. Videomocap: A video based motion capture system. http://www.cis.upenn.edu/cjtaylor/RESEARCH/projects/Johansson/VideoMoCap.html.Google Scholar
- DiFranco, D. E., Cham, T.-J., and Rehg, J. M. 2001. Reconstruction of 3D figure motion from 2D correspondences. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1:307--314.Google Scholar
- Elgammal, A., and Lee, C. 2004. Inferring 3D body pose from silhouettes using activity manifold learning. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2: 681--688. Google ScholarDigital Library
- Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., and Stahel, W. A. 1986. Robust Statistics: The Approach Based on Influence Functions. Wiley.Google Scholar
- Howe, N., Leventon, M., and Freeman, W. 1999. Bayesian Reconstruction of 3D Human Motion from Single-camera Video. In Advances in Neural Information Processing Systems 12. 820--826.Google Scholar
- Huber, P. J. 1981. Robust Statistics. Wiley.Google Scholar
- Kanaujia, C. S. A., and Metaxas, D. 2007. BM3E: Discriminative density propagation for visual tracking. In IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI). 29(11):2030--2044. Google ScholarDigital Library
- Liu, K., Hertzmann, A., and Popović, Z. 2005. Learning Physics-Based Motion Style with Nonlinear Inverse Optimization. In ACM Transactions on Graphics. 23(3):1071--1081. Google ScholarDigital Library
- Lourakis, M. I. A. 2009. levmar: Levenberg marquardt nonlinear least squares algorithms in c/c++. In http://www.ics.forth.gr/lourakis/levmar/.Google Scholar
- Loy, G., Eriksson, M., Sullivan, J., and Carlsson, S. 2004. Monocular 3D Reconstruction of Human motion in Long Action Sequences. In European Conference on Computer Vision. 442--455.Google Scholar
- MatchMover, 2008. http://www.realviz.com/.Google Scholar
- Pavlović, V., Rehg, J. M., and MacCormick, J. 2000. Learning Switching Linear Models of Human Motion. In Advances in Neural Information Processing Systems 13, 981--987.Google Scholar
- Pollard, N., and Reitsma, P. 2001. Animation of Human-like Characters: Dynamic Motion Filtering with A Physically Plausible Contact Model. In In Yale Workshop on Adaptive and Learning Systems.Google Scholar
- Popović, Z., and Witkin, A. P. 1999. Physically Based Motion Transformation. In Proceedings of ACM SIGGRAPH 1999. 11--20. Google ScholarDigital Library
- Rosales, R., and Sclaroff, S. 2000. Specialized Mappings and the Estimation of Human Body Pose from a Single Image. In Proceedings of the Workshop on Human Motion. 19--24. Google ScholarDigital Library
- Safonova, A., Hodgins, J., and Pollard, N. 2004. Synthesizing Physically Realistic Human Motion in Low-Dimensional, Behavior-Specific Spaces. In ACM Transactions on Graphics. 23(3):514--521. Google ScholarDigital Library
- Sidenbladh, H., Black, M. J., and Sigal, L. 2002. Implicit Probabilistic Models of Human Motion for Synthesis and Tracking. In European Conference on Computer Vision. 784--800. Google ScholarDigital Library
- Sminchisescu, C., and Jepson, A. 2004. Generative Modeling for Continuous Non-Linearly Embedded Visual Inference. In ICML, 759--766. Google ScholarDigital Library
- Sulejmanpasic, A., and Popović, J. 2005. Adaptation of Performed Ballistic Motion. In ACM Transactions on Graphics. 24(1):165--179. Google ScholarDigital Library
- Taylor, C. J. 2000. Reconstruction of Articulated Objects from Point Correspondences in a Single Uncalibrated Image. In Computer Vision and Image Understanding. 80(3):349--363. Google ScholarDigital Library
- Urtasun, R., Fleet, D. J., Hertzmann, A., and Fua., P. 2005. Priors for people tracking from small training sets. In IEEE International Conference on Computer Vision, 403--410. Google ScholarDigital Library
- Vicon Systems, 2009. http://www.vicon.com.Google Scholar
- Vondrak, M., Sigal, L., and Jenkins, O. C. 2008. Physical simulation for probabilistic motion tracking. In IEEE Conference on Computer Vision and Pattern Recognition, 1--8.Google Scholar
- Wei, X., and Chai, J. 2008. Interactive Tracking of 2D Generic Objects with Spacetime Optimization. In Proceedings of European Conference on Computer Vision. 1:657--670. Google ScholarDigital Library
- Wei, X., and Chai, J. 2009. Modeling 3D Human Poses from Uncalibrated Monocular Images. Proceedings of IEEE Conference on Computer Vision.Google Scholar
- Witkin, A., and Kass, M. 1988. Spacetime Constraints. In Proceedings of ACM SIGGRAPH 1998. 159--168. Google ScholarDigital Library
Index Terms
- VideoMocap: modeling physically realistic human motion from monocular video sequences
Recommendations
VideoMocap: modeling physically realistic human motion from monocular video sequences
SIGGRAPH '10: ACM SIGGRAPH 2010 papersThis paper presents a video-based motion modeling technique for capturing physically realistic human motion from monocular video sequences. We formulate the video-based motion modeling process in an image-based keyframe animation framework. The system ...
Realtime human motion control with a small number of inertial sensors
I3D '11: Symposium on Interactive 3D Graphics and GamesThis paper introduces an approach to performance animation that employs a small number of motion sensors to create an easy-to-use system for an interactive control of a full-body human character. Our key idea is to construct a series of online local ...
Physics-based character animation with cascadeur
SIGGRAPH '19: ACM SIGGRAPH 2019 StudioIn this workshop we will create a realistic acrobatic 3D fighting animation using the animation software Cascadeur. We will learn the key features of physics-based character animation and will immediately apply the learned knowledge by creating an ...
Comments