doi:10.1016/j.patrec.2006.11.011
Copyright © 2006 Elsevier B.V. All rights reserved.
3D human model and joint parameter estimation from monocular image
Minglei Tonga,
,
, Yuncai Liua and Thomas S. Huangb
aInstitute of Image Processing and Pattern Recognition, Shanghai JiaoTong University, Shanghai, 200030, People’s Republic of China
bBeckman Institute, University of Illinois at Urbana-Champaign, Urbana, IL 61801, United States
Received 18 October 2004;
revised 20 January 2006.
Communicated by S. Dickinson.
Available online 23 January 2007.
References and further reading may be available for this article. To view references and further reading you must
purchase this article.
Abstract
In this paper we present a novel class of human model described by convolution surface attached to articulated kinematics skeletons. The human pose can be estimated from silhouette in monocular images. The contribution of this paper consists of three points: First, human model of convolution surface is presented and its shape is deformable when changing polynomial parameters and radius parameters. Second, convolution surface and curve correspondence theorem is presented to give a map between 3D pose and 2D contour. Third, we model the human silhouette with convolution curve in order to estimate joint parameters from monocular images and we also give an effective constraint function. Evaluation of this approach is performed on some video frames about a walking man. The experiment result shows that our method works well without self-occlusion.
Keywords: 3D human model; 2D images; Convolution surface; Model initialization; Motion estimation
Fig. 1. Line segment L and vector
.
Fig. 2. Let s = constant, line segment [−0.25, 0.25] convolved with Cauchy function, in first row changing s can adjust the shape of the convolution surface; in second row, keep s = constant, adjusting (q0, q1) can change the shape of the convolution surface; in third row, adjusting (q0, q1) can change the shape of the convolution curve.
Fig. 3. a–c and d–f show human body with two gestures in different viewpoints, h is the correspondence convolution curve of model g.
Fig. 4. First two images show the skeleton and convolution surface model without image information. Last four images show the initialization process with a specified gesture.
Fig. 5. First row is the gray video sequences, second row is image after foreground segmentation.
Fig. 6. The figure with red line is the rough contour and the figure with blue line is the contour after smoothing. (For interpretation of the references in colour in this figure legend, the reader is referred to the web version of this article.)
Fig. 7. The sketch map of the joint and human skeleton likelihood.
Fig. 8. Continuous 10 frame image and the recovered 3D model of a walking man from CMU video datasets: a row shows the gray image sequence. The b–k row show the estimation of a walking man. The 1th column shows the human contour and convolution curve, the cross-mark means the projection of 3D joint point; the 2th column shows the skeleton after image skeletonization; the 3th column shows the estimated 3D skeleton; the 4th column shows the reconstruction 3D human model.
Fig. 9. First image shows us the optimization with joint and skeleton constrain. Second image shows us the optimization without joint and skeleton constrain.
Fig. 10. The sketch map of convolution surface projection under orthogonal projection, the broken line on convolution surface is its contour generator and the real line on image plane is convolution curve.
Fig. 11. Human model under orthogonal projection.
Table 1.
The DOF numbers of main model joints
