Fig. 1. Varying pose face detection cum registration system overview. (a) The skin-colour detection is first used to estimate the initial position and size of varying pose faces. The detection outcomes are then fed to the varying pose registration stage, which aligns each input face patch with a varying pose face model and outputs the position of facial features. (b) Varying pose face registration consists of three components: (1) a varying pose face model I^M(α,γ) capturing permissible image variation; (2) a combined FTSM accounting for image differences between model image and given face images; and (3) a face alignment algorithm used to find the best model parameters corresponding to the minima of the proposed similarity measure.

Fig. 2. Skin-colour detection. Left: input colour image; right: skin colour converted image with each pixel classified by a Gaussian classifier; bottom: face detection result after applying X, Y projection and thresholding.

Fig. 3. False skin-colour detection. Left and right: skin-coloured books on left are false positives; middle: a hand is misclassified as faces. The false detection can be eliminated by examining the appearance and structure of input object images.

Fig. 4. Examples of varying pose face images (in the MIT Beymer database and the Olive face database).

Fig. 5. Feature based image warping. Original image I₀, feature points correspondence map C, and corresponding warped image I are shown from left to right.

Fig. 6. Left to right: reference image I₀, and warped images with first three eigenvectors, given pose parameter = 1000 (I₀1000E_i, i=1,2,3).

Fig. 7. Distribution of the first three principle components of varying pose faces (pose 1: right rotation; pose 2: minor right rotation; pose 3: frontal; pose 4: minor left rotation; and pose 5: left rotation).

Fig. 8. A face prototype and feature points, which consist of 16 line segments and 24 end points of line segments.

Fig. 9. Unknown pose face alignment with feature points located (Left to right: iteration 0, 1, 5 and 9).

Fig. 10. Comparison of FTSMs for non-faces and faces. X-axis: index of frame; Y-axis: similarity measure outputs for faces and non-face objects.

Fig. 11. Face detection results. Row 1: varying pose face detection; row 2: partial occlusion; row 3: false detection eliminated (compared with Fig. 3).

Fig. 12. Feature registration results (rows 1–3: good registration; row 4: fair registration; row 5: mis-registration).

Fig. 13. Face registration results with complex scene.

Table 1. Pose parameters (first three components) and error measures

Table 2. Feature registration results––we define a good registration as having feature points at most 5% of face size away from correct positions, and a fair registration at most 10% of face size away, otherwise, a mis-registration (see Fig. 12)