Fig. 1. Overall architecture for motion tracking.

Fig. 2. Interior feature points flow, where feature i starts in frame 0 and disappears in frame k + 1, and feature j starts in frame k + 1 and is still active in current frame n.

Fig. 3. (A) Selection of boundary points by thresholding chord lengths. (B) First image in the video sequence. (C) Edge map of the first image. (D) Segmented object region and selected feature points. The interior points are marked + and the boundary points ×.

Fig. 4. Feature correspondence and motion estimation in frame-n for a group of feature points born in frame-k.

Fig. 5. Uncertainty prediction for feature extraction.

Fig. 6. (A) Original registered feature points represented by white “+”s in frame k, and the white polygons show the optimized matching templates. (B) Corresponding feature match in frame n, where the ellipses stand for predicted search regions and white polygons show NCC calculation at different candidate positions. (C) Calculated NCC map of the search region for point A. (D) NCC map after non-maximum suppression from (C).

Fig. 7. Flowchart for feature correspondence matching.

Fig. 8. Feature point prediction and extraction. The solid ellipses in frame n represent the predicted feature uncertainties from . The “×”s stand for the initial extracted features . The bold-dot ellipses demonstrate the new feature uncertainties projected by . The dash-dot ellipse illustrates the new feature uncertainty predicted by , and the “+” shows the re-extracted feature point for the initial invalid measurement.

Fig. 9. Feature observability and unobservability for m features initially appeared in frame k.

Fig. 10. Feature grouping based on genesis indices, where frame 47 is current processed frame with features initiated from frames 18, 26, 34,…Different marks of features denote features with different genesis indices.

Fig. 11. Sequential motion uncertainty updating by integrating multiple groups of features.

Fig. 12. Motion vector and shape vector updating.

Fig. 13. An example of boundary occlusion due to self-rotation. The nose of the toy is not visible to the camera in the left image.

Fig. 14. ROI boundary refining processing diagrams.

Fig. 15. (A) Predicted ROI boundary is shown as the white contour. (B) Boundary uncertainty field is enclosed by the black and gray contours.

Fig. 16. (A) Splitting of the eroded boundary . The alternate segments of the decomposition are shown in black and white. (B) Boundary partitioning after the merging step.

Fig. 17. A visual demonstration of growing of the regions starting from the partitioned boundary segments. The points marked “×”s and “+” represent the eroded boundary . The different segments obtained by recursive partitioning are shown as “×”s and “+”s. The accepted 8-neighbors during one iteration of the growing step are shown as filled circles and diamonds.

Fig. 18. (A) Grown region from the seed segments of Fig. 16B. (B) Final updated object boundary.

Fig. 19. Synthetic video sequence. (A) Frame 0. (B) Trajectories and uncertainty regions for all feature points.

(A) Estimated motion parameters added with Gaussion noise of 2 pixel standard deviation.

(B) Estimated motion parameters added with Gaussion noise of 4 pixel standard deviation.

(C) Estimated motion parameters added with Gaussion noise of 10 pixel standard deviation.

Fig. 20. Motion estimation performance evaluation with different standard deviations of the Gaussian noise model.

Fig. 21. (A) Frame 0 from the video sequence. (B) Human delineated toy object to be tracked. (This is the output of CISE.) (C) Automatically selected feature points in frame 0. (D) Predicted feature uncertainty regions for a few chosen features are displayed in red dotted ellipses for illustration.

(A) Frame 15.

(B) Frame 30.

(C) Frame 50.

(D) Frame 70.

(E) Frame 85.

(F) Frame 100.

(G) Frame 115.

(H) Frame 130.

(I) Frame 150.

Fig. 22. Object tracking in video 1. The black contours in the first, third, and the fifth rows show the estimated object boundaries. The white contours are the eroded version of initially predicted boundaries, used for boundary updating. The second, fourth, and the sixth rows show the extracted object from indicated frames.

Fig. 23. (A) Frame 0 from the video sequence. (B) Human delineated car object to be tracked. (C) Automatically selected feature points in frame 0. (D) Predicted feature uncertainty regions for a few chosen features.

Fig. 24. Un-represented regions during region growing process.

(A) Frame 5.

(B) Frame 10.

(C) Frame 30.

(D) Frame 40.

(E) Frame 50.

(F) Frame 60.

Fig. 25. Object tracking in video 2. The black contours in the first and the third rows show the estimated object boundaries. The white contours are the eroded version of initially predicted boundaries, used for boundary updating. The second and the forth rows show the extracted object from indicated frames.

(A) Frame 1.

(B) Frame 10.

(C) Frame 20.

(D) Frame 50.

(E) Frame 70.

(F) Frame 90.

Fig. 26. Object tracking in video 3. The black contours in the first and the third rows show the estimated object boundaries. The white contours are the eroded version of initially predicted boundaries, used for boundary updating. The second and the fourth rows show the extracted object from indicated frames.

(A) Frame 1.

(B) Frame 10.

(C) Frame 20.

(D) Frame 50.

(E) Frame 70.

(F) Frame 90.

Fig. 27. Video 3 sequence tracking without boundary updating.

(A) Frame 1.

(B) Frame 10.

(C) Frame 20.

(D) Frame 50.

(E) Frame 70.

(F) Frame 90.

(G) Frame 80.

(H) Frame 90.

(I) Frame 103.

Fig. 28. Object tracking in video 4. The black contours in the first, third, and the fifth rows show the estimated object boundaries. The white contours are the eroded version of initially predicted boundaries, used for boundary updating. The second, fourth, and the sixth rows show the extracted object from indicated frames.

(A) Frame 1.

(B) Frame 10.

(C) Frame 20.

(D) Frame 30.

(E) Frame 50.

(F) Frame 65.

(G) Frame 80.

(H) Frame 90.