Abstract

A framework for the logical and statistical analysis and annotation of dynamic scenes containing occlusion and other uncertainties is presented. This framework consists of three elements; an object tracker module, an object recognition/classification module and a logical consistency, ambiguity and error reasoning engine. The principle behind the object tracker and object recognition modules is to reduce error by increasing ambiguity (by merging objects in close proximity and presenting multiple hypotheses). The reasoning engine deals with error, ambiguity and occlusion in a unified framework to produce a hypothesis that satisfies fundamental constraints on the spatio-temporal continuity of objects. Our algorithm finds a globally consistent model of an extended video sequence that is maximally supported by a voting function based on the output of a statistical classifier. The system results in an annotation that is significantly more accurate than what would be obtained by by-frame evaluation of the classifier output. The framework has been implemented and applied successfully to the analysis of team sports with a single camera.

Keywords: Visual surveillance; Spatial reasoning; Temporal reasoning; Resolving ambiguity; Continuity

Article Outline

1. Introduction

2. An architecture for tracking and recognition with error correction based on consistency reasoning

3. The blob tracker

4. Object classification

4.1. Box classifier output

5. Ensuring spatio-temporal continuity

5.1. Coarse object grouping with ‘envelopes’

5.2. Enforcing exclusivity and continuity at the envelope level

5.3. Observational likelihood of box and envelope occupancy

5.4. Selecting the best hypothesis for an extended frame sequence

5.5. Off-Line implementation of the continuity reasoner

6. Evaluation

7. Discussion and future work

Acknowledgements

References