Combining patch matching and detection for robust pedestrian tracking in monocular calibrated cameras

Highlights

• We found a way of using the information of a pedestrian detector in a WVMF framework.

• Motion prediction and detections can help the tracker recover from failure.

• We compare different methods for pedestrian tracking in publicly available videos.

• Our method outperforms some of the current state-of-the-art trackers.

Abstract

This paper presents a new approach for tracking multiple people in monocular calibrated cameras combining patch matching and pedestrian detection. Initially, background removal and pedestrian detection are used in conjunction with the vertical standing hypothesis to initialize the targets with multiples patches. In the tracking step, each patch related to a given target is matched individually across frames, and their translation vectors are combined robustly with pedestrian detection results in the world coordinate frame using weighted vector median filters. Additionally, the algorithm uses the camera parameters to both estimate the person scale in a straightforward manner and to limit the search region used to track each fragment. Our experimental results indicate that our tracker can deal with occlusions and video sequences with strong appearance variations, presenting results comparable to or better than existing state-of-the-art algorithms.

Introduction

Object tracking is an active research topic in the computer vision community. In particular, pedestrian tracking represents an important task in a wide range of applications, such as video analytics, surveillance and analysis of athletic performance in collective sports.

The development of a complete framework for pedestrian tracking involves a variety of challenges. In the initialization step, each new pedestrian that enters the scene must be detected. In general, background removal and/or pedestrian detection algorithms are used in this stage, and they are strongly affected by illumination changes, shadows and varying poses. The tracking phase consists of constantly localizing a pedestrian across time, and it is a complex task due to several reasons: people can move fast and unpredictably, the subject’s appearance can change throughout the sequence and the video can present noise and blur, among others. In a common surveillance scenario, the problem can be even harder because of the many occlusions (between a group of people or the person and scene) that can occur. Additionally, these systems often require on-the-fly execution, so non-causal methods that use future information to estimate the current state are not well suited for these applications.

Instead of considering the target as a whole, some approaches split the target into multiple fragments (Adam et al., 2006, Dihl et al., 2011, Führ and Jung, 2012). These approaches have been shown to increase robustness in the presence of partial occlusions, since non-occluded fragments can be matched correctly. This paper extends the fragments-based approach described in Führ and Jung (2012) for pedestrian tracking, by including several new features:

• Multiple people tracking: in this paper, we present an automatic procedure for target initialization based on background removal, pedestrian detection and a vertical standing hypothesis, so that several targets can be detected and tracked simultaneously.

• Use of patch matching and pedestrian detection to improve robustness: the information from a pedestrian detection algorithm is included in the tracker, increasing accuracy and helping to recover the target after occlusions.

• Inclusion of a predicted position based on the history of movement of each target, which provides smoother tracks and helps the tracker during occlusions.

The remainder of this paper is organized as follows. Section 2 reviews some relevant work on object tracking, focusing on pedestrians. The proposed approach is described in Section 3, and the experimental validation is presented in Section 4. Finally, conclusions are drawn in Section 5.