Abnormality detection using low-level co-occurring events
Research highlights
► A method for behavior modeling and abnormal events detection which uses lowlevel features. ► The notion of co-occurring events is defined using Mutual Information. ► The co-occurrence matrix is used as a potential function in a Markov random field framework to describe the probability of observing new volumes of activity. ► The Markov random field implicitly accounts for speed, direction and average size of the objects without any higher-level intervention.
Introduction
In this paper, we present a low-level location-based approach for activity analysis and abnormal detection. In several traditional approaches (e.g. Hu et al., 2004), moving objects are first detected, analyzed and then tracked. Subsequently, behavior models are built based on object tracks and non-conformant ones are deemed abnormal. The main problem with this approach is that in case of complex environments, object extraction and tracking are performed directly on cluttered raw video or motion labels. We propose performing activity analysis and abnormal behavior detection first, followed possibly by object extraction and tracking. If the abnormal activity is reliably identified, then object extraction and tracking focus on region of interest (ROI) and thus is relatively straightforward. A question arises: How to reliably identify abnormalities from a raw video?
Some approaches have been proposed to perform such low-level abnormality detection (Adam et al., 2008, Jodoin et al., 2008). Nevertheless, we point out that these methods process each pixel independently and thus ignore spatial correlation across space and time. These correlations may not only be important in improving false alarms and misses but also in detecting abnormality of event sequences, such as a person in the act of dropping a baggage or a car making an illegal u-turn, etc. In our method, we account for these scenarios through spatio-temporal models. Although this model is simple, it nonetheless produces interesting results.
Section snippets
Previous work
Video analytics can be divided into two broad families of approaches namely shape/pattern-recognition-based methods and the machine-learning-based methods. The shape/pattern recognition approaches are typically those for which the type of activity or object is known a priori. Examples of such methods include facial recognition systems (Zhao et al., 2003, Hu et al., 2009), restricted-area access detection (Konrad, 2005), car counting (Friedman and Russell, 1997), detection of people carrying
Context
Although many video analytics methods use motion labels only in early stages of processing (mainly to locate moving objects) we argue that they carry fundamental information on the content of the scene and thus, can be used to perform high-level tasks. Motivated by this perspective, some authors have already shown that low-level motion labels can be used to summarize videos (Pritch et al., 2008), recognize human movements (Bobick and Davis, 2001) and detect abnormalities (Jodoin et al., 2008).
Our method
In this section, we present how, for a given site s, a co-occurrence matrix and its associated statistical model can be estimated from a training video sequence. Our statistical model is a Markov-random field (MRF) model that accounts for the likelihood of the co-occurrences. Since we account for normal scenarios in which objects follow typical paths, these paths manifest themselves as spatio-temporal dependencies across pixels as shown in Eq. (1). Our location-based approach for modeling
Experimental results
We present in this section some results obtained on various indoor and outdoor sequences representing different challenges. For each sequence, a co-occurrence matrix of size ranging between 130 × 70 × 300 and 210 × 210 × 150 have been used. The size of the co-occurrence matrix is chosen so that a typical normal activity is entirely included in the volume. The reader shall note that since the matrix’ size stay fix for the entire process, it has to be fixed only once while setting up the system. The
Conclusion
We propose in this paper a method to perform behavior modeling and abnormality detection based on low-level characteristics. We use the spatial and temporal dependencies between motion labels vectors obtained with simple background subtraction. To do this, we built a Markov Random Field model parameterized by a co-occurrence matrix. Although simple, this matrix contains the average behavior observed in a training sequence. It also implicitly contains information about direction, speed and size
References (31)
- et al.
Human action recognition using shape and CLG-motion flow from multi-view image sequences
Pattern Recognition
(2008) Learning and understanding dynamic scene activity: a review
Image and Vision Computing
(2003)- et al.
Direct kernel neighborhood discriminant analysis for face recognition
Pattern recognition letters
(2009) Motion detection and estimation
Minimum volume sets and generalized quantile processes
Stochastic Processes and Applications
(1997)- et al.
Robust real-time unusual event detection using multiple fixed-location monitors
Transactions on Pattern Analysis and Machine Intelligence
(2008) - et al.
Abnormal events detection based on spatio-temporal co-occurences
International Conference on Computer Vision and Pattern Recognition
(2009) - et al.
Comparative study of background subtraction algorithm
Journal of Electronic Imaging
(2010) - et al.
The recognition of human movement using temporal templates
Transactions on Pattern Analysis and Machine Intelligence
(2001) - et al.
Detecting irregularities in images and in video
International Journal on Computer Vision
(2007)