Elsevier

Pattern Recognition

Volume 59, November 2016, Pages 302-311
Pattern Recognition

Video anomaly detection based on locality sensitive hashing filters

https://doi.org/10.1016/j.patcog.2015.11.018Get rights and content

Highlights

  • We present a locality sensitive hashing filters based method for anomaly detection.

  • Normal activities are hashed by hash functions into buckets to build filters.

  • Abnormality of a test sample is estimated by filter response of its nearest bucket.

  • Online updating mechanism increase the adaptability to scene changes.

  • Searching for optimal hash functions improves the detection accuracy.

  • Our method performs favorably against previous anomaly detection algorithms.

Abstract

In this paper, we propose a novel anomaly detection approach based on Locality Sensitive Hashing Filters (LSHF), which hashes normal activities into multiple feature buckets with Locality Sensitive Hashing (LSH) functions to filter out abnormal activities. An online updating procedure is also introduced into the framework of LSHF for adapting to the changes of the video scenes. Furthermore, we develop a new evaluation function to evaluate the hash map and employ the Particle Swarm Optimization (PSO) method to search for the optimal hash functions, which improves the efficiency and accuracy of the proposed anomaly detection method. Experimental results on multiple datasets demonstrate that the proposed algorithm is capable of localizing various abnormal activities in real world surveillance videos and outperforms state-of-the-art anomaly detection methods.

Introduction

Intelligent video surveillance plays an irreplaceable role in safe city construction due to the ability of understanding and analyzing the monitoring contents using techniques such as computer vision. As an important part of intelligent video surveillance, video anomaly detection can automatically detect the abnormal events in the monitoring scene and produce alarms to assist the security officers to deal with the unexpected events.

There have been proposed various approaches [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11] for anomaly detection in recent years. Most methods follow the principle that abnormal events rarely occur are different from the normal training observations. Specifically, in [1], [12], [13], the normal training samples are employed to reconstruct the test samples, of which the samples with large reconstruction error will be considered as abnormal. It is worth mentioning that Zhao et al. [1] introduced the dictionary updating with new observations to adapt to the scene changes in testing phase.

Another kind of methods tries to build a normal activity model, and the behaviors of low compatibility with the model will be detected as anomalies. The Markov Random Field (MRF) model is utilized in [2], [14] to characterize the distribution of normal motion patterns or co-occurrence patterns, and the maximum a posteriori (MAP) is computed to estimate the abnormal degree of the test sample. In [3], [15], [16], [17] the Hidden Markov Model (HMM) is presented to describe the temporal and spatial relationship between and statistics of local motion patterns, and the confidence measure of the new observations is captured to reject anomalies. While Mehran et al. adopted social force model (SFM) in [4] to depict the individual motion dynamics and interaction forces in crowds for anomaly judgements. Chen et al. [11] decomposed a complex behavior using a cascade of Dynamic Bayesian Networks (CasDBNs) for the detection of subtle anomalies in surveillance videos. Hiroyuki et al. [18] proposed the Normality Sensitive Hashing (NSH) where a set of hash functions are selected that instances within the normal region are allocated into the same bucket while instances across the region boundary are assigned to different buckets.

Similarities of the test samples to the training data are computed in [19], [20], [21], [9], where the samples with low similarities will be given high abnormal scores in the test videos. Clustering [21] and sub-class discovering [20] were proposed to reduce the computation cost in training phase. Some other algorithms were also designed for anomaly detection such as scan statistics [7], chaotic invariants [6], multiple fixed-location monitors [22], and energy model based methods [10], [23], [24].

In this paper, we propose a novel framework based on Locality Sensitive Hashing Filters (LSHF) for anomaly detection in video scenes. To build the normal activity model, training videos are hashed by LSH functions into a list of buckets with each bucket represented as a miniature filter, and the abnormal degree of a test sample is estimated by its nearest filter. Consider that the scene context tends to change over time, we introduce an online updating mechanism, where the new normal behaviors will be added into the LSHF model and the outmoded miniature filters will be removed. Moreover, we develop a new evaluation function to evaluate the LSH functions and adopt the Particle Swarm Optimization (PSO) algorithm to search for the optimal hashing functions. The LSHF built by the optimal functions shows remarkable advantages by increasing the accuracy of anomaly detection. The overview of our approach is shown in Fig. 1.

It should be noted that the Normality Sensitive Hashing (NSH) [18] has also made improvements on LSH for anomaly detection, which selects a set of hash functions to define a normal region and instances across the region boundary are regarded as abnormal. However, the feature data usually scatter in the high-dimensional space, and the normal region surrounded by a number of hyperplanes may be so large that the abnormal points are easily incorporated, which may greatly reduce the detection rate. On the other hand, by selecting the hash functions minimizing the objective function from random candidates, the NSH may be unstable to get the optimal solution. In contrast, we build a list of buckets as a fine partition of the data, which can filter out abnormal events with higher detection rate. We estimate the anomalies by both considering the location and the distance from the point to the bucket, leading to higher accuracy and lower false alarm rate. In addition, we seek the optimal LSH functions with the PSO algorithm, which enhances the stability of the proposed method.

We summarize the contributions of this paper as follows:

  • We propose a novel anomaly detection algorithm based on Locality Sensitive Hashing Filters (LSHF), where the abnormal events are filtered out by the nearest hash buckets.

  • We present an online updating mechanism in LSHF to handle the scene variation, which is simple but effective in video streams.

  • We develop an evaluation function for LSH functions and employ the Particle Swarm Optimization (PSO) algorithm to search for the optimal hash functions, which helps improve the detection accuracy.

The remainder of this paper is organized as follows. In Section 2, we briefly introduce the related algorithms of Locality Sensitive Hashing (LSH) and bloom filter. Section 3 provides detailed demonstration of the proposed LSHF model for anomaly detection, followed by the PSO algorithm searching for optimal hash functions presented in Section 4. The experimental results on three public datasets illustrating the superiority of our approach are shown in Section 5. Finally we conclude our work in Section 6.

Section snippets

Locality Sensitive Hashing (LSH)

As an important technique for fast approximate similarity search, hashing has gained much popularity in recent years. For instance, the Minimal Loss Hashing (MLH) [25], Kernel-based Supervised Hashing (KSH) [26], Supervised Discrete Hashing (SDH) [27], and Bit-Scalable Deep Hashing (BSDH) [28], etc. Among various hashing methods, Locality Sensitive Hashing (LSH) is widely used for approximate nearest neighbor searching in large high-dimensional databases [29]. The key idea of LSH is to hash the

LSH filters for anomaly detection

In this section, we propose a novel method based on Locality Sensitive Hashing Filters (LSHF) for anomaly detection in video streams. Training data points are hashed by the LSH functions into a list of hash buckets where each bucket is represented as a miniature filter, then a given test sample is hashed in the same way into the test bucket, the nearest filter to the test bucket is utilized to compute the abnormal degree of the test sample.

Searching for optimal LSH functions

The LSH algorithm finds the neighbor points via hashing the data points into a list of buckets in the Hamming space. The “good” hash functions are capable of mapping the similar data points into the same bucket, however, the “bad” hash functions may lead to undesirable mapping results where the points far apart are hashed into the same bucket, which may reduce the accuracy of neighbors searching especially for datasets with non-uniform distributions. Therefore, the original LSH algorithm adopts

Experiments

In this section, we show the performance of our approach on the UCSD dataset [5], the Subway dataset [22], the UMN dataset [38], and the car videos captured from moving cameras. Experimental results show that the proposed method is effective for abnormal activities detection in various scenes.

The proposed algorithm is also compared with 10 state-of-the-art methods of SS13 [7], SC13 [13], NSH13 [18], SR11 [12], OSC11 [1], MDT10 [5], CI10 [6], SF09 [4], MPPCA09 [2], MFLM08 [22]. The comparison

Conclusion

In this paper we proposed a novel method based on locality sensitive hashing filters to detect abnormal events in video surveillance. The training samples are hashed into a list of buckets and the center and radius of each bucket are computed to build locality sensitive hashing filters. The abnormality degree of a new test sample is estimated by calculating the filter response of the test sample to its nearest filter. Furthermore, the locality sensitive hashing filters are online updated with

Conflict of interest

None declared.

Ying Zhang received her B.E. degree in Electronics and Information Engineering and M.Sc. degree in Electronics and Communication Engineering, Dalian University of Technology (DUT), China, in 2013 and 2015 respectively. She is currently a Ph.D. student in Signal and Information Processing, Dalian University of Technology (DUT). Her research interest is in saliency detection and anomaly detection.

References (38)

  • C.C. Loy et al.

    Detecting and discriminating behavioural anomalies

    Pattern Recognit.

    (2011)
  • K. Ouivirach et al.

    Incremental behavior modeling and suspicious activity detection

    Pattern Recognit.

    (2013)
  • D. Gorisse et al.

    SALSASsub-linear active learning strategy with approximate k-nn search

    Pattern Recognit.

    (2011)
  • B. Zhao, F. Li, E.P. Xing, Online detection of unusual events in videos via dynamic sparse coding, in: Proceedings of...
  • J. Kim, K. Grauman, Observe locally, infer globally: a space–time mrf for detecting abnormal activities with...
  • E.L. Andrade, S. Blunsden, R.B. Fisher, Modelling crowd scenes for event detection, in: Proceedings of IEEE...
  • R. Mehran, A. Oyama, M. Shah, Abnormal crowd behavior detection using social force model, in: Proceedings of IEEE...
  • V. Mahadevan, W. Li, V. Bhalodia, N. Vasconcelos, Anomaly detection in crowded scenes, in: Proceedings of IEEE...
  • S. Wu, B.E. Moore, M. Shah, Chaotic invariants of lagrangian particle trajectories for anomaly detection in crowded...
  • Y. Hu, Y. Zhang, L.S. Davis, Unsupervised abnormal crowd activity detection using semiparametric scan statistic, in:...
  • H. Zhong, J. Shi, M. Visontai, Detecting unusual activity in video, in: Proceedings of IEEE Conference on Computer...
  • V. Saligrama, Z. Chen, Video anomaly detection based on local statistical aggregates, in: Proceedings of IEEE...
  • X. Cui, Q. Liu, M. Gao, D.N. Metaxas, Abnormal detection using interaction energy potentials, in: Proceedings of IEEE...
  • Y. Cong, J. Yuan, J. Liu, Sparse reconstruction cost for abnormal event detection, in: Proceedings of IEEE Conference...
  • C. Lu, J. Shi, J. Jia, Abnormal event detection at 150 FPS in MATLAB, in: Proceedings of IEEE International Conference...
  • Y. Benezeth, P. Jodoin, V. Saligrama, C. Rosenberger, Abnormal events detection based on spatio-temporal co-occurences,...
  • L. Kratz, K. Nishino, Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models, in:...
  • D. Zhang, D. Gatica-Perez, S. Bengio, I. McCowan, Semi-supervised adapted hmms for unusual event detection, in:...
  • H. Hachiya, M. Matsugu, Nsh: normality sensitive hashing for anomaly detection, in: Proceedings of IEEE International...
  • Cited by (0)

    Ying Zhang received her B.E. degree in Electronics and Information Engineering and M.Sc. degree in Electronics and Communication Engineering, Dalian University of Technology (DUT), China, in 2013 and 2015 respectively. She is currently a Ph.D. student in Signal and Information Processing, Dalian University of Technology (DUT). Her research interest is in saliency detection and anomaly detection.

    Huchuan Lu (SM׳12) received the M.Sc. degree in Signal and Information Processing, and Ph.D. degree in System Engineering, Dalian University of Technology (DUT), China, in 1998 and 2008 respectively. He has been a faculty since 1998 and a professor since 2012 in the School of Information and Communication Engineering of DUT. His research interests are in the areas of computer vision and pattern recognition. In recent years, he focuses on visual tracking and segmentation. Now, he serves as an associate editor of the IEEE Transactions on Systems, Man, and Cybernetics: Part B.

    Lihe Zhang received the Ph.D. degree in signal and information processing from Beijing University of Posts and Telecommunications, Beijing, China, in 2004. He is currently an associate professor with the School of Information and Communication Engineering, Dalian University of Technology. His research interests include pattern recognition and computer vision.

    Xiang Ruan received the B.E. degree from Shanghai Jiao Tong University, Shanghai China, in 1997, and the M.E. and Ph.D. degrees from Osaka City University, Osaka, Japan, in 2001 and 2004, respectively. He is currently a research engineer with OMRON Corporation, Kyoto, Japan. His current research interests include computer vision, machine learning, and image processing.

    Shun Sakai received the B.E degree from Tokushima University, Tokushima, Japan, in 2007, and Master degrees from Tokushima University, Tokushima, Japan, in 2009. He is currently research engineer with EMC AOB of OMRON Corporation, Kyoto, Japan. His research interests include computer vision, machine learning and image processing.

    View full text