Dimitrios Kosmopoulos
ABSTRACT
We propose a novel framework for online analysis of visual structured processes, using fusion from multiple cameras. Online recognition is performed through particle filters supported by hidden Markov models. We evaluate three fusion methods, an early fusion, a simple multiplication of the observation probabilities and a multi-stream one implying cross-stream coupling of observations and states. The performance is thoroughly evaluated under two complex visual behavior understanding scenarios: a visual process for table preparation in a kitchen and a real life manufacturing process in an industrial plant. The obtained results are compared and discussed.
- D. Arnaud, G. Simon, and A. Christophe. On sequential monte carlo sampling methods for bayesian filtering. Statistics and Computing, 10(3):197--208, 2000. Google Scholar
Digital Library
- M. S. Arulampalam, S. Maskell, and N. Gordon. A tutorial on particle filters for online nonlinear/non-gaussian bayesian tracking. IEEE Transactions on Signal Processing, 50(2):174--188, 2002. Google ScholarDigital Library
- K. Bernardin, T. Gehrig, and R. Stiefelhagen. Multimodal technologies for perception of humans. chapter Multi-level Particle Filter Fusion of Features and Cues for Audio-Visual Person Tracking, pages 70--81. Springer-Verlag, Berlin, Heidelberg, 2008. Google ScholarDigital Library
- Y. Chen and Y. Rui. Real-time speaker tracking using particle filter sensor fusion. Proceedings of the IEEE, 92(3): 485--494, mar 2004.Google ScholarCross Ref
- S. Eickeler, A. Kosmala, and G. Rigoll. Hidden markov model based continuous online gesture recognition. In In Int. Conference on Pattern Recognition (ICPR, pages 1206--1208, 1998. Google ScholarDigital Library
- H. Fei. A hybrid hmm/particle filter framework for non-rigid hand motion recognition. In Acoustics, Speech, and Signal Processing, 2004. Proceedings. (ICASSP '04). IEEE International Conference on, volume 5, pages V -- 889--92 vol.5, may 2004.Google Scholar
- S. Fine, Y. Singer, and N. Tishby. The hierarchical hidden markov model: Analysis and applications. Machine Learning, 32(1):41--62, 1998. Google ScholarDigital Library
- G. Gravier, G. Potamianos, and C. Neti. Asynchrony modeling for audio-visual speech recognition. In Proceedings of the second international conference on Human Language Technology Research, HLT '02, pages 1--6, 2002. Google ScholarDigital Library
- D. Kosmopoulos and S. Chatzis. Robust visual behavior recognition. Signal Processing Magazine, IEEE, 27(5):34--45, sep. 2010.Google ScholarCross Ref
- D. Kosmopoulos, A. Voulodimos, and T. Varvarigou. Robust human behavior modeling from multiple cameras. In Pattern Recognition (ICPR), 2010 20th 697 International Conference on, pages 3575--3578, 2010. Google ScholarDigital Library
- D. I. Kosmopoulos, N. D. Doulamis, and A. S. Voulodimos. Bayesian filter based behavior recognition in workflows allowing for user feedback. Computer Vision and Image Understanding, 116(3):422--434, 2011. Google ScholarDigital Library
- F. Lv and R. Nevatia. Recognition and segmentation of 3-d human action using hmm and multi-class adaboost. In ECCV06, pages IV: 359--372, 2006. Google ScholarDigital Library
- A. Nefian, L. Liang, X. Pi, L. Xiaoxiang, C. Mao, and K. Murphy. A coupled HMM for audio-visual speech recognition. In Acoustics, Speech, and Signal Processing, 2002. Proceedings. (ICASSP '02). IEEE International Conference on, volume 2, pages 2013--2016, 2002.Google Scholar
- N. Oliver, A. Garg, and E. Horvitz. Layered representations for learning and inferring office activity from multiple sensory channels. Comput. Vis. Image Underst., 96(2):163--180, 2004. Google ScholarDigital Library
- N. Padoy, D. Mateus, D. Weinland, M.-O. Berger, and N. Navab. Workflow Monitoring based on 3D Motion Features. In Workshop on Video-Oriented Object and Event Classification in Conjunction with ICCV 2009, pages 585--592, Kyoto Japan, 2009. IEEE.Google ScholarCross Ref
- L. R. Rabiner. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257--286, 1989.Google ScholarCross Ref
- D. G. Stork and M. E. Hennecke. Speech reading by humans and machines. In NATO ASI Series F, volume 150. Springer Verlag, 1996.Google ScholarCross Ref
- M. Tenorth, J. Bandouch, and M. Beetz. The TUM Kitchen Data Set of Everyday Manipulation Activities for Motion Tracking and Action Recognition. In IEEE Int. Workshop on Tracking Humans for the Evaluation of their Motion in Image Sequences (THEMIS). In conjunction with ICCV2009, 2009.Google ScholarCross Ref
- C. Vogler and D. Metaxas. A framework for recognizing the simultaneous aspects of American sign language. Computer Vision and Image Understanding, 81(358--384), 2001. Google ScholarDigital Library
- A. Voulodimos, D. Kosmopoulos, G. Vasileiou, E. Sardis, V. Anagnostopoulos, C. Lalos, A. Doulamis, and T. Varvarigou. A threefold dataset for activity and workflow recognition in complex industrial environments. MultiMedia, IEEE, 19(3):42--52, July 2012. Google ScholarDigital Library
- X. Xiaoling and L. Layuan. Real time analysis of situation events for intelligent surveillance. In Computational Intelligence and Design, 2008. ISCID '08. International Symposium on, volume 2, pages 122--125, oct. 2008. Google ScholarDigital Library
- Z. Zeng, J. Tu, B. M. P. Jr., and T. S. Huang. Audio--visual affective expression recognition through multistream fused HMM. IEEE Trans. Multimedia, 10(4):570--577, 2008. Google ScholarDigital Library
- D. Zhang, X. Ning, and X. Liu. Smc method for online prediction in hidden markov models. Kybernetes, 38(10):1819--1827, 2009.Google ScholarCross Ref
Index Terms
- Multicamera fusion for online analysis of structured processes
Recommendations
Bayesian Tree-Structured Image Modeling
SSIAI '00: Proceedings of the 4th IEEE Southwest Symposium on Image Analysis and InterpretationWavelet-domain hidden Markov models have proven to be useful tools for statistical signal and image processing. The hidden Markov tree (HMT) model captures the key features of the joint statistics of the wavelet coefficients of real-world data. One ...
Semi-hidden Markov models for generation and analysis of sequences
In this work a new kind of stochastic model is presented, the semi-hidden Markov model (SHMM). The proposed model is related to the hidden Markov model (HMM), and it is called semi-hidden because generated sequences need less information than HMM ...
Coding with partially hidden Markov models
DCC '95: Proceedings of the Conference on Data CompressionPartially hidden Markov models (PHMM) are introduced. They are a variation of the hidden Markov models (HMM) combining the power of explicit conditioning on past observations and the power of using hidden states. (P)HMM may be combined with arithmetic ...
Comments