Skip to main content
SpringerLink
Log in

Smoke vehicle detection based on multi-feature fusion and hidden Markov model

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Existing smoke vehicle detection methods and vision-based smoke detection methods are vulnerable to false alarms. This paper presents an automatic smoke vehicle detection method based on multi-feature fusion and hidden Markov model (HMM). In this method, we first detect moving objects using an improved visual background extractor (ViBe) algorithm and obtain smoke-colored blocks using color histogram features in the HSI (hue, saturation, and intensity) color space. The adaptive scale local binary pattern (AS-LBP) and the discriminative edge orientation histogram (disEOH) are proposed and combined to characterize the smoke-colored blocks. More specifically, the proposed AS-LBP, a texture feature descriptor, is based on the quadratic fitting of our labelled data to obtain the best scale. The proposed disEOH, a gradient-based feature descriptor, is robust to noise by extracting discriminative edge information using Gaussian filters and principal component analysis (PCA). The discrete cosine transform (DCT) is employed to extract frequency domain information from the region fused by smoke blocks. To utilize the dynamic features, the HMMs are employed to analyze and classify the smoke-colored block sequences and region sequences in continuous frames. The experimental results show that the proposed method achieves better performances than existing smoke detection methods, especially achieves lower false alarms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Liu, Y.H., Liao, W.Y., Li, L., et al.: Vehicle emission trends in China’s Guangdong Province from 1994 to 2014. Sci. Total Environ. 3(15), 512–521 (2017)

    Article  Google Scholar 

  2. Asano, I., Shinohara, M., Hamada, K.: Exhaust gas analysis system and exhaust gas analysis program, U.S. Patent 9 568 411 B2, Feb. 14, (2017)

  3. Liu, H., Chen, S., Kubota, N.: Intelligent video systems and analytics: a survey. IEEE Trans. Ind. Inf. 9(3), 1222–1233 (2013)

    Article  Google Scholar 

  4. Pyykonen, P., Peussa, P., Kutila, M., et al.: Multi-camera-based smoke detection and traffic pollution analysis system. Proc. Int. Conf. Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, 2016, pp. 233–238

  5. Tao, H., Lu, X.: Smoke vehicle detection based on multi-scale block Tamura features. Signal Image Video Process. 12(6), 1061–1068 (2018)

    Article  Google Scholar 

  6. Tao, H., Lu, X.: Smoke vehicle detection based on multi-feature fusion and ensemble neural networks. Multimed. Tools Appl. 77(24), 32153–32177 (2018)

    Article  Google Scholar 

  7. Tao, H., Lu, X.: Smoke vehicle detection in surveillance video based on gray level co-occurrence matrix. in Proc. Int. Conf. on Digital Image Processing, Shanghai, SPIE, vol. 10806, id.1080642, pp. 1–7, Aug. 2018

  8. Tao, H., Lu, X.: Automatic smoky vehicle detection from traffic surveillance video based on vehicle rear detection and multi-feature fusion” IET Intel. Transport Syst. (2018). https://doi.org/10.1049/iet-its.2018.5039

    Article  Google Scholar 

  9. Tao, H., Lu, X.: Contour-based smoke vehicle detection from surveillance video for alarm systems. SIViP. (2018). https://doi.org/10.1007/s11760-018-1348-z

    Article  Google Scholar 

  10. Tao, H., Lu, X.: Smoky vehicle detection based on range filtering on three orthogonal planes and motion orientation histogram. IEEE Access. 6(1), 57180–57190, (2018)

    Google Scholar 

  11. Saponara, S., Pilato, L., Fanucci, L.: Early video smoke detection system to improve fire protection in rolling stocks. Proc. SPIE 9139(913903), 9 (2014)

    Google Scholar 

  12. Saponara, S., Pilato, L., Fanucci, L.: Exploiting CCTV camera system for advanced passenger services on-board trains. IEEE Int. Smart Cities Conf. pp. 1–6 (2016)

  13. Gunay, O., Toreyin, B.U., Kose, K., et al.: ‘Entropy-functional-based online adaptive decision fusion framework with application to wildfire detection in video. IEEE Trans. Image Process. 21(5), 2853–2865 (2012)

    Article  MathSciNet  Google Scholar 

  14. Kolesov, I., Karasev, P., Tannenbaum, A., et al.: ‘Fire and smoke detection in video with optimal mass transport based optical flow and neural networks,” in Proc. IEEE International Conference on Image Processing, 2010, pp. 761–764

  15. Wang, S., He, Y., Yang, H., et al.: Video smoke detection using shape, color and dynamic features. J. Intell. Fuzzy Syst. 33(1), 305–313 (Feb. 2017)

    Article  Google Scholar 

  16. Calderara, S., Piccinini, P., Cucchiara, R.: Vision based smoke detection system using image energy and color information. Mach. Vis Appl. 22(4), 705–719, (2011)

    Article  Google Scholar 

  17. Jakovcevic, T., Stipanicev, D., Krstinic, D.: Visual spatial-context based wildfire smoke sensor. Mach. Vis. Appl. 24(4), 707–719 (2013)

    Article  Google Scholar 

  18. Millan-Garcia, L., Sanchez-Perez, G., Nakano, M., et al.: An early fire detection algorithm using IP cameras. Sensors 12(5), 5670–5686 (2012)

    Article  Google Scholar 

  19. Prema, C.E., Vinsley, S.S., Suresh, S.: Multi feature analysis of smoke in YUV color space for early forest fire detection. Fire Technol 52(5), 1319–1342 (2016)

    Article  Google Scholar 

  20. Ugur-Töreyin, B., Dedeoglu, Y., Enis-Çetin, A.: Contour Based smoke detection in video using wavelets. Proceedings of European Signal Processing Conference; Florence, Italy. 4–8 September 2006

  21. Yu, C., Faon, J., Wang, J., et al.: Video fire smoke detection using motion and color features. Fire Technol 46, 651–663 (2010)

    Article  Google Scholar 

  22. Ko, B., Park, J., Nam, J.Y.: Spatiotemporal bag-of-features for early wildfire smoke detection. Image Vis. Comput. 31(10), 786–795, (2013)

    Article  Google Scholar 

  23. Toreyin, B.U., Dedeoglu, Y., Cetin, A.E.: Wavelet based real-time smoke detection in video. in Proc. European Signal Processing Conference, (2005)

  24. Wang, Y., Chua, T.W., Chang, R., et al.: Real-time smoke detection using texture and color features. In Proc. International Conference on Pattern Recognition, pp. 1727–1730 (2012)

  25. Tian, H., Li, W., Ogunbona, P., et al.: Smoke detection in videos using non-redundant local binary pattern-based features. In Proc. IEEE International Workshop on Multimedia Signal Processing, pp. 1–4 (2011)

  26. Yuan, F.: Video-based smoke detection with histogram sequence of LBP and LBPV pyramids. Fire Saf. J. 46(3), 132–139, (2011)

    Article  Google Scholar 

  27. Lin, G., Zhang, Y., Zhang, Q., et al.: Smoke detection in video sequences based on dynamic texture using volume local binary patterns. Ksii Trans. Internet Inf. Syst. 11(11), 5522–5536 (2017)

    MathSciNet  Google Scholar 

  28. Favorskaya, M., Pyataeva, A., Popov, A.: Verification of smoke detection in video sequences based on spatio-temporal local binary patterns. Proc. Comput. Sci. 60(1), 671–680 (2015)

    Article  Google Scholar 

  29. Yuan, F., Shi, J., Xia, X., et al.: High-order local ternary patterns with locality preserving projection for smoke detection and image classification. Inf. Sci. 372(C), 225–240 (2016)

    Article  Google Scholar 

  30. Datondji, S.R.E., Dupuis, Y., Subirats, P.: A survey of vision-based traffic monitoring of road intersections. IEEE Trans. Intell. Transp. Syst. 17(10), 2681–2698 (2016)

    Article  Google Scholar 

  31. Barnich, O., Droogenbroeck, M.V.: ViBe: a universal background subtraction algorithm for video sequences. IEEE Trans. Image Process. 20(6), 1709–1724 (2011)

    Article  MathSciNet  Google Scholar 

  32. Ojala, T., Pietikainen, M., Maenpaa, T.T.: Multiresolution gray-scale and rotation invariant texture classification with local binary pattern. IEEE Trans. Pattern Analysis Mach. Intell. 7(24), 971–987 (2002)

    Article  Google Scholar 

  33. Li, Z., Liu, G., Yang, Y., et al.: Scale- and rotation-invariant local binary pattern using scale-adaptive texton and subuniform-based circular shift. IEEE Trans. Image Process 21(4), 2874–2886 (2012)

    MathSciNet  MATH  Google Scholar 

  34. Guo, Z.H., Zhang, L., Zhang, D.: Rotation invariant texture classification using LBP variance (LBPV) with global matching. Pattern Recognit 43(3), 706–719 (2010)

    Article  Google Scholar 

  35. Guo, Z., Zhang, L., Zhang, D.: A completed modeling of local binary pattern operator for texture classification. IEEE Trans. Image Process. 19(6), 1657 (2010)

    Article  MathSciNet  Google Scholar 

  36. Thanh, N.D., Ogunbona, P.O., Li, W.: A novel shape-based non-redundant local binary pattern descriptor for object detection. Pattern Recogn. 46(5), 1485–1500 (2013)

    Article  Google Scholar 

  37. Zhao, G., Ahonen, T., Matas, J., et al.: Rotation-invariant image and video description with local binary pattern features. IEEE Trans. Image Process. 21(4), 1465–1477 (2012)

    Article  MathSciNet  Google Scholar 

  38. Zhu, C., Wang, R.: Local multiple patterns based multiresolution gray-scale and rotation invariant texture classification. Inf. Sci. 187(1), 93–108 (2012)

    Article  Google Scholar 

  39. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  40. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings IEEE International Conference Computer Vision and Pattern Recognition, 2005, Vol. 1, pp. 886–893

  41. Levi, K., Weiss Y.: Learning object detection from a small number of examples: the importance of good features. Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit. 2, 53–60

  42. Baum, E., Petrie, T., Soules, G., Weiss, N.: A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Stat. 41, 164–171 (1970)

    Article  MathSciNet  Google Scholar 

  43. Ronao, C., Ann, Cho, S.B.: Human activity recognition using smartphone sensors with two-stage continuous hidden Markov models. Int. Conf. Nat. Comput. IEEE. 681–686 (2014)

  44. Hu, J., Brown, M.K., Turin, W.: HMM based on-line hand-writing recognition. IEEE Trans. Pattern Anal. Mach. Intell. 18(10), 1039–1045 (1996)

    Article  Google Scholar 

  45. Lee, L.M., Jean, F.R.: High-order hidden Markov model for piecewise linear processes and applications to speech recognition. J. Acoust. Soc. Am. 140(2), EL204 (2016)

    Article  Google Scholar 

  46. Lawrence, R., Rabiner, A.: Tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989)

    Article  Google Scholar 

  47. Yuan, F.: A double mapping framework for extraction of shape-invariant features based on multi-scale partitions with AdaBoost for video smoke detection. Pattern Recogn. 45(12), 4326–4336 (2012)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 61871123), Key Research and Development Program in Jiangsu Province (No. BE2016739), a Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions, the Postgraduate Research and Practice Innovation Program of Jiangsu Province (No. KYCX18_0101), the Scientific Research Foundation of Graduate School of Southeast University (No. YBPY1871), and the State Scholarship Fund from China Scholarship Council.

Author information

Authors and Affiliations

  1. School of Automation, Southeast University, Nanjing, 210096, China

    Huanjie Tao & Xiaobo Lu

  2. Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education, Southeast University, Nanjing, 210096, China

    Huanjie Tao & Xiaobo Lu

Authors
  1. Huanjie Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaobo Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaobo Lu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tao, H., Lu, X. Smoke vehicle detection based on multi-feature fusion and hidden Markov model. J Real-Time Image Proc 17, 745–758 (2020). https://doi.org/10.1007/s11554-019-00856-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-019-00856-z

Keywords

Search

Navigation