Skip to main content
SpringerLink
Log in

Real-time human action recognition based on depth motion maps

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

This paper presents a human action recognition method by using depth motion maps (DMMs). Each depth frame in a depth video sequence is projected onto three orthogonal Cartesian planes. Under each projection view, the absolute difference between two consecutive projected maps is accumulated through an entire depth video sequence forming a DMM. An l 2-regularized collaborative representation classifier with a distance-weighted Tikhonov matrix is then employed for action recognition. The developed method is shown to be computationally efficient allowing it to run in real-time. The recognition results applied to the Microsoft Research Action3D dataset indicate superior performance of our method over the existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Schuldt, C., Laptev, I., Caputo, B.: Recognition human actions: a local SVM approach. Proceedings of IEEE International Conference on Pattern Recognition, vol. 3, pp. 32–36, Cambridge, UK (2004)

  2. Dollar, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp. 65–72, Beijing, China (2005)

  3. Sun, J., Wu, X., Yan, S., Cheong, L.F., Chua, T., Li, J.: Hierarchical spatio-temporal context modeling for action recognition. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2004–2011, Miami, FL (2009)

  4. Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8, Anchorage, AK (2008)

  5. Bobick, A., Davis, J.: The recognition of human movement using temporal templates. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 3, pp. 257–267 (2001)

  6. Davis, J.: Hierarchical motion history images for recognizing human motion. Proceedings of IEEE Workshop on Detection and Recognition of Events in Video, pp. 39–46, Vancouver, BC (2001)

  7. Xia, L., Chen, C., Aggarwal, J.-K.: View invariant human action recognition using histograms of 3D joints. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 20–27, Providence, RI (2012)

  8. Yang X., Tian, Y.: Eigen joints-based action recognition using Naïve-Bayes-Nearest-Neighbor. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 14–19, Province, RI (2012)

  9. Li, W., Zhang, Z., Liu, Z.: Action recognition based on a bag of 3D points. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 9–14, San Francisco, CA (2010)

  10. Yang, X., Zhang, C., Tian, Y.: Recognizing actions using depth motion maps-based histograms of oriented gradients. Proceedings of ACM International Conference on Multimedia, pp. 1057–1060, Nara, Japan (2012)

  11. Wang, J., Liu, Z., Chorowski, J., Chen, Z., Wu, Y.: Robust 3D action recognition with random occupancy patterns. Proceedings of IEEE European Conference on Computer Vision, pp. 872–885, Florence, Italy (2012)

  12. Vieira, A., Nascimento, E., Oliveira, G., Liu, Z., Campos, M.: Stop: Space-time occupancy patterns for 3D action recognition from depth map sequences. Iberoamerican Congress on Pattern Recognition, pp. 252–259, Buenos Aires, Argentina (2012)

  13. Jiang, W., Liu, Z., Wu, Y., Yuan, J.: Mining actionlet ensemble for action recognition with depth cameras. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1290–1297, Province, RI (2012)

  14. Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-time human pose recognition in parts from single depth images. Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1297–1304, Colorado springs, CO (2011)

  15. Wright, J., Yang, A., Ganesh, A., Sastry, S., Ma, Y.: Robust face recognition via sparse representation. IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 210–227 (2009)

  16. Wright, J., Ma, Y.: Dense error correction via l1 minimization. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 3033–3036, Taipei, Taiwan (2009)

  17. Wright, J., Ma, Y., Mairal, J., Sapiro, G., Huang, T., Yan, S.: Sparse representation for computer vision and pattern recognition. Proceedings of the IEEE, vol. 98, no. 6, pp. 1031–1044 (2010)

  18. Gao, S., Tsang, I.W.-H., Chia, L.: Kernel sparse representation for image classification and face recognition. Proceedings of IEEE European Conference on Computer Vision, pp. 1–14, Crete, Greece (2010)

  19. Yang, J., Yu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1794–1801, Miami, FL (2009)

  20. Zhang, L., Yang, M., Feng, X.: Sparse representation or collaborative representation: which helps face recognition? Proceedings of IEEE International Conference on Computer Vision, pp. 471–478, Barcelona, Spain (2011)

  21. Shi, Q., Eriksson, A., Hengel, A., Shen, C.: Is face recognition really a compressive sensing problem? Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 553–560, Colorado springs, CO (2011)

  22. Tikhonov. A., Arsenin, V.: Solutions of Ill-Posed Problems. V. H. Winston & Sons, Washington, DC (1977)

  23. Chen, C., Tramel, E., Fowler, J.: Compressed-sensing recovery of images and video using multi hypothesis predictions. Proceedings of Asilomar Conference on Signals, Systems, and Computer, pp. 1193–1198, Pacific Grove, CA (2011)

  24. Golub, G., Hansen, P.C., O’Leary, D.: Tikhonov regularization and total least squares. SIAM J Matrix Anal. Appl. 21(1), 185–194 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  25. Hansen, P., O’Leary, D.: The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J Sci. Comput. 14(6), 1487–1503 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  26. Mairal, J.: (SPArse Modeling Software), spams-devel.gforge.inria.fr

  27. Chang, C., Lin, C.: LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, vol. 2, no. 3, pp. 27:1–27:27 (2011). Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm/

  28. Li, W., Zhang, Z., Liu, Z.: Expandable data-driven graphical modeling of human actions based on salient postures. IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, no. 11, pp. 1499–1510 (2008)

  29. Fine, S., Singer, Y., Tishby, N.: The hierarchical hidden Markov model: analysis and applications. Mach. Learn. 32(1), 41–62 (1998)

    Article  MATH  Google Scholar 

  30. Liu, K., Ma, B., Du, Q., Chen, G.: Fast Motion Detection from Airborne Videos Using Graphics computing units. J.Appl. Remote Sens. vol. 6, no. 1 (2012)

  31. Tsang, I., Kwok, J., Cheung, P.-M.: Core vector machines: Fast SVM training on very large data sets. J. Mach. Learn. Res. vol. 6, no. 1, pp. 363–392 (2005)

Download references

Author information

Authors and Affiliations

  1. The University of Texas at Dallas, Richardson, TX, USA

    Chen Chen, Kui Liu & Nasser Kehtarnavaz

Authors
  1. Chen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kui Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nasser Kehtarnavaz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chen Chen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chen, C., Liu, K. & Kehtarnavaz, N. Real-time human action recognition based on depth motion maps. J Real-Time Image Proc 12, 155–163 (2016). https://doi.org/10.1007/s11554-013-0370-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-013-0370-1

Keywords

Search

Navigation