Skip to main content
SpringerLink
Log in

FPGA-Based Vision Processing System for Automatic Online Player Tracking in Indoor Sports

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

In recent years, there has been an increasing growth of using vision-based systems for tracking the players in team sports to evaluate and enhance their performance. Player tracking using vision systems is a very challenging task due to the nature of sports games, which includes severe and frequent interactions (e.g. occlusions) between the players. Additionally, these vision systems have high computational demands since they require processing of a huge amount of video data based on the utilization of multiple cameras with high resolution and high frame rate. As a result, most of the existing systems based on general-purpose computers are not able to perform online real-time player tracking, but track the players offline using pre-recorded video files, limiting, e.g., direct feedback on the player performance during the game. In this paper, we present a reconfigurable system to track the players in indoor sports automatically and without user interaction. The proposed system performs real-time processing of the incoming video streams from the cameras, achieving online player tracking. The teams are identified, and the players’ positions are detected based on the colors of their jerseys. FPGA technology is used to handle the compute-intensive vision processing tasks by implementing the video acquisition, video preprocessing, player segmentation, and team identification & player detection modules in hardware, realizing an online real-time system. While the pixel processing is performed in the FPGA, the less compute-intensive player tracking is performed on a general purpose computer. The maximum achieved frame rate for the FPGA implementation is 96.7 fps using a mature Xilinx Virtex-4 FPGA, and can be increased to 136.4 fps using a Xilinx Virtex-7 device. The Player tracking requires an average time of 2.5 ms per frame in the host-PC. As a result, the proposed reconfigurable system supports a maximum frame rate of 78.9 fps using two cameras with a resolution of 1392 × 1040 pixels each. Our results show that the achieved average precision and recall for player detection are up to 84.02% and 96.6%, respectively. Including player tracking, the achieved average precision and recall are up to 94.85% and 94.72%, respectively. Using the proposed FPGA implementation, a speedup by a factor of 15.2 is achieved compared to an OpenCV-based software implementation on a PC equipped with a 2.93 GHz Intel i7-870 CPU.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15
Figure 16
Figure 17
Figure 18
Figure 19
Figure 20
Figure 21
Figure 22
Figure 23
Figure 24

Similar content being viewed by others

References

  1. APIDIS Basketball dataset. http://sites.uclouvain.be/ispgroup/index.php/Softwares/APIDIS.

  2. Acuna, D. (2017). “Towards Real-Time Detection and Tracking of Basketball Players using Deep Neural Networks”. In: 31st Conference on Neural Information Processing Systems (NIPS 2017).

  3. Automated Imaging Association (AIA). (2016). GigE Vision - True Plug and Play Connectivity. http://www.visiononline.org.

  4. Alahi, A., Boursier, Y., Jacques, L., Vandergheynst, P. (2009). Sport players detection and tracking with a mixed network of planar and omnidirectional cameras. In 3rd ACM/IEEE international conference on distributed smart cameras, ICDSC 2009 (pp. 1–8), https://doi.org/10.1109/ICDSC.2009.52893406.

  5. Albuquerque, E.S., Ferreira, A.P.A., Silva, G.M., Carlos, R.L.M., Albuquerque, D.S., Barros, E.N.S. (2016). An FPGA-based accelerator for multiple real-time template matching. In 29th symposium on integrated circuits and systems design (SBCCI) (pp. 1–4).

  6. Bailey, D.G. (2011). Design for embedded image processing on FPGAs. New York: Wiley.

    Book  Google Scholar 

  7. Benton, S. (2008). Background subtraction, MATLAB models. EETimes.

  8. Bernardin, K., & Stiefelhagen, R. (2008). Evaluating multiple object tracking performance: the CLEAR MOT metrics. Eurasip Journal on Image and Video Processing. https://doi.org/10.1155/2008/246309.

  9. Biresaw, T.A., Nawaz, T., Ferryman, J., Dell, A.I. (2016). ViTBAT: Video tracking and behavior annotation tool. In 2016 13th IEEE international conference on advanced video and signal based surveillance, AVSS 2016 (Vol. 1, pp. 295–301). https://doi.org/10.1109/AVSS.2016.7738055.

  10. Butt, A.A., & Collins, R.T. (2013). Multi-target tracking by lagrangian relaxation to min-cost network flow. In Proceedings of the IEEE Computer Society conference on computer vision and pattern recognition (pp. 1846–1853), https://doi.org/10.1109/CVPR.2013.241.

  11. Cheung, S.C., & Kamath, C. (2004). Robust techniques for background subtraction in urban traffic video. Proceedings of Video Communications and Image Processing, SPIE Electronic Imaging, 881–892. https://doi.org/10.1117/12.526886.

  12. de Pádua, P.H., Pádua, F.L, de A. Pereira, M., Sousa, M.T., de Oliveira, M.B., Wanner, E.F. (2017). A vision-based system to support tactical and physical analyses in futsal. Machine Vision and Applications, 28(5–6), 475–496. https://doi.org/10.1007/s00138-017-0849-z.

    Article  Google Scholar 

  13. Foley, J.D., van Dam, A., Feiner, S. K., Hughes, J.F. (1996). Computer graphics: principles and practice. Boston: Addison-Wesley.

    MATH  Google Scholar 

  14. Godil, A., Bostelman, R., Shneier, M., Shackleford, W. (2014). Performance metrics for evaluating object and human detection and tracking systems (pp. 1–13) https://doi.org/10.6028/NIST.IR.7972.

  15. Hu, M.C., Chang, M.H., Wu, J.L., Chi, L. (2011). Robust camera calibration and player tracking in broadcast basketball video. IEEE Transactions on Multimedia, 13(2), 266–279. https://doi.org/10.1109/TMM.2010.2100373.

    Article  Google Scholar 

  16. Ibraheem, O.W., Irwansyah, A., Hagemeyer, J., Porrmann, M., Rueckert, U. (2015). A resource-efficient multi-camera GigE vision IP Core for embedded vision processing platforms. In 2015 international conference on ReConFigurable computing and FPGAs (ReConFig) (pp. 1–6), https://doi.org/10.1109/ReConFig.2015.7393282.

  17. Ibraheem, O.W., Irwansyah, A., Hagemeyer, J., Porrmann, M., Rueckert, U. (2017). Reconfigurable vision processing system for player tracking in indoor sports. In 2017 Conference on design and architectures for signal and image processing (DASIP) (pp. 1–6), https://doi.org/10.1109/DASIP.2017.8122114.

  18. Irwansyah, A., Ibraheem, O.W., Hagemeyer, J., Porrmann, M., Rueckert, U. (2015). FGPA-based Circular Hough Transform with Grpah Clustering. In 2015 International Conference on ReConFigurable Computing and FPGAs (ReConFig) (pp. 1–8), https://doi.org/10.1109/ReConFig.2015.7393313.

  19. Irwansyah, A., Ibraheem, O.W., Hagemeyer, J., Porrmann, M., Rueckert, U. (2017). FPGA-based multi-robot tracking. Journal of Parallel and Distributed Computing, 107, 146–161. https://doi.org/10.1016/j.jpdc.2017.03.008.

    Article  Google Scholar 

  20. Jacobsen, M., Sampangi, S., Freund, Y., Kastner, R. (2014). Improving FPGA accelerated tracking with multiple online trained classifiers. In 24th International Conference on Field Programmable Logic and Applications, FPL 2014, https://doi.org/10.1109/FPL.2014.6927505.

  21. Land, E.H., & McCann, J.J. (1971). Lightness and Retinex theory. Journal of the Optical Society of America, 61(1), 1–11.

    Article  Google Scholar 

  22. Lam, E. (2005). Combining gray world and retinex theory for automatic white balance in digital photography. In Proceedings of the ninth international symposium on consumer electronics, (ISCE 2005) (pp. 1–6).

  23. Li, Y., Huang, C., Nevatia, R. (2009). Learning to associate: hybridboosted multi-target tracker for crowded scene. In 2009 IEEE Computer Society conference on computer vision and pattern recognition workshops, CVPR workshops 2009 (pp. 2953–2960), https://doi.org/10.1109/CVPRW.2009.5206735.

  24. Li, C., Yee, L.Y., Maruyama, H., Yamaguchi, Y. (2017). FPGA-based volleyball player tracker. In ACM SIGARCH Computer Architecture News (Vol. 44, pp. 80–86), https://doi.org/10.1145/3039902.3039917.

  25. Liu, J., Cao, X., Li, Y., Zhang, B. (2018). Online multi-object tracking using hierarchical constraints for complex scenarios. IEEE Transactions on Intelligent Transportation Systems, 19(1), 151–161. https://doi.org/10.1109/TITS.2017.2750058.

    Article  Google Scholar 

  26. Lu, W.l. (2011). Learning to track and identify players from broadcast sports videos. Ph.D. thesis, The University Of British Columbia.

  27. Lu, W.L., Ting, J.A., Little, J.J., Murphy, K.P. (2013). Learning to track and identify players from broadcast sports videos. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(7), 1704–1716. https://doi.org/10.1109/TPAMI.2012.242.

    Article  Google Scholar 

  28. Mazzeo, P.L., Giove, L., Moramarco, G.M., Spagnolo, P., Leo, M. (2011). HSV and RGB color histograms comparing for objects tracking among non overlapping FOVs, using CBTF. In 8th IEEE international conference on advanced video and signal based surveillance, AVSS (pp. 498–503), https://doi.org/10.1109/AVSS.2011.6027383.

  29. McFarlane, N.J.B., & Schofield, C.P. (1995). Segmentation and tracking of piglets in images. Machine Vision and Applications, 8(3), 187–193. https://doi.org/10.1007/BF01215814.

    Article  Google Scholar 

  30. Monier, E. (2011). Vision based tracking in team sports. Ph.D. thesis, Paderborn University, Germany.

  31. Monier, E., Wilhelm, P., Rückert, U. (2009). A computer vision based tracking system for indoor team sports. In The fourth international conference on intelligent computing and information systems (pp. 1–5).

  32. Munkres, J. (1957). Algorithms for the Assignment and Transportation Problems. Journal of the Society for Industrial and Applied Mathematics, 5(1), 32–38. https://doi.org/10.1137/0105003. http://epubs.siam.org/doi/10.1137/0105003.

    Article  MathSciNet  MATH  Google Scholar 

  33. Okuma, K., Taleghani, A., Freitas, N., Little, J.J., Lowe, D.G. (2004). A boosted particle filter: multitarget detection and tracking. In European conference on computer vision (pp. 28–39).

  34. Porrmann, M., Hagemeyer, J., Romoth, J., Strugholtz, M., Pohl, C. (2010). RAPTOR-A scalable platform for rapid prototyping and FPGA-based cluster computing. Advances in Parallel Computing, 19, 592–599. https://doi.org/10.3233/978-1-60750-530-3-592.

    Google Scholar 

  35. Santiago, C., Gomes, L., Sousa, A., Reis, L., Estriga, M. (2012). Tracking players in indoor sports using a vision system inspired in fuzzy and parallel processing. Cutting Edge Research in New Technologies. https://doi.org/10.5772/2431.

  36. Santiago, C.B., Sousa, A., Reis, L.P. (2013). Vision system for tracking handball players using fuzzy color processing. Machine Vision and Applications, 24(5), 1055–1074. https://doi.org/10.1007/s00138-012-0471-z.

    Article  Google Scholar 

  37. Schaeffer, S.E. (2007). Graph clustering. Computer Science Review, 1, 27–64. https://doi.org/10.1016/j.cosrev.2007.05.001.

    Article  MATH  Google Scholar 

  38. STATS Website. www.stats.com.

Download references

Acknowledgements

This work was funded as part of the Cluster of Excellence Cognitive Interaction Technology ‘CITEC’ (EXC 277), Bielefeld University, and the European Fond for Regional Development (Europaeischer Fond fuer regionale Entwicklung (EFRE no. 0400079)), and the German Academic Exchange Service (DAAD). The authors are responsible for the contents of this publication.

Author information

Author notes

  1. Arif Irwansyah

    Present address: Electronic Engineering Polytechnic Institute of Surabaya, Surabaya, Indonesia

Authors and Affiliations

  1. Cognitronics and Sensor Systems Group, CITEC, Bielefeld University, Bielefeld, Germany

    Omar W. Ibraheem, Arif Irwansyah, Jens Hagemeyer, Mario Porrmann & Ulrich Rueckert

Authors
  1. Omar W. Ibraheem
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arif Irwansyah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jens Hagemeyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mario Porrmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ulrich Rueckert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Omar W. Ibraheem.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ibraheem, O.W., Irwansyah, A., Hagemeyer, J. et al. FPGA-Based Vision Processing System for Automatic Online Player Tracking in Indoor Sports. J Sign Process Syst 91, 703–729 (2019). https://doi.org/10.1007/s11265-018-1381-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-018-1381-8

Keywords

Search

Navigation