Skip to main content
SpringerLink
Log in

AI-Based Cropping of Soccer Videos for Different Social Media Representations

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14557))

Included in the following conference series:

  • 400 Accesses

Abstract

The process of re-publishing soccer videos on social media often involves labor-intensive and tedious manual adjustments, particularly when altering aspect ratios while trying to maintain key visual elements. To address this issue, we have developed an AI-based automated cropping tool called SmartCrop which uses object detection, scene detection, outlier detection, and interpolation. This innovative tool is designed to identify and track important objects within the video, such as the soccer ball, and adjusts for any tracking loss. It dynamically calculates the cropping center, ensuring the most relevant parts of the video remain in the frame. Our initial assessments have shown that the tool is not only practical and efficient but also enhances accuracy in maintaining the essence of the original content. A user study confirms that our automated cropping approach significantly improves user experience compared to static methods. We aim to demonstrate the full functionality of SmartCrop, including visual output and processing times, highlighting its efficiency, support of various configurations, and effectiveness in preserving the integrity of soccer content during aspect ratio adjustments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Apostolidis, K., Mezaris, V.: A fast smart-cropping method and dataset for video retargeting. In: Proceedings of IEEE ICIP, pp. 2618–2622 (2021)

    Google Scholar 

  2. Bourke, P.: Interpolation methods. Miscellaneous Projection Model. Rendering 1(10), 1–9 (1999)

    Google Scholar 

  3. Castellano, B.: SceneDetect. https://github.com/Breakthrough/PySceneDetect/tree/main (2023)

  4. Chai, T., Draxler, R.R.: Root mean square error (RMSE) or mean absolute error (MAE)? Arguments against avoiding RMSE in the literature. Geosci. Model Dev. 7(3), 1247–1250 (2014)

    Article  Google Scholar 

  5. Deselaers, T., Dreuw, P., Ney, H.: Pan zoom scan - time-coherent trained automatic video cropping. In: Proceedings of IEEE CVPR, pp. 1–8 (2008)

    Google Scholar 

  6. Dorcheh, S.M.M., et al.: SmartCrop: AI-based cropping of soccer videos. In: Proceedings of IEEE ISM (2023)

    Google Scholar 

  7. Gautam, S.: Bridging multimedia modalities: enhanced multimodal AI understanding and intelligent agents. In: Proceedings of ACM ICMI (2023)

    Google Scholar 

  8. Gautam, S., Midoglu, C., Shafiee Sabet, S., Kshatri, D.B., Halvorsen, P.: Assisting soccer game summarization via audio intensity analysis of game highlights. In: Proceedings of 12th IOE Graduate Conference, vol. 12, pp. 25–32. Institute of Engineering, Tribhuvan University, Nepal (2022)

    Google Scholar 

  9. Gautam, S., Midoglu, C., Shafiee Sabet, S., Kshatri, D.B., Halvorsen, P.: Soccer game summarization using audio commentary, metadata, and captions. In: Proceedings of of ACM MM NarSUM. pp. 13–22 (2022)

    Google Scholar 

  10. Husa, A., Midoglu, C., Hammou, M., Halvorsen, P., Riegler, M.A.: HOST-ATS: automatic thumbnail selection with dashboard-controlled ML pipeline and dynamic user survey. In: Proceedings of ACM MMSys, pp. 334–340 (2022)

    Google Scholar 

  11. Husa, A., et al.: Automatic thumbnail selection for soccer videos using machine learning. In: Proceedings of ACM MMSys, pp. 73–85 (2022)

    Google Scholar 

  12. Jain, E., Sheikh, Y., Shamir, A., Hodgins, J.: Gaze-driven video re-editing. ACM TOG 34(2), 1–12 (2015)

    Article  Google Scholar 

  13. Jocher, G., Chaurasia, A., Qiu, J.: Ultralytics YOLOv8. https://github.com/ultralytics/ultralytics (2023)

  14. Kaur, H., Kour, S., Sen, D.: Video retargeting through spatio-temporal seam carving using kalman filter. IET Image Proc. 13(11), 1862–1871 (2019)

    Article  Google Scholar 

  15. Kemper, M., Rosso, G., Monnone, B., Kemper, M., Rosso, G.: Creating animated effects. In: Advanced Flash Interface Design, pp. 255–288 (2006)

    Google Scholar 

  16. Kopf, S., Haenselmann, T., Kiess, J., Guthier, B., Effelsberg, W.: Algorithms for video retargeting. Multimedia Tools Appl 51(2), 819–861 (2011)

    Article  Google Scholar 

  17. Lee, H.S., Bae, G., Cho, S.I., Kim, Y.H., Kang, S.: Smartgrid: video retargeting with spatiotemporal grid optimization. IEEE Access 7, 127564–127579 (2019)

    Article  Google Scholar 

  18. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of IEEE ICCV, pp. 2980–2988 (2017)

    Google Scholar 

  19. Liu, F., Gleicher, M.: Video retargeting: automating pan and scan. In: Proceedings of ACM MM, pp. 241–250 (2006)

    Google Scholar 

  20. Liu, W., et al.: SSD: Single shot multibox detector. In: Proceedings of ECCV, pp. 21–37 (2016)

    Google Scholar 

  21. Midoglu, C., et al.: AI-based sports highlight generation for social media. In: Proceedings of ACM MHV (2024)

    Google Scholar 

  22. Nam, H., Park, D., Jeon, K.: Jitter-robust video re-targeting with kalman filter and attention saliency fusion network. In: Proceedings of IEEE ICIP, pp. 858–862 (2020)

    Google Scholar 

  23. Nergård Rongved, O.A., et al.: Using 3D convolutional neural networks (CNN) for real-time detection of soccer events. Int. J. Seman. Comput. 15(2), 161–187 (2021)

    Article  Google Scholar 

  24. Nergård Rongved, O.A., et al.: Real-time detection of events in soccer videos using 3D convolutional neural networks. In: Proceedings of IEEE ISM, pp. 135–144 (2020)

    Google Scholar 

  25. Nergård Rongved, O.A., et al.: Automated event detection and classification in soccer: the potential of using multiple modalities. Mach. Learn. Knowl. Extr. 3(4), 1030–1054 (2021)

    Article  Google Scholar 

  26. Noorkhokhar: YOLOv8-football: how to detect football players and ball in real-time using YOLOv8: a computer tutorial. https://github.com/noorkhokhar99/YOLOv8-football

  27. Rachavarapu, K.K., Kumar, M., Gandhi, V., Subramanian, R.: Watch to edit: video retargeting using gaze. Comput. Graph. Forum 37, 205–215 (2018)

    Article  Google Scholar 

  28. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once (YOLO): unified, real-time object detection. In: Proceedings of IEEE CVPR, pp. 779–788 (2016)

    Google Scholar 

  29. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, vol. 28 (2015)

    Google Scholar 

  30. Roboflow: football players detection dataset. https://universe.roboflow.com/roboflow-jvuqo/football-players-detection-3zvbc (2023)

  31. Saleem, S., Aslam, M., Shaukat, M.R.: A review and empirical comparison of universe outlier detection methods. Pakistan J. Stat. 37(4), 447–462 (2021)

    Google Scholar 

  32. Sarkhoosh, M.H., Dorcheh, S.M.M., Gautam, S., Midoglu, C., Sabet, S.S., Halvorsen, P.: Soccer on social media. arXiv preprint arXiv:2310.12328 (2023)

  33. Soucek, T., Lokoc, J.: TransNet V2: an effective deep network architecture for fast shot transition detection. CoRR (2020)

    Google Scholar 

  34. Valand, J.O., et al.: Automated clipping of soccer events using machine learning. In: Proceedings of IEEE ISM, pp. 210–214 (2021)

    Google Scholar 

  35. Valand, J.O., Kadragic, H., Hicks, S.A., Thambawita, V.L., Midoglu, C., Kupka, T., Johansen, D., Riegler, M.A., Halvorsen, P.: AI-based video clipping of soccer events. Mach. Learn. Knowl. Extr. 3(4), 990–1008 (2021)

    Article  Google Scholar 

  36. Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M.: YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv preprint arXiv:2207.02696 (2022)

  37. Wang, S., Tang, Z., Dong, W., Yao, J.: Multi-operator video retargeting method based on improved seam carving. In: Proceedings of IEEE ITOEC, pp. 1609–1614 (2020)

    Google Scholar 

  38. Wang, Y.S., Lin, H.C., Sorkine, O., Lee, T.Y.: Motion-based video retargeting with optimized crop and warp. In: Proceedings of ACM SIGGRAPH, pp. 1–9 (2010)

    Google Scholar 

Download references

Acknowledgment

This research was funded by the Research Council of Norway, project number 346671 (AI-storyteller).The authors would like to thank the Norwegian Professional Football League (“Norsk Toppfotball”) for making videos available for the research.

Author information

Authors and Affiliations

  1. SimulaMet, Oslo, Norway

    Cise Midoglu, Saeed Shafiee Sabet, Michael A. Riegler & Pål Halvorsen

  2. Oslo Metropolitan University, Oslo, Norway

    Mehdi Houshmand Sarkhoosh, Sayed Mohammad Majidi Dorcheh, Michael A. Riegler & Pål Halvorsen

  3. Forzasys, Oslo, Norway

    Mehdi Houshmand Sarkhoosh, Sayed Mohammad Majidi Dorcheh, Cise Midoglu, Saeed Shafiee Sabet, Tomas Kupka & Pål Halvorsen

  4. UIT The Arctic University of Norway, Tromsø, Norway

    Dag Johansen

Authors
  1. Mehdi Houshmand Sarkhoosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sayed Mohammad Majidi Dorcheh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cise Midoglu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Saeed Shafiee Sabet
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tomas Kupka
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dag Johansen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Michael A. Riegler
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Pål Halvorsen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mehdi Houshmand Sarkhoosh .

Editor information

Editors and Affiliations

  1. University of Amsterdam, Amsterdam, The Netherlands

    Stevan Rudinac

  2. Delft University of Technology, Delft, The Netherlands

    Alan Hanjalic

  3. Delft University of Technology, Delft, The Netherlands

    Cynthia Liem

  4. University of Amsterdam, Amsterdam, The Netherlands

    Marcel Worring

  5. Reykjavik University, Reykjavik, Iceland

    Björn Þór Jónsson

  6. Microsoft Research Lab – Asia, Beijing, China

    Bei Liu

  7. The University of Tokyo, Tokyo, Japan

    Yoko Yamakata

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sarkhoosh, M.H. et al. (2024). AI-Based Cropping of Soccer Videos for Different Social Media Representations. In: Rudinac, S., et al. MultiMedia Modeling. MMM 2024. Lecture Notes in Computer Science, vol 14557. Springer, Cham. https://doi.org/10.1007/978-3-031-53302-0_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-53302-0_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-53301-3

  • Online ISBN: 978-3-031-53302-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation