Skip to main content
SpringerLink
Log in

Real-time selection of video streams for live TV broadcasting based on Query-by-Example using a 3D model

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The emergence of low-cost cameras with nearly professional features in the consumer market represents a new important source of video information. For example, using an increasing number of these cameras in live TV broadcastings enables obtaining varied contents without affecting the production costs. However, searching for interesting shots (e.g., a certain view of a specific car in a race) among many video sources in real-time can be difficult for a Technical Director (TD). So, TDs require a mechanism to easily and precisely represent the kind of shot they want to obtain abstracting them from the need to be aware of all the views provided by the cameras. In this paper we present our proposal to help a TD to visually define, using an interface for the definition of 3D scenes, an interesting sample view of one or more objects in the scenario. We recreate the views of the cameras in a 3D engine and apply 3D geometric computations on their virtual view, instead of analyzing the real images they provide, to enable an efficient and precise real-time selection. Specifically, our system computes a similarity measure to rank the candidate cameras. Moreover, we present a prototype of the system and an experimental evaluation that shows the interest of our proposal.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. QBI is QBE applied to Content-Based Image Retrieval (CBIR), by using an image as a query.

  2. Broadcasters are now beginning to use 3D stereo cameras, which can be represented as two traditional cameras (one per lens) in our approach, as considering the special capabilities of 3D stereo images is out of the scope of this paper.

  3. Notice that the paddles are mainly shown in blue as they are considered as belonging to the rear view of the object.

  4. One rendering to obtain the percentage visible of the object and the percentage of the shot filled by it, and nine additional renderings to obtain the kind of view of the object (the system needs one rendering per view covering it completely—up to 6 in total—, and 3 renderings more to compute the percentage of each view covered).

  5. The prototype is available at http://sid.cps.unizar.es/MultiCAMBA/QBE.

  6. See http://jmonkeyengine.com.

  7. See http://www.sketchup.com.

References

  1. Ansary TF, Daoudi M, Vandeborre JP (2007) A bayesian 3-D search engine using adaptive views clustering. IEEE Trans Multimedia 9(1):78–88

    Article  Google Scholar 

  2. Assfalg J, Bimbo AD, Pala P (2002) Three-dimensional interfaces for querying by example in content-based image retrieval. IEEE Trans Vis Comput Graph 8(4):305–318

    Article  Google Scholar 

  3. Bimbo AD, Pala P Visual image retrieval by elastic matching of user sketches. IEEE Trans Pattern Anal Mach Intell 19(2):121–132 (1997)

    Article  Google Scholar 

  4. Cheng E, Jing F, Zhang L (2009) A unified relevance feedback framework for web image retrieval. IEEE Trans Image Process 18(6):1350–1357

    Article  MathSciNet  Google Scholar 

  5. Chmielewski J (2012) Finding interactive 3D objects by their interaction properties. Multimed Tools Appl. http://link.springer.com/article/10.1007%2Fs11042-012-1125-x. Accessed 19 Jun 2013

  6. Choi K, Lee SW, Seo Y (2009) Automatic broadcast video generation for ball sports from multiple views. In: International Workshop on Advanced Image Technology (IWAIT’09)

  7. Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: Ideas, influences, and trends of the new age. ACM Comput Surv 40(2):5:1–5:60

    Article  Google Scholar 

  8. Erozel G, Cicekli NK, Cicekli I (2008) Natural language querying for video databases. Inf Sci 178(12):2534–2552

    Article  Google Scholar 

  9. Fagin R, Kumar R, Sivakumar D (2003) Comparing top k lists. In: 14th annual ACM-SIAM Symposium on Discrete Algorithms (SODA’03). Society for Industrial and Applied Mathematics, pp 28–36

  10. Flickner M, Sawhney H, Niblack W, Ashley J, Huang Q, Dom B, Gorkani M, Hafner J, Lee D, Petkovic D, Steele D, Yanker P (1995) Query by image and video content: the QBIC system. Comput 28(9):23–32

    Article  Google Scholar 

  11. Gao Y, Tang J, Hong R, Yan S, Dai Q, Zhang N, Chua TS (2012) Camera constraint-free view-based 3-D object retrieval. IEEE Trans Image Process 21(4):2269–2281

    Article  MathSciNet  Google Scholar 

  12. Gao Y, Wang M, Zha ZJ, Tian Q, Dai Q, Zhang N (2011) Less is more: efficient 3-D object retrieval with query view selection. IEEE Trans Multimedia 13(5):1007–1018

    Article  Google Scholar 

  13. Gudivada VN, Raghavan VV (1995) Design and evaluation of algorithms for image retrieval by spatial similarity. ACM Trans Inf Syst 13(2):115–144

    Article  Google Scholar 

  14. Hearst MA, Dumais S, Osman E, Platt JC, Schölkopf B (1998) Support vector machines. IEEE Intell Syst Their Appl 13(4):18–28

    Article  Google Scholar 

  15. Ilarri S, Mena E, Illarramendi A (2010) Location-dependent query processing: Where we are and where we are heading. ACM Comput Surv 42(3):12:1–12:73

    Article  Google Scholar 

  16. Ilarri S, Mena E, Illarramendi A, Yus R, Laka M, Marcos G (2012) A friendly location-aware system to facilitate the work of technical directors when broadcasting sport events. Mobile Inf Syst 8(1):17–43

    Article  Google Scholar 

  17. Kendall MG (1938) A new measure of rank correlation. Biometrika 30(1/2):81–93

    Article  MATH  MathSciNet  Google Scholar 

  18. Li B, Johan H (2013) Sketch-based 3D model retrieval by incorporating 2D-3D alignment. Multimed Tools Appl 65(3):363–385

    Article  Google Scholar 

  19. Liu Y, Zhang D, Lu G, Ma WY (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recogn 40(1):262–282

    Article  MATH  Google Scholar 

  20. Lu Y, Zhang H, Wenyin L, Hu C (2003) Joint semantics and feature based image retrieval using relevance feedback. IEEE Trans Multimedia 5(3):339–347

    Article  Google Scholar 

  21. McCown F, Nelson ML (2007) Agreeing to disagree: search engines and their public interfaces. In: Seventh ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL’07). ACM, pp 309–318

  22. Miller GA (1956) The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev 63(2):81–97

    Article  Google Scholar 

  23. Morrison D, Tsikrika T, Hollink V, de Vries AP, Bruno É, Marchand-Maillet S (2012) Topic modelling of clickthrough data in image search. Multimed Tools Appl. http://link.springer.com/article/10.1007%2Fs11042-012-1038-8. Accessed 19 Jun 2013

  24. Pedronette DCG, da Silva Torres R, Calumby RT (2012) Using contextual spaces for image re-ranking and rank aggregation. Multimed Tools Appl. http://link.springer.com/article/10.1007%2Fs11042-012-1115-z. Accessed 19 Jun 2013

  25. Rui Y, Huang TS, Ortega M, Mehrotra S (1998) Relevance feedback: a power tool for interactive content-based image retrieval. IEEE Trans Circuits Syst Video Technol 8(5):644–655

    Article  Google Scholar 

  26. Scovanner P, Ali S, Shah M (2007) A 3-dimensional SIFT descriptor and its application to action recognition. In: 15th international conference on Multimedia (MULTIMEDIA’07). ACM, pp 357–360

  27. Shi Z, Liu X, Li Q, He Q, Shi Z (2012) Extracting discriminative features for CBIR. Multimed Tools Appl 61(2):263–279

    Article  Google Scholar 

  28. Shirahama K, Uehar K (2010) Video retrieval from few examples using ontology and rough set theory. In: 12th international workshop of the multimedia metadata community. Second workshop on Semantic Multimedia Database Technologies (SMDT 2010), vol 680. CEUR Workshop Proceedings (CEUR-WS.org), pp 5–16

  29. Shirahama K, Uehara K (2011) Query by virtual example: video retrieval using example shots created by virtual reality techniques. In: 2011 sixth International Conference on Image and Graphics (ICIG 2011). IEEE Computer Society, pp 829–834

  30. Shirahatti N, Barnard K (2005) Evaluating image retrieval. In: Conference on Computer Vision and Pattern Recognition (CVPR’05), vol 1. IEEE Computer Society, pp 955–961

  31. Thomee B (2010) A picture is worth a thousand words—content-based image retrieval techniques. Ph.D. thesis, Leiden University, Germany

  32. Thomee B, Lew MS (2012) Interactive search in image retrieval: a survey. Int J Multimed Info Retr 1(2):71–86

    Article  Google Scholar 

  33. Trillo R, Po L, Ilarri S, Bergamaschi S, Mena E (2011) Using semantic techniques to access web data. Inf Syst (Special Issue on Semantic Integration of Data, Multimedia, and Services) 36(2):117–133

    Google Scholar 

  34. Wang J, Xu C, Chng E, Lu H, Tian Q (2008) Automatic composition of broadcast sports video. Multimed Syst 14(4):179–193

    Article  Google Scholar 

  35. Wang XY, Zhang BB, Yang HY (2012) Content-based image retrieval by integrating color and texture features. Multimed Tools Appl. http://link.springer.com/article/10.1007%2Fs11042-012-1055-7. Accessed 19 Jun 2013

  36. Yus R, Mena E, Bernad J, Ilarri S, Illarramendi A (2011) Location-aware system based on a dynamic 3D model to help in live broadcasting of sport events. In: 19th ACM international conference on Multimedia (MM 2011). ACM, pp 1005–1008

  37. Zloof MM (1975) Query by example. In: AFIPS national computer conference. AFIPS Press, pp 431–438

Download references

Acknowledgements

This research work has been supported by the CICYT project TIN2010-21387-C02-02 and DGA-FSE. We would also like to thank the anonymous reviewers for their useful comments.

Author information

Authors and Affiliations

  1. IIS Department, University of Zaragoza, Maria de Luna 1, 50018, Zaragoza, Spain

    Roberto Yus, Sergio Ilarri & Eduardo Mena

Authors
  1. Roberto Yus
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sergio Ilarri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eduardo Mena
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roberto Yus.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yus, R., Ilarri, S. & Mena, E. Real-time selection of video streams for live TV broadcasting based on Query-by-Example using a 3D model. Multimed Tools Appl 74, 2659–2685 (2015). https://doi.org/10.1007/s11042-013-1550-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-013-1550-5

Keywords

Search

Navigation