Abstract
Automatic generation of metadata, facilitating the retrieval of multimedia items, potentially saves large amounts of manual work. However, the high specialization degree of feature extraction algorithms makes them unaware of the context they operate in, which contains valuable and often necessary information. In this paper, we show how Semantic Web technologies can provide a context that algorithms can interact with. We propose a generic problem-solving platform that uses Web services and various knowledge sources to find solutions to complex requests. The platform employs a reasoner-based composition algorithm, generating an execution plan that combines several algorithms as services. It then supervises the execution of this plan, intervening in case of errors or unexpected behavior. We illustrate our approach by a use case in which we annotate the names of people depicted in a photograph.
Similar content being viewed by others
References
Atrey PK, Hossain MA, El Saddik A, Kankanhalli MS (2010) Multimodal fusion for multimedia analysis: a survey. Multimedia Syst 16(6):345–379. doi:10.1007/s00530-010-0182-0
Berners-Lee T, Connolly D (2009) Notation3 (N3): a readable RDF syntax. W3C recommendation. http://www.w3.org/TeamSubmission/n3/. Accessed 12 December 2010
Berners-Lee T, Connolly D, Kagal L, Scharf Y, Hendler J (2008) N3Logic: a logical framework for the World Wide Web. Theory and Practice of Logic Programming 8(3):249–269. doi:10.1017/S1471068407003213
Berners-Lee T, Hendler J, Lassila O (2001) The Semantic Web. Sci Am 284(5):34
Bizer C, Heath T, Berners-Lee T (2009) Linked Data - the story so far. Int J Semant Web Inf 5(3):1–22. doi10.4018/jswis.2009070101
Bloehdorn S, Petridis K, Saathoff C, Simou N, Tzouvaras V, Avrithis Y, Handschuh S, Kompatsiaris Y, Staab S, Strintzis M (2005) Semantic annotation of images and videos for multimedia analysis. In: The Semantic Web: research and applications, pp 592–607
Corkill DD (1991) Blackboard systems. AI Expert 6(9):40–47
DBpedia (2010) http://dbpedia.org/. Accessed 12 December 2010
De Roo J (2010) Euler proof mechanism. http://eulersharp.sourceforge.net/. Accessed 12 December 2010
Fensel D, Bussler C (2002) Semantic Web enabled Web services. In: 2nd annual diffuse conference
Finzi A, Pirri F, Reiter R (2000) Open world planning in the situation calculus. In: Proceedings of the national conference on artificial intelligence, pp 754–760
Freebase (2010) http://www.freebase.com/. Accessed 12 December 2010
Hanjalic A, Lienhart R, Ma WY, Smith RJ (2008) The holy grail of multimedia information retrieval: so close or yet so far away? In: Proceedings of the IEEE Multimedia, vol 96(4), pp 541–547. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4457924
Horrocks I, Patel-Schneider PF, Boley H, Tabet S (2004) SWRL: a Semantic Web rule language combining OWL and RuleML. W3C member submission. http://www.w3.org/Submission/SWRL/. Accessed 12 December 2010
Hristoskova A, Volckaert B, Turck FD (2009) Dynamic composition of semantically annotated web services through QoS-aware HTN planning algorithms. In: ICIW ’09: Proceedings of the 2009 fourth international conference on Internet and Web applications and services. IEEE Computer Society, Washington, pp 377–382
Junghans M, Agarwal S, Studer R (2010) Towards practical Semantic Web service discovery. In: The Semantic Web: research and applications, 7th extended Semantic Web conference, ESWC 2010, Heraklion, Crete, Greece, May 30–June 3, 2010, Proceedings. Springer, New York
Klusch M, Fries B, Khalid M (2005) OWLS-MX: Hybrid OWL-S service matchmaking. In: Proceedings of 1st intl. AAAI fall symposium on agents and the Semantic Web
Klyne G, Carrol JJ (2004) Resource description framework (RDF): concepts and abstract syntax. W3C recommendation. http://www.w3.org/TR/2004/REC-rdf-concepts-20040210/. Accessed 12 December 2010
Li L, Horrocks I (2003) A software framework for matchmaking based on Semantic Web technology. In: WWW ’03: Proceedings of the 12th international conference on World Wide Web. ACM Press, New York, pp 331–339. doi:10.1145/775152.775199
Martin D, Burstein M, Hobbs J, Lassila O (2004) OWL-S: Semantic markup for web services. W3C member submission. http://www.w3.org/Submission/OWL-S/. Accessed 12 December 2010
Medjahed B, Bouguettaya A, Elmagarmid AK (2003) Composing web services on the Semantic Web. VLDB J 12(4):333–351. doi:10.1007/s00778-003-0101-5
Osmun T (2010) Euler Eye installation, demo, and deep taxonomy benchmark. http://ruleml.org/WellnessRules/files/WellnessRulesN3-2009-11-10.pdf. Accessed 12 December 2010
Osmun T (2010) Eye deep taxonomy benchmark results. http://eulersharp.sourceforge.net/2003/03swap/dtb-2010.txt. Accessed 12 December 2010
Prud’hommeaux E, Seaborne A (2008) SPARQL query language for RDF. W3C recommendation. http://www.w3.org/TR/rdf-sparql-query/. Accessed 12 December 2010
Rahurkar M, Tsai S, Dagli C, Huang T (2010) Image interpretation using large corpus: wikipedia. Proc IEEE 98(8):1509–1525. doi:10.1109/JPROC.2010.2050410. http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=5484723
Redavid D, Iannone L, Payne T (2007) OWL-S atomic services composition with SWRL rules. In: Proceedings of the 4th Italian Semantic Web workshop. http://eprints.ecs.soton.ac.uk/15658/
Roy PV, Haridi S (2004) Concepts, techniques, and models of computer programming. MIT Press, Cambridge
Shin D, Lee K, Suda T (2009) Automated generation of composite Web services based on functional semantics. Web Semantics: Science
Smith JR, Schirling P (2006) Metadata standards roundup. IEEE MultiMedia 13:84–88. doi:10.1109/MMUL.2006.34
Sycara K, Paolucci M, Ankolekar A, Srinivasan N (2003) Automated discovery, interaction and composition of Semantic Web services. Journal of Web Semantics 1(1):27–46
Troncy R, Mannens E, Pfeiffer S, Van Deursen D (2010) Media fragments URI 1.0. W3C working draft. http://www.w3.org/TR/media-frags/. Accessed 12 December 2010
Verborgh R (2010) Generic semantic problem-solving platform demonstration: road trip. http://multimedialab.elis.ugent.be/organon/arseco/demonstrations. Accessed 12 December 2010
Verborgh R, Van Deursen D, De Roo J, Mannens E, Van de Walle R (2010) SPARQL endpoints as front-end for multimedia processing algorithms. In: Proceedings of the fourth international workshop on service matchmaking and resource retrieval in the Semantic Web at the 9th international Semantic Web conference (ISWC 2010)
Verstockt S, Van Leuven S, Van de Walle R, Dermaut E, Torelle S, Gevaert W (2009) Actor recognition for interactive querying and automatic annotation in digital video. In: 13th, Proceedings of the IASTED international conference on Internet and multimedia systems and applications. ACTA Press, Honolulu, pp 149–155
Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154. doi:10.1023/B:VISI.0000013087.49260.fb
Acknowledgements
The research activities as described in this paper were funded by Ghent University, the Interdisciplinary Institute for Broadband Technology (IBBT), the Institute for the Promotion of Innovation by Science and Technology in Flanders (IWT), the Fund for Scientific Research Flanders (FWO Flanders), and the European Union.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Verborgh, R., Van Deursen, D., Mannens, E. et al. Enabling context-aware multimedia annotation by a novel generic semantic problem-solving platform. Multimed Tools Appl 61, 105–129 (2012). https://doi.org/10.1007/s11042-010-0709-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-010-0709-6