Abstract
A structured vocabulary of terms, such as a textual thesaurus, provides a way to conceptually describe visual information. The retrieval model described in this paper combines a conceptual and a visual layer as a first step towards the integration of ontologies and content-based image retrieval. Terms are related with image regions through a weighted association. This model allows the execution of concept-level queries, fulfilling user expectations and reducing the so-called semantic gap. Region-based relevance feedback is used to improve the quality of results in each query session and to help in the discovery of associations between text and image. The learning mechanism, whose function is to discover existing term-region associations, is based on a clustering algorithm applied over the features space and on propagation functions, which acts in each cluster where new information is available from user interaction. This approach is validated with the presentation of promising results obtained using the VOIR - Visual Object Information Retrieval system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barthes, R.: Rhetoric of the Image (trans. by Stephen Heath). In: Barthes, R. (ed.) Image, music, text, pp. 32–51. Fontana, London (1977)
Carson, C., et al.: Blobworld: Image segmentation using Expectation-Maximization and its application to image querying. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(8), 1026–1038 (2002)
Duygulu, P.: Translating images to words: A novel approach for object recognition. PhD Middle East Technical University, Dept. of Computer Engineering (2003)
Dwork, C., et al.: Rank aggregation methods for the Web. In: Proc. of tenth international conference on World Wide Web, Hong Kong, pp. 613–622 (2001)
Eakins, J.P.: Towards intelligent image retrieval. Pattern Recognition 35, 3–14 (2002)
Jain, A.K., Murty, M.N., Flynn, P.J.: Data Clustering: A Review. ACM Computing Surveys 31(3), 264–323 (1999)
Jing, F., et al.: Relevance Feedback in Region-Based Image Retrieval. IEEE Transactions on Circuits and Systems for Video Technology 14(5), 672–681 (2004)
Kingscote, A.: The Australian Pictorial Thesaurus 2 years on. In: DC-ANZ Metadata Conference, Australian National University, Canberra (2003)
Lee, C.S., Ma, W.Y., Zhang, H.J.: Information Embedding Based on User’s Relevance Feedback for Image Retrieval. SPIE Photonic East, Boston, USA (1999)
Li, J., Wang, J., Wiederhold, G.: IRM: Integrated Region Matching for Image Retrieval. In: Proc. ACM Multimedia 2000, Los Angeles, CA, USA, pp. 147–156. ACM, New York (2000)
Lu, Y., et al.: A unified framework for semantics and feature based relevance feedback in image retrieval systems. In: Proc. of ACM Multimedia 2000, Los Angeles, USA, pp. 31–38. ACM Press, New York (2000)
Ma, W.Y., Manjunath, B.S.: NeTra: A toolbox for navigating large image databases. Multimedia Systems 7(3), 184–198 (1999)
Manjunath, B.S., Salembier, P., Sikora, T.: Introduction to MPEG-7, Multimedia Content Description Interface. John Wiley & Sons, Chichester (2002)
Martin, D., et al.: A database of human segmented natural images and its applications to evaluating segmentation algorithms and measuring ecological statistics. In: Proc. IEEE 8th Int. Conf. Computer Vision, Vancouver, Canada, pp. 416–423. IEEE, Los Alamitos (2001)
Mezaris, V., Kompatsiaris, I., Strintzis, M.G.: Region-based Image Retrieval using an Object Ontology and Relevance Feedback. EURASIP Journal on Applied Signal Processing 2004(6), 886–901 (2004)
Panofsky, E.: Iconography and Iconology: An Introduction to the Study of Renaissance Art. In: Meaning in the visual arts, pp. 26–54. Penguin Books (1970)
Rui, Y., Huang, T.S.: A Novel Relevance Feedback Technique in Image Retrieval. In: Proc.of ACM Multimedia, pp. 67–70. ACM Press, New York (1999)
Rui, Y., et al.: Relevance Feedback: A Power Tool for Interactive Content-Based Image Retrieval. IEEE Transactions on Circuits and Systems for Video Technology 8(5), 644–655 (1998)
Torres, J., Parkes, A., Corte-Real, L.: Region-Based Relevance Feedback in Concept-Based Image Retrieval. In: Proc. of the 5th International Workshop on Image Analysis for Multimedia Interactive Services, Lisboa, Portugal (2004)
Wang, T., Rui, Y., Sun, J.-G.: Constraint Based Region Matching for Image Retrieval. International Journal of Computer Vision 56(1/2), 37–45 (2004)
Wu, L., et al.: FALCON Feedback Adaptive Loop for Content-Based Retrieval. In: VLDB 2000 (2000)
Zhang, R., Zhang, Z.: Hidden Semantic Concept Discovery in Region Based Image Retrieval. In: Proc. of the 2004 IEEE Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. II-996–II-1001. IEEE Computer Society Press, Los Alamitos (2004)
Zhuang, Y., Yang, J., Li, Q.: A Graphic-Theoretic Model for Incremental Relevance Feedback in Image Retrieval. In: Proc. IEEE Int. Conf. on Image Processing 2002, New York, USA, IEEE Computer Society Press, Los Alamitos (2002)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Torres, J.M., Hutchison, D., Reis, L.P. (2007). Semantic Image Retrieval Using Region-Based Relevance Feedback. In: Marchand-Maillet, S., Bruno, E., Nürnberger, A., Detyniecki, M. (eds) Adaptive Multimedia Retrieval: User, Context, and Feedback. AMR 2006. Lecture Notes in Computer Science, vol 4398. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71545-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-540-71545-0_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-71544-3
Online ISBN: 978-3-540-71545-0
eBook Packages: Computer ScienceComputer Science (R0)