Abstract
Automatic image annotation has been intensively studied for content-based image retrieval recently. In this paper, we propose a novel approach to automatic image annotation based on two key components: (a) an adaptive visual feature representation of image contents based on matching pursuit algorithms; and (b) an adaptive two-level segmentation method. They are used to address the important issues of segmenting images into meaningful units, and representing the contents of each unit with discriminative visual features. Using a set of about 800 training and testing images, we compare these techniques in image retrieval against other popular segmentation schemes, and traditional non-adaptive feature representation methods. Our preliminary results indicate that the proposed approach outperforms other competing systems based on the popular Blobworld segmentation scheme and other prevailing feature representation methods, such as DCT and wavelets. In particular, our system achieves an F1 measure of over 50% for the image annotation task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aslandogan, Y.A., Yu, C.T.: „Multiple evidence combination in image retrieval: Diohenese searches for people on the web. In: ACM SIGIR 2000, Athens, Greece (2000)
Bergeaud, F., Mallat, S.: „Matching pursuits of images. In: Proc. IEEE ICIP 1995, Washington DC, October 1995, vol. 1, pp. 53–56 (1995)
Carson, C., Thomas, M., Belongie, S., Hellerstein, J.M., Malik, J.: „Blobworld: A system for region-based image indexing and retrieval. In: Proc. Int’l Conf. Visual Information System (1999)
Chang, C.-C., Lin, C.-J.: „Training nu-support vector classifiers: theory and algorithms. Neural Computation 13(9), 2119–2147 (2001)
Deng, Y., Manjunath, B.S., Shin, H.: „Color image segmentation. In: IEEE CVPR (1999)
Duygulu, P., Vural, F.Y.: „Multi-Level image segmentation and object representation for content based image retrieval. In: SPIE Electronic Imaging 2001, Storage and Retrieval for Media Databases, San Jose, CA, January 21-26 (2001)
Feng, H.M., Chua, T.S.: „A Bootstrapping Approach to Annotating Large Image Collection. In: ACM SIGMM International Workshop on Multimedia Information Retrieval, Berkeley, November 2003, pp. 55–62 (2003)
Jeon, J., Lavrenko, V., Manmatha, R.: „Automatic image annotation and retrieval using cross-media relevance models. In: ACM SIGIR 2003, July 28-August 1 (2003)
Lee, C.-H., Soong, F.K., Paliwal, K.K.: Automatic Speech and Speaker Recognition: Advanced Topics. Kluwer Academic Press, Dordrecht (1996)
Mallat, S.G., Zhang, Z.F.: „Matching pursuits with time-frequency dictionaries. IEEE Trans. on Signal Processing 41(12), 3397–3415 (1993)
Manjunath, B., Ma, W.: „Texture features for browsing and retrieval of image data. IEEE Trans. on Pattern Analysis and Machine Intelligence 18(8), 837–842 (1996)
Mori, Y., Takahashi, H., Oka, R.: „Image-to-word transformation based on dividing and vector quantizing images with words. Proc. of First International Journal of Computer Vision 40(2), 99–121 (2000)
Rissanen, J.: „Modeling by shortest data description. Automatica 14, 465–471 (1978)
Rodden, K.: „How do people organize their photographs? In: BCS IRSG 21st Ann. Colloq. on Info. Retrieval Research (1999)
Shenier, M., Abedel-Mottaleb, M.: „Exploiting the JPEG compression scheme for image retrieval. IEEE Trans. on Pattern Analysis and Machine Intelligence 18(8), 849–853 (1996)
Smith, J.R., Chang, S.F.: „VisualSeek: A fully automated content-based query system. ACM Multimedia (1996)
Smith, J.R., Li, C.S.: „Image classification and querying using composite region templates. Journal of Computer Vision and Image Understanding (2000)
Swain, M., Ballard, D.: „Color indexing. International Journal of Computer Vision 7(1), 11–32 (1991)
Szummer, M., Picard, R.W.: „Indoor-outdoor image classification. In: IEEE Intl Workshop on Content-based Access of Image and Video Databases (January 1998)
Tuceryan, M., Jain, A.K.: „Texture analysis. In: Handbook Pattern Recognition and Computer Vision. ch. 2, pp. 235–276. World Scientific, Singapore (1993)
Unser, M.: „Texture classification and segmentation using wavelet frames. IEEE Trans. on Image Processing 4(11), 1549–1560 (1995)
Valois, R.D., Valois, K.D.: „Spatial Vision. Oxford, New York (1988)
Wang, J.Z., Li, J.: „Learning-based linguistic indexing of pictures with 2-D MHMMs. In: Proc. ACM Multimedia, Juan Les Pins, France, December 2002, pp. 436–445 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Shi, R., Feng, H., Chua, TS., Lee, CH. (2004). An Adaptive Image Content Representation and Segmentation Approach to Automatic Image Annotation. In: Enser, P., Kompatsiaris, Y., O’Connor, N.E., Smeaton, A.F., Smeulders, A.W.M. (eds) Image and Video Retrieval. CIVR 2004. Lecture Notes in Computer Science, vol 3115. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-27814-6_64
Download citation
DOI: https://doi.org/10.1007/978-3-540-27814-6_64
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22539-3
Online ISBN: 978-3-540-27814-6
eBook Packages: Springer Book Archive