ABSTRACT
The advent of media-sharing sites like Flickr has drastically increased the volume of community-contributed multimedia resources on the web. However, due to their magnitudes, these collections are increasingly difficult to understand, search and navigate. To tackle these issues, a novel search system, ContextSeer, is developed to improve search quality (by reranking) and recommend supplementary information (i.e., search-related tags and canonical images) by leveraging the rich context cues, including the visual content, high-level concept scores, time and location metadata. First, we propose an ordinal reranking algorithm to enhance the semantic coherence of text-based search result by mining contextual patterns in an unsupervised fashion. A novel feature selection method, wc-tf-idf is also developed to select informative context cues. Second, to represent the diversity of search result, we propose an efficient algorithm cannoG to select multiple canonical images without clustering. Finally, ContextSeer enhances the search experience by further recommending relevant tags. Besides being effective and unsupervised, the proposed methods are efficient and can be finished at query time, which is vital for practical online applications. To evaluate ContextSeer, we have collected 0.5 million consumer photos from Flickr and manually annotated a number of queries by pooling to form a new benchmark, Flickr550. Ordinal reranking achieves significant performance gains both in Flcikr550 and TRECVID search benchmarks. Through a subjective test, cannoG expresses its representativeness and excellence for recommending multiple canonical images.
- L. Kennedy et al, "How Flickr helps us make sense of the world: Context and content in community-contributed media collections," ACM Multimedia, pp. 631--640, 2007. Google ScholarDigital Library
- M. Ames et al, "Why we tag: Motivations for annotation in mobile and online media," ACM CHI, pp. 971--980, 2007. Google ScholarDigital Library
- L. Kennedy, S.-F. Chang, and I. Kozintsev, "To search or to label?: Predicting the performance of search-based automatic image classifiers," Proc. ACM Int. workshop on Multimedia information retrieval, pp. 249--258, 2006. Google ScholarDigital Library
- A. K. Dey, "Understanding and using context," Personal and Ubiquitous Computing, vol. 5, no. 1, 2001. Google ScholarDigital Library
- M. Naphade et al, "Large-scale concept ontology for multimedia," IEEE Multimedia Magazine, vol. 13, no. 3, pp. 86--91, 2006. Google ScholarDigital Library
- K. Toyama et al, "Geographic location tags on digital images," ACM Multimedia, pp. 156--166, 2003. Google ScholarDigital Library
- W. Hsu et al, "Video search reranking via information bottleneck principle," ACM Multimedia, pp. 35--44, 2006. Google ScholarDigital Library
- W. Hsu, L. Kennedy, and S.-F. Chang, "Video search reranking through random walk over document-level context graph," Proc. ACM Multimedia, pp. 971--980, 2007. Google ScholarDigital Library
- L. Kennedy and S.-F. Chang, "A reranking approach for context-based concept fusion in video indexing and retrieval," ACM CIVR, pp. 333--340, 2007. Google ScholarDigital Library
- Y.-H. Yang and W.-H. Hsu, "Video search reranking via online ordinal reranking," IEEE ICME, 2008.Google Scholar
- A. Natsev et al, "Semantic concept-based query expansion and re-ranking for multimedia retrieval," ACM Multimedia, pp. 991--1000, 2007. Google ScholarDigital Library
- J. Battelle, The Search: How Google and Its Rivals Rewrote the Rules of Business and Transformed Our Culture, 2006. Google ScholarDigital Library
- NIST TREC Video Retrieval Evaluation. {online} http://www-nlpir.nist.gov/projects/trecvid/.Google Scholar
- X. Li et al, "Video search in concept subspace: a text like paradigm," ACM CIVR, pp. 603--610, 2007. Google ScholarDigital Library
- A. Aizawa, "An information-theoretic perspective of tf-idf measures," Information Processing and Management, vol. 39, pp. 45--65, 2003. Google ScholarDigital Library
- S. Palmer et al, "Canonical perspective and the perception of objects," Attention and Performance IX, pp. 135--151, 1981.Google Scholar
- L. Kennedy et al, "Generating diverse and representative image search results for landmarks," WWW, 2008. Google ScholarDigital Library
- Y. Jing et al, "Canonical image selection from the web," ACM CIVR, pp. 280--287, 2007. Google ScholarDigital Library
- R. Yan, A. Hauptmann, and R. Jin, "Multimedia search with pseudo-relevance feedback," ACM CIVR, 2003. Google ScholarDigital Library
- R. Herbrich et al, "Support vector learning for ordinal regression," IEEE ICANN, pp. 97--102, 1999.Google Scholar
- Z. Cao et al, "Learning to rank: from pairwise approach to listwise approach," IEEE ICML, pp. 129--136, 2007. Google ScholarDigital Library
- S.-F. Chang et al, "Columbia University TRECVID-2005 video search and high-level feature extraction," NIST TRECVID workshop, 2005.Google Scholar
- I. Simon et al, "Scene summarization for online image collections," IEEE ICCV, pp. 1--8, 2007.Google Scholar
- S. Wang et al, "IGroup: presenting web image search results in semantic clusters," ACM CHI, 2007, pp. 377--384. Google ScholarDigital Library
- N. J. Belkin, "Helping people find what they don't know," Communication of the ACM, vol. 43, no. 8, pp. 58--61, 2000. Google ScholarDigital Library
- C.-K. Huang et al, "Relevant term suggestion in interactive web search based on contextual information in query session logs," Journal of the American Society for Information Science and Technology, vol. 54, no. 7, pp. 638--649, 2003. Google ScholarDigital Library
- R. Jones, B. Rey, O. Madani, and W. Greiner, "Generating query substitutions," ACM WWW, 2006. Google ScholarDigital Library
- J. Xu and W. Croft, "Query expansion using local and global document analysis," ACM SIGIR, pp. 4--11, 1996. Google ScholarDigital Library
- J. Sivic and A. Zisserman, "Video Google: A text retrieval approach to object matching in videos, ICCV, 2003. Google ScholarDigital Library
- K. Mikolajczyk and C. Schmid, "Scale & affine invariant interest point detectors," IJCV, vol.60, no.1, pp. 63--86, 2004. Google ScholarDigital Library
- D. Lowe, "Distinctive image features from scale-invariant keypoints," IJCV, vol. 60, no. 2, pp. 91--110, 2004. Google ScholarDigital Library
- S. Sontag, On Photography, Picador USA, 2001.Google Scholar
- L. Page et al., "The PageRank citation ranking: Bringing order to the web," Stanford University, 1998.Google Scholar
Index Terms
- ContextSeer: context search and recommendation at query time for shared consumer photos
Recommendations
Context-based ranking in folksonomies
HT '09: Proceedings of the 20th ACM conference on Hypertext and hypermediaWith the advent of Web 2.0 tagging became a popular feature. People tag diverse kinds of content, e.g. products at Amazon, music at Last.fm, images at Flickr, etc. Clicking on a tag enables the users to explore related content. In this paper we ...
Leveraging multi-faceted tagging to improve search in folksonomy systems
HT '10: Proceedings of the 21st ACM conference on Hypertext and hypermediaIn this paper we present ranking algorithms for folksonomy systems that exploit additional contextual information attached to tag assignments available. We evaluate the algorithms in the TagMe! system, a tagging front-end for Flickr, and show that our ...
Towards a Relevant and Diverse Search of Social Images
Recent years have witnessed the great success of social media websites. Tag-based image search is an important approach to accessing the image content on these websites. However, the existing ranking methods for tag-based image search frequently return ...
Comments