SmartSketcher: Sketch-based Image Retrieval with Dynamic Semantic Re-ranking

Portenier, Tiziano; Hu, Qiyang; Favaro, Paolo; Zwicker, Matthias (2017). SmartSketcher: Sketch-based Image Retrieval with Dynamic Semantic Re-ranking. In: EXPRESSIVE 2017 (pp. 1-12). ACM 10.1145/3092907.3092910

Text SmartSketcher- Sketch-based Image Retrieval with Dynamic Semantic Re-ranking.pdf - Published Version Restricted to registered users only Available under License Publisher holds Copyright. Download (20MB) | Request a copy

We present a sketch-based image retrieval system, designed to answer arbitrary queries that may go beyond searching for prede ned object or scene categories. While sketching is fast and intuitive to formulate visual queries, pure sketch-based image retrieval often re- turns many outliers because it lacks a semantic understanding of the query. Our key idea is to combine sketch-based queries with interactive, semantic re-ranking of query results. We leverage progress in deep learning and use a feature representation learned for image classi cation for re-ranking. This allows us to cluster semantically similar images, re-rank based on the clusters, and present more meaningful query results to the user. We report on two large-scale benchmarks and demonstrate that our re-ranking approach leads to signi cant improvements over the state of the art. Finally, a user study designed to evaluate a practical use case con rms the bene ts of our approach.

Interest & Impact

Downloads

1 since deposited on 20 Apr 2018
0 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item