Skip to main content
SpringerLink
Log in

Learning from User Behavior in Image Retrieval: Application of Market Basket Analysis

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This article describes an approach to learn feature weights for content-based image retrieval (CBIR) from user interaction log files. These usage log files are analyzed for images marked together by a user in the same query step. The problem is somewhat similar to one of the traditional data mining problems, the market basket analysis problem, where items bought together in a supermarket are analyzed. This paper outlines similarities and differences between the two fields and explains how to use the interaction data for deriving a better feature weighting.

Experiments with existing log files are done and a significant improvement in performance is reached with a feature weighting calculated from the information contained in the log files. Even with several steps of relevance feedback the results remain much better than without the learning, which means that not only information from feedback is taken into account earlier, but a better quality of retrieval is reached in all steps.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Agrawal, R., Imielinski, T., and Swami, A. 1993. Mining association rules between sets of items in large databases. In Proceedings of the 1993 ACM SIGMOD Conference, Washington DC, USA, pp. 207–216.

  • Agrawal, R. and Srikant, R. 1994. Fast algorithms for mining association rules. In Proceedings of the 20th VLDB Conference Santiago, Chile, pp. 487–499.

  • Annotated groundtruth database. 1999. Department of Computer Science and Engineering, University of Washington, http:// www.cs.washington.edu/research/image-database/ groundtruth/.

  • Berendt, B. and Spiliopoulou, M. 2000. Analysis of navigation behaviour in web sites integrating multiple information systems. VLDB Journal: Special Issue on Databases and theWeb, 9(1):56–75.

    Google Scholar 

  • Brin, S., Motwani, R., and Silverstein, C. 1997. Beyond market baskets: Generalizing association rules to correlations. In Proceedings of the Annual International ACM SIGMOD Conference on Research and Development in Management of Data (SIGMOD'97), J. Peckham (Ed.), Tuscon AR, USA, pp. 255–264.

  • Cox, I.J., Miller, M.L., Omohundro, S.M., and Yianilos, P.N. 1996. Target testing and the PicHunter Bayesian multimedia retrieval system. In Advances in Digital Libraries (ADL'96), Library of Congress: Washington, DC, pp. 66–75.

    Google Scholar 

  • Han, J., Pei, J., and Yin, Y. 2000. Mining frequent patterns without candicate generation. In Proceedings of the Annual International ACMSIGMODConference on Research andDevelopment in Management of Data (SIGMOD2000), W. Chen, J.F. Naughton, and P.A. Bernstein (Eds.), Dallas, TX, USA.

  • Harman, D. 1992. Overview of the first Text REtrieval Conference (TREC–1). In Proceedings of the First Text REtrieval Conference (TREC–1), Washington DC, USA, pp. 1–20.

  • Hipp, J., Gützer, U., and Nakhaeizadeh, G. 2000. Algorithms for association rule mining—A general survey and comparison. SIGKDD Explorations, 2(1):58–64.

    Google Scholar 

  • Jermain, C. and Miller, R.J. 2001. Association mining without support thresholds. Technical report, Georgia Institute of Technology

  • Kohrs, A. and Merialdo, B. 1999. Clustering for collaborative filtering applications. In Proceedings of the International Conference on Computational Intelligence for Modelling Control andAutomation. IOS Press: Vienna, Austria, pp. 199–204.

    Google Scholar 

  • Lee, C.S., Ma, W.-Y., and Zhang, H. 1999. Information embedding based on user's relevance feedback in image retrieval. In Multimedia Storage and Archiving Systems IV(VV03) vol. 3246 of SPIE Proceedings, S. Panchanathan, S.-F. Chang and C.-C.J. Kuo (Eds.), pp. 294–304 (SPIE Symposium on Voice, Video and Data Communications).

  • Li, B., Chang, E., and Li, C.-S. 2001. Learning image query concepts via intelligent sampling. In Proceedings of the Second International Conference on Multimedia and Exposition (ICME'2001), IEEE Computer Society: Tokyo, Japan, pp. 1168–1171.

    Google Scholar 

  • Li, M., Chen, Z., Wenyin, L., and Zhang, H.-J. 2001. A statistical correlation model for image retrieval. In Proceedings of the ACM MultimediaWorkshop on Multimedia Information Retrieval (ACM MIR 2001), The Association for Computing Machinery: Ottawa, Canada, pp. 42–45.

    Google Scholar 

  • Ma, W.Y., Deng, Y., and Manjunath, B.S. 1997. Tools for texture and color-based search of images. In Human Vision and Electronic Imaging II, volume 3016 of SPIE Proceedings, B.E. Rogowitz and T.N. Pappas (Eds.), San Jose, CA, pp. 496–507.

  • Mannila, H. and Toivonen, H. 1996. Discovering generalized episodes using minimal occurences. In Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining (SIGKDD1996), Portland, OR, USA, pp. 146–151.

  • Minka, T. 1996. An image database browser that learns from user interaction. Master's thesis, MIT Media Laboratory, 20 Ames St., Cambridge, MA 02139.

    Google Scholar 

  • Müller, H., Müller, W., Squire, D.M., Marchand-Maillet, S., and Pun, T. 2000a. Learning feature weights from user behavior in contentbased image retrieval. In ACMSIGKDD International Conference on Knowledge Discovery and Data Mining (Workshop on Multimedia Data Mining MDM/KDD2000), S. Simoff and O. Zaiane (Eds.). Boston, MA, USA, pp. 67–72.

  • Müller, H., Müller, W., Squire, D.M., Marchand-Maillet, S., and Pun, T. 2000b. Strategies for positive and negative relevance feedback in image retrieval. In Proceedings of the 15th International Conference on Pattern Recognition (ICPR 2000), A. Sanfeliu, J.J. Villanueva, M. Vanrell, R. Alcézar, J.-O. Eklundh, and Y. Aloimonos (Eds.) IEEE: Barcelona, Spain, pp. 1043–1046.

    Google Scholar 

  • Müller, H., Müller, W., Squire, D.M., Marchand-Maillet, S., and Pun, T. 2001. Performance evaluation in content-based image retrieval: Overview and proposals. Pattern Recognition Letters, 22(5):593–601.

    Google Scholar 

  • Müller, W., Pečenović, Z., Müller, H., Marchand-Maillet, S., Pun, T., Squire, D.M., Vries, A.P.D., and Giess, C. 2000c. MRML: An extensible communication protocol for interoperability and benchmarking of multimedia information retrieval systems. In SPIE Photonics East—Voice, Video, and Data Communications, Boston, MA, USA, pp. 961–968.

  • Müller, W., Squire, D.M., Müller, H., and Pun, T. 1999. Hunting moving targets: An extension to Byesian methods in multimedia databases. In Multimedia Storage and Archiving Systems IV(VV02), vol. 3846 of SPIE Proceedings, S. Panchanathan, S.-F. Chang, and C.-C.J. Kuo (Eds.), pp. 328–337 (SPIE Symposium on Voice, Video and Data Communications).

  • Panchanathan, S., Chang, S.-F., and Kuo, C.-C.J. (Eds.). 1999. Multimedia Storage and Archiving Systems IV (VV02), vol. 3846 of SPIE Proceedings. Boston, Massachusetts, USA. (SPIE Symposium on Voice, Video and Data Communications).

  • Rui, Y., Huang, T.S., Ortega, M., and Mehrotra, S. 1998. Relevance feedback: A power tool for interactive content-based image retrieval. IEEE Transactions on Circuits and Systems for Video Technology, 8(5):644–655. (Special Issue on Segmentation, Description, and Retrieval of Video Content).

    Google Scholar 

  • Salton, G. and Buckley, C. 1988. Term weighting approaches in automatic text retrieval. Information Processing and Management, 24(5):513–523.

    Google Scholar 

  • Salton, G. and Buckley, C. 1990. Improving retrieval performance by relevance feedback. Journal of the American Society for Information Science, 41(4):288–287.

    Google Scholar 

  • Savasere, A., Omiecinski, E., and Navathe, S. 1995. An efficient algorithm for mining association rules in large databases. In Proceedings of the 22nd International Conference on Very Large Databases (VLDB'95), U. Dayal, P.M.D. Gray, and S. Nishio (Eds.), Zürich, Switzerland.

  • Shekhar, S. and Huang, Y. 2001. Discovering spatial co-location patterns: A summary of results. In Proceedings of the 7th International Symposium on Spatial and Temporal Databases, Retondo Beach CA, USA.

  • Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., and Jain, R. 2000. Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12):1349–1380.

    Google Scholar 

  • Smith, J.R. 1998. Image retrieval evaluation. In IEEE Workshop on Content-Based Access of Image andVideo Libraries (CBAIVL'98). Santa Barbara, CA, USA, pp. 112–113.

  • Smith, J.R. and Chang, S.-F. 1996. VisualSEEk: A fully automated content-based image query system. In The Fourth ACM International Multimedia Conference and Exhibition. Boston, MA, USA.

  • Squire, D.M., Müller, W., Müller, H., and Pun, T. 2000. Content-based query of image databases: Inspirations from text retrieval. In Pattern Recognition Letters (Selected Papers from The 11th Scandinavian Conference on Image Analysis SCIA' 99), B.K. Ersboll and P. Johansen (Eds.), 21(13/14):1193–1198.

  • Vasconcelos, N. and Lippman, A. 2000. Learning over multiple temporal scales in image databases. In 6th European Conference on Computer Vision (ECCV2000), D. Vernon (Ed.), number 1842 in Lecture Notes in Computer Science. Springer-Verlag: Dublin, Ireland, pp. 33–47.

    Google Scholar 

  • Voorhees, E.M. and Harmann, D. 1998. Overview of the seventh Text REtrieval Conference (TREC–7). In The Seventh Text Retrieval Conference. Gaithersburg, MD, USA, pp.1–23.

  • Worring, M., Smeulders, A.W.M., and Santini, S. 2000. Interaction in content-based image retrieval: An evaluation of the state of the art. In Fourth International Conference on Visual Information Systems (VISUAL'2000), R. Laurini (Ed.), number 1929 in Lecture Notes in Computer Science, Springer–Verlag: Lyon, France, pp. 26–36.

    Google Scholar 

  • Wu, K.-L., Yu, P.S., and Ballman, A. 1998. Speedtracer: A web usage mining and analysis tool. IBM Systems Journal on Internet Computing, 37(1):89–105.

    Google Scholar 

  • Zaki, M.J., Parthgasarathy, S., Ogihara, M., and Li, W. 1997. New algorithms for fast discovery of association rules. In Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining (SIGKDD1997). Newport Beach, CA, USA.

Download references

Author information

Authors and Affiliations

  1. Computer Vision Group, University of Geneva, 24 Rue du Général Dufour, CH-1211, Genève 4, Switzerland

    Henning Müller & Thierry Pun

  2. Computer Science and Software Engineering, Monash University, Melbourne, Australia

    David Squire

Authors
  1. Henning Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thierry Pun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Squire
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Müller, H., Pun, T. & Squire, D. Learning from User Behavior in Image Retrieval: Application of Market Basket Analysis. International Journal of Computer Vision 56, 65–77 (2004). https://doi.org/10.1023/B:VISI.0000004832.02269.45

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:VISI.0000004832.02269.45

Search

Navigation