skip to main content
10.1145/3240167.3240170acmotherconferencesArticle/Chapter ViewAbstractPublication PagesnordichiConference Proceedingsconference-collections
research-article

Evaluation of a search interface for preference-based ranking: measuring user satisfaction and system performance

Published:29 September 2018Publication History

ABSTRACT

Finding a product online can be a challenging task for users. Faceted search interfaces, often in combination with recommenders, can support users in finding a product that fits their preferences. However, those preferences are not always equally weighted: some might be more important to a user than others (e.g. red is the favorite color, but blue is also fine) and sometimes preferences are even contradictory (e.g. the lowest price vs. the highest performance). Often, there is even no product that meets all preferences. In those cases, faceted search interfaces reach their limits. In our research, we investigate the potential of a search interface, which allows a preference-based ranking based on weighted search and facet terms. We performed a user study with 24 participants and measured user satisfaction and system performance. The results show that with the preference-based search interface the users were given more alternatives that best meet their preferences and that they are more satisfied with the selected product than with a search interface using standard facets. Furthermore, in this work we study the relationship between user satisfaction and search precision within the whole search session and found first indications that there might be a relation between them.

References

  1. Svetlin Bostandjiev, John O'Donovan, and Tobias Höllerer. 2012. TasteWeights: a visual interactive hybrid recommender system. In Proceedings of the sixth ACM conference on Recommender systems. ACM, 35--42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. R. D. Burke, K. J. Hammond, and B. C. Yound. 1997. The FindMe approach to assisted browsing. IEEE Expert 12, 4 (Jul 1997), 32--40. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Claudio Carpineto and Giovanni Romano. 2012. A Survey of Automatic Query Expansion in Information Retrieval. ACM Comput. Surv. 44, 1 (Jan. 2012), 1:1--1:50. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Li Chen and Pearl Pu. 2012. Critiquing-based recommenders: survey and emerging trends. User Modeling and User-Adapted Interaction 22, 1 (2012), 125--150. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. William S Cleveland. 1979. Robust locally weighted regression and smoothing scatterplots. Journal of the American statistical association 74, 368 (1979), 829--836.Google ScholarGoogle ScholarCross RefCross Ref
  6. Shuguang Han, Danchen Zhang, Daqing He, and Qikai Cheng. 2016. User exploration of slider facets in interactive people search system. IConference 2016 Proceedings (2016).Google ScholarGoogle ScholarCross RefCross Ref
  7. F. Maxwell Harper, Funing Xu, Harmanpreet Kaur, Kyle Condiff, Shuo Chang, and Loren Terveen. 2015. Putting Users in Control of Their Recommendations. In Proceedings of the 9th ACM Conference on Recommender Systems (RecSys '15). ACM, New York, NY, USA, 3--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Marti Hearst, Ame Elliott, Jennifer English, Rashmi Sinha, Kirsten Swearingen, and Ka-Ping Yee. 2002. Finding the Flow in Web Site Search. Commun. ACM 45, 9 (Sept. 2002),42--49. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Kalervo Järvelin and Jaana Kekäläinen. 2000. IR Evaluation Methods for Retrieving Highly Relevant Documents. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '00). ACM, New York, NY, USA, 41--48. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Matt Lindner. 2016. "Online sales will reach $523 billion by 2020 in the U.S.". Online. (29 January 2016). Retrieved September 20, 2016 from https://www.internetretrailer.com/2016/02/29/online-sales-will-reach-523-billion-2020-us.Google ScholarGoogle Scholar
  11. Benedikt Loepp, Katja Herrmanny, and Jürgen Ziegler. 2015. Blended Recommending: Integrating Interactive Information Filtering and Algorithmic Recommender Techniques. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 975--984. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. David McSherry. 2003. Similarity and Compromise. Springer Berlin Heidelberg, Berlin, Heidelberg, 291--305. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Denis Parra, Peter Brusilovsky, and Christoph Trattner. 2014. See What You Want to See: Visual User-driven Approach for Hybrid Recommendation. In Proceedings of the 19th International Conference on Intelligent User Interfaces (IUI '14). ACM, New York, NY, USA, 235--240. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Hans peter Frei and Yonggang Qiu. 1993. Effectiveness of Weighted Searching in an Operational IR Environment. (1993).Google ScholarGoogle Scholar
  15. Pearl Pu and Li Chen. 2005. Integrating Tradeoff Support in Product Search Tools for e-Commerce Sites. In Proceedings of the 6th ACM Conference on Electronic Commerce (EC '05). ACM, New York, NY, USA, 269--278. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Pearl Huan Z. Pu and Pratyush Kumar. 2004. Evaluating Example-based Search Tools. In Proceedings of the 5th ACM Conference on Electronic Commerce (EC '04). ACM, New York, NY, USA, 208--217. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Francesco Ricci, Lior Rokach, and Bracha Shapira. 2011. Introduction to Recommender Systems Handbook. Springer US, Boston, MA, 1--35. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. J. J. Rocchio. 1971. Relevance feedback in information retrieval. In The Smart retrieval system - experiments in automatic document processing, G. Salton (Ed.). Englewood Cliffs, NJ: Prentice-Hall, 313--323.Google ScholarGoogle Scholar
  19. Nicolas Spyratos and Vassilis Christophides. 2006. Querying with Preferences in a Digital Library. In Proceedings of the 2005 International Conference on Federation over the Web. Springer-Verlag, Berlin, Heidelberg, 130--142. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Markus Stolze. 2000. Soft navigation in electronic product catalogs. International Journal on Digital Libraries 3, 1 (2000), 60--66.Google ScholarGoogle ScholarCross RefCross Ref
  21. Daniel Tunkelang. 2009. Faceted Search. Synthesis Lectures on Information Concepts, Retrieval, and Services 1, 1 (2009), 1--80. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Martin Voigt, Artur Werstler, Jan Polowinski, and Klaus Meissner. 2012. Weighted Faceted Browsing for Characteristics-based Visualization Selection Through End Users. In Proceedings of the 4th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS '12). ACM, New York, NY, USA, 151--156. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Bifan Wei, Jun Liu, Qinghua Zheng, Wei Zhang, Xiaoyu Fu, and Boqin Feng. 2013. A Survey of Faceted Search. J. Web Eng. 12, 1-2 (Feb. 2013), 41--64. http://dl.acm.org/citation.cfm?id=2481562.2481564 Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Ka-Ping Yee, Kirsten Swearingen, Kevin Li, and Marti Hearst. 2003. Faceted Metadata for Image Search and Browsing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, New York, NY, USA, 401--408. Google ScholarGoogle ScholarDigital LibraryDigital Library

Recommendations

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Sign in
  • Published in

    cover image ACM Other conferences
    NordiCHI '18: Proceedings of the 10th Nordic Conference on Human-Computer Interaction
    September 2018
    1002 pages
    ISBN:9781450364379
    DOI:10.1145/3240167

    Copyright © 2018 ACM

    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    • Published: 29 September 2018

    Permissions

    Request permissions about this article.

    Request Permissions

    Check for updates

    Qualifiers

    • research-article

    Acceptance Rates

    NordiCHI '18 Paper Acceptance Rate59of240submissions,25%Overall Acceptance Rate379of1,572submissions,24%

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader