Qualitative analysis of user-based and item-based prediction algorithms for recommendation agents

Abstract

Recommendation agents employ prediction algorithms to provide users with items that match their interests. In this paper, several prediction algorithms are described and evaluated, some of which are novel in that they combine user-based and item-based similarity measures derived from either explicit or implicit ratings. Both statistical and decision-support accuracy metrics of the algorithms are compared against different levels of data sparsity and different operational thresholds. The first metric evaluates the accuracy in terms of average absolute deviation, while the second evaluates how effectively predictions help users to select high-quality items. The experimental results indicate better performance of item-based predictions derived from explicit ratings in relation to both metrics. Category-boosted predictions lead to slightly better predictions when combined with explicit ratings, while implicit ratings, in the context that have been defined in this paper, perform much worse than explicit ratings.

Introduction

Recommendation systems (Resnick and Varian, 1997) have been a popular topic of research ever since the ubiquity of the web made it clear that people of hugely varying backgrounds would be able to access and query the same underlying data. The initial human–computer interaction challenge has been made even more challenging by the observation that customized services require sophisticated data structures and well thought-out architectures to be able to scale up to thousands of users and beyond.

In recent years, recommendation agents are extensively adopted by both research and e-commerce recommendation systems in order to provide an intelligent mechanism to filter out the excess of information available and to provide customers with the prospect to effortlessly find out items that they will probably like according to their logged history of prior transactions.

Recommendation agents need to employ efficient prediction algorithms so as to provide accurate recommendations to users. If a prediction is defined as a value that expresses the predicted likelihood that a user will “like” an item, then a recommendation is defined as the list of n items with respect to the top-n predictions from the set of items available. Improved prediction algorithms indicate better recommendations. This explains the essentiality of exploring and understanding the broad characteristics and potentials of prediction algorithms and the reason why this work concentrates on this research direction.

There are generally two methods to formulate recommendations both depending on the type of items to be recommended, as well as, on the way that user models (Allen, 1990) are constructed. The two different approaches are content-based (Balabanovic and Sholam, 1997; Kalles et al., 2003) and collaborative filtering (Herlocker et al., 2000; Hofmann, 2003), while additional hybrid techniques have been proposed as well (Balabanovic and Sholam, 1997).

Content based recommendation algorithms: Content-based algorithms are principally used when documents are to be recommended, such as web pages, publications, jokes or news. The agent maintains information about user preferences either by initial input about user's interests during the registration process or by rating documents. Recommendations are formed by taking into account the content of documents and by filtering in the ones that better match the user's preferences and logged profile.

Collaborative filtering based recommendation algorithms: Collaborative-filtering algorithms aim to identify users that have relevant interests and preferences by calculating similarities and dissimilarities between user profiles (Herlocker et al., 2004). The idea behind this method is that, it may be of benefit to one's search for information to consult the behavior of other users who share the same or relevant interests and whose opinion can be trusted.

The challenges for recommendation algorithms expand to three key dimensions, identified as sparsity, scalability and cold-start.

Sparsity: Even users that are very active, result in rating just a few of the total number of items available in a database. As the majority of the recommendation algorithms are based on similarity measures computed over the co-rated set of items, large levels of sparsity can be detrimental to recommendation agents. In Huang et al. (2004), authors propose to deal with sparsity problem by applying an associative retrieval framework and related spreading activation algorithms to explore transitive associations among consumers through their past transactions and feedback.

Scalability: Recommendation algorithms seem to be efficient in filtering in items that are interesting to users. However, they require computations that are very expensive and grow non-linearly with the number of users and items in a database. Therefore, in order to bring recommendation algorithms successfully on the web, and succeed in providing recommendations with acceptable delay, sophisticated data structures and advanced, scalable architectures are required. In Cosley et al. (2002), authors describe an open framework for practical testing of recommendation systems in an attempt to provide a standard, public testbed to evaluate recommendation algorithms in real-world conditions.

Cold-start: An item cannot be recommended unless it has been rated by a substantial number of users. This problem applies to new and obscure items and is particularly detrimental to users with eclectic taste (Schein et al., 2002; Melville et al., 2002). Likewise, a new user has to rate a sufficient number of items before the recommendation algorithm be able to provide reliable and accurate recommendations.

The primary contributions of this work are:

• The utilization of explicit ratings in an “implicit” sense so as to enrich a user's model, without actually prompting users to express their preference to categories.

• The description of item-based and user-based similarity measures derived from either explicit or implicit ratings.

• The formation of a range of item-based and user-based prediction algorithms according to item-based and user-based similarity measures.

• The qualitative analysis and experimental evaluation of presented prediction algorithms.

Section 2 describes a set of similarity measures to compare the relevance between users or items. Section 3 describes a set of existing and newly introduced prediction algorithms that integrate the similarity measures. Section 4 presents the experimental evaluation metrics that are employed in order to compare the algorithms and the results of the evaluation are discussed. Section 5 summarizes the contributions of this work and draws directions for further research.