Mining user-contributed photos for personalized product recommendation☆
Introduction
With the rapid development of e-commerce, products recommender system has been exploited to suggest attractive and useful products' information to facilitate user's decision-making process. The intelligent of products recommendation can help users to deal with information overload and provide them personalized services [39]. Product recommendation is popular in e-commerce sites. Some e-commerce sites such as Amazon and Bingshopping recommend products to users based on previous buys as well as what others have been bought when they bought the same product. They keep tracks of users spending and analyze their interests by collaborative filtering [3]. However, in collaborative filtering based products recommendation approaches, only the relevance of users is considered. Thus, they are not personalized to user's interest.
With the booming of social networks, more and more people are will to share their personal affairs, new things and their favorite photos with their friends. For example, Facebook has about one billion users. Flickr is photo sharing website, it also have a very large amount of users. The total number of photos shared by users in Flickr had reached 6 billion by August 2011. There are about 3 million photos uploaded by users each day. The user contributed photos and user generated content can reveal the user's interests very well [44], [45], [46], [50], [51]. Thus the social media websites are the ideal platforms to facilitate the personalized products recommendation.
To improve user experience by making ads relevant to the webpage content, Broder et al. proposed a system for contextual ad matching based on a combination of semantic and syntactic features [40]. They used the semantic phrase to classify the webpage and the ads into taxonomy, and then ranked ads by the proximity of the ads and webpage categories. Although they classified both ads and page content within a large taxonomy, they ignored that some ads are relevant to several topics. For instance, the tag canon is relevant to digital camera and also relevant to the bags of camera. Thus, it is better to represent user interests by topic distribution vector rather than taxonomy.
Taking above mentions into consideration, we propose a novel hierarchical user interest mining (Huim) method to explore user's potential shopping needs based on user-contributed photos in her/his social media sites. We recommend personalized products according to the mined user interests. There are three main problems needed to be solved: (1) the gap between appearances of user-contributed photos and their textual descriptions (i.e. UGC). For example, when a user uploaded some images of her new iphone, she may label images by the words “the amazing apple”. In this circumstance, the tag of apple has two meanings: fruit and electronic product. So we need to mine more content related information from both the visual information and the UGC of user contributed photos. (2) The noise and ambiguous tags in UGC have negative effect to mine user's interests. The textual descriptions contain informal expressions with noise tags (such as the preposition and other non-topic tags) and some ambiguous tags which generated by users. Thus, to recommend personalized products, we need to suppress the noise and ambiguous tags, and to enhanced user interest topics for the user contributed photos. (3) How to measure the relevance of user and product. If a user shared some photos of basketball game, then it is reasonable to suggest some products about basketball like knee pad, and it is also acceptable to recommend basketball video games of 360 Xbox. Among the above three problems measuring the relevance is the core problem in product recommendation.
To fulfill personalized product recommendation, we propose a hierarchical topic vector representation approach to represent user interest and product. Our approach is carried out as follows: (1) tag enrichment for the user contributed photos, (2) introduce a public topic space and map user interest and product descriptions to it and get their topic distribution vectors, (3) enhance user interested topics by a hierarchical approach is proposed to suppress noise and ambiguous textual descriptions, (4) measure the relevance of user and product in the hierarchical public topic space and rank the products for the user.
The main contributions of this paper are summarized as follows: (1) propose a personalized product recommendation system which mining users' interests from their contributed photos; (2) propose an effective user and product relevant measurement approach by introducing a hierarchical public topic space; (3) propose an effective hierarchical user interest representation approach which is robust to suppress noise and ambiguous textual description and enhance user interested topics.
Compared to our preliminary version [50], several improvements are made: (1) the detailed steps of the proposed hierarchical user interest mining approach are provided; (2) we extend the approach from brand recommendation to more general product recommendation, and (3) more experimental results and discussions are provided.
The remainder of this paper is organized as follows. In Section 2, we present the related works on products recommendation, user interest mining and multimedia advertising. In Section 3, our personalized products recommendation based on hierachical user interest mining approach is introduced in detail. Experiments and discussions are given in Section 4 and conclusions are drawn in Section 5.
Section snippets
Related work
In this section, we briefly review the related works on products recommendation, user interest mining from social media, and multimedia advertising.
Overview
This section details our proposed solutions on personalized products recommendation based on user-contributed photos from social media sites. The input of our approach is user shared photos of the same webpage and their corresponding textual descriptions. The system is shown in Fig. 1. Our approach consists of three parts: (1) hierarchical user interest representation. We map user's information (UGC and enriched tags of the photos) to a hierarchical public topic space. And we represent user
Experiments and discussions
To show the effectiveness of the proposed personalized product recommendation approach using Huim, we compare it with Argo [20]. Experiments are conducted on real Flickr users and products of Bingshopping [43].
Conclusion
In this paper, a personalized product recommendation approach is proposed by mining user interest from user-contributed photos in social media sites. User's interests can be well disclosed from their shared photos in a webpage. The topic space can be utilized to measure the bridge the gaps in measuring the relevance of a user and products. The hierarchical structures of topic spaces are valuable for enhancing user's interested topics and suppress noise and ambiguous textual descriptions. The
He Feng was a member of SMILES LAB of Xi'an Jiaotong Univerisity from Sept 2011 to July 2013. He received the M.S. degree in school of electronic and information engineering from Xi'an Jiaotong University, Xi'an, China, in 2013. His interest is related to social recommendation.
References (51)
- et al.
Mining association rules procedure to support on-line recommendation by customers and products fragmentation
Expert Syst. Appl.
(2001) - et al.
Integrating AHP and data mining for product recommendation based on customer lifetime value
Inf. Manage.
(2005) - et al.
An intelligent fuzzy-based recommendation system for consumer electronic products
Expert Syst. Appl.
(2007) - et al.
Tagging photos using users' vocabularies
Neurocomputing
(2013) - A.Evans, M.Fernandez, et al., Adaptive multimedia access: from user needs semantic personalization, in: IEEE...
- et al.
ViSA: virtual spotlighted advertising
ACM Multimedia
(2008) - et al.
Random-walk computation of similar-ities between nodes of a graph, with application to collaborative recommendation
IEEE Trans. Knowl. Data Eng.
(2007) - H.H.Dai, L.Z.Zhao, Z.Q.Nie, J.-R.Wen, L.Wang, Y.Li, Detecting online commercial intention, ACM International WWW...
- et al.
A generic virtual content insertion system based on visual attention analysis
ACM Multimedia
(2008) - H.N.Zhou, Z.Y.Liu, Page frame segmentation for contextural advertising in print on demand books, in: CVPR Workshops,...
Improving ontology-based user profiles
RIAO
Robust goal-mouth detection for virtual content insertion
ACM Multimedia
GameSense: game-like in-image advertising
Multimedia Tools Appl.
Modeling the clickstream: implications for web-based advertising efforts
Marketing Sci.
Contextual in-image advertising
ACM Multimedia
Contextual internet multimedia advertising
IEEE Spec. Issues Internet Vision
VideoSense: a contextual in-video advertising system
IEEE Circuits Syst. Soc.
VideoSense: towards effective online video advertising
ACM Multimedia
AdOn: toward contextual overlay in-video advertising
ACM Multimedia
AdImage: video advertising by image matching and Ad scheduling optimization
ACM SIGIR
Advertising based on users' photos
ICME
Argo: intelligent advertising by mining a user's interest from his photo collections
ACM Data Min. Audience Intell. Advertising
Annotating images by mining image search results, pattern analysis and machine intelligence
IEEE Trans. Pattern Anal. Mach. Intell.
Cited by (40)
Impact of user-generated travel posts on travel decisions: A comparative study on Weibo and Xiaohongshu
2022, Annals of Tourism Research Empirical InsightsSocial media monitoring: What can marketers learn from Facebook brand photos?
2020, Journal of Business ResearchCitation Excerpt :In the field of computer vision, scholars have developed systems to generate recommendations from the photos posted to social media platforms, but they have not evaluated these systems in terms of whether their recommendations successfully match users' interests. Feng and Qian (2014) implement a system that derives personalized product recommendations from social media photos. Wang, Yu, Zhang, Cai, and Ma (2009) present a system that proposes advertisements based on a user's collection of photos.
Salience-aware adaptive resonance theory for large-scale sparse data clustering
2019, Neural NetworksRecommender systems based on multiple social networks correlation
2018, Future Generation Computer SystemsCitation Excerpt :The interpersonal relationship, especially the circles of friends, of social networks makes it possible to solve the cold start and sparsity problem. The rich of social media give us some valuable clues to recommend user favorite items [7], such as music [8], video [9], preferred brand/products [10], user’s preferred tags when sharing a photo to social media networks [11], and user interested travel places by exploring social community contributed photos [12]. Many social network based models [13] have been proposed to improve the performance of the RS.
An intelligent movie recommendation system through group-level sentiment analysis in microblogs
2016, NeurocomputingCitation Excerpt :In the final section, the conclusion is made and more discussions are put together for future efforts.2 Our work is related to two research fields, recommender systems [20] and user generated content mining [21,22]. Recommender system: There exist two types of collaborative filtering approaches, namely, neighborhood-based and model-based.
He Feng was a member of SMILES LAB of Xi'an Jiaotong Univerisity from Sept 2011 to July 2013. He received the M.S. degree in school of electronic and information engineering from Xi'an Jiaotong University, Xi'an, China, in 2013. His interest is related to social recommendation.
Xueming Qian (M'10) received the B.S. and M.S. degrees in Xi'an University of Technology, Xi'an, China, in 1999 and 2004, respectively, and the Ph.D. degree in the School of Electronics and Information Engineering, Xi'an Jiaotong University, Xi'an, China, in 2008. He was awarded Microsoft fellowship in 2006. From 1999 to 2001, he was an Assistant Engineer at Shannxi Daily. From 2008 till now, he is a faculty member of the School of Electronics and Information Engineering, Xi'an Jiaotong University. Now he is an associate professor of the School of Electronics and Information Engineering, Xi'an Jiaotong University. He is the director of SMILES LAB. He was a visit scholar at Microsoft research Asia from Aug. 2010 to March 2011. His research interests include video/image analysis, indexing, and retrieval.
- ☆
This work was supported in part by National Natural Science Foundation of China (NSFC) Project No. 60903121 and No. 61173109, and Microsoft Research Asia.