Research Article
Evolutionary Study of Web Spam: Webb Spam Corpus 2011 versus Webb Spam Corpus 2006
@INPROCEEDINGS{10.4108/icst.collaboratecom.2012.250689, author={De Wang and Danesh Irani and Calton Pu}, title={Evolutionary Study of Web Spam: Webb Spam Corpus 2011 versus Webb Spam Corpus 2006}, proceedings={8th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing}, publisher={IEEE}, proceedings_a={COLLABORATECOM}, year={2012}, month={12}, keywords={web spam evolutionary spam corpus}, doi={10.4108/icst.collaboratecom.2012.250689} }- De Wang
Danesh Irani
Calton Pu
Year: 2012
Evolutionary Study of Web Spam: Webb Spam Corpus 2011 versus Webb Spam Corpus 2006
COLLABORATECOM
ICST
DOI: 10.4108/icst.collaboratecom.2012.250689
Abstract
With over 2.5 hours a day spent browsing websites online[1] and with over a billion pages[2], identifying and detecting web spam is an important problem. Although large corpora of legitimate web pages are available to researchers, the same cannot be said about web spam or spam web pages.
We introduce the Webb Spam Corpus 2011 -- a corpus of approximately 330,000 spam web pages -- which we make available to researchers in the fight against spam. By having a standard corpus available, researchers can collaborate better on developing and reporting results of spam filtering techniques. The corpus contains web pages crawled from links found in over 6.3 million spam emails. We analyze multiple aspects of this corpus including redirection, HTTP headers and web page content.
We also provide insights into changes in web spam since the last Webb Spam Corpus was released in 2006. These insights include: 1) spammers manipulate social media in spreading spam; 2) HTTP headers also change over time (e.g. hosting IP addresses of web spam appear in more IP ranges); 3) Web spam content has evolved but the majority of content is still scam.