ABSTRACT
Conversational search engines such as YouChat and Microsoft Copilot use large language models (LLMs) to generate responses to queries. It is only a small step to also let the same technology insert ads within the generated responses - instead of separately placing ads next to a response. Inserted ads would be reminiscent of native advertising and product placement, both of which are very effective forms of subtle and manipulative advertising. Considering the high computational costs associated with LLMs, for which providers need to develop sustainable business models, users of conversational search engines may very well be confronted with generated native ads in the near future. In this paper, we thus take a first step to investigate whether LLMs can also be used as a countermeasure, i.e., to block generated native ads. We compile the Webis Generated Native Ads 2024 dataset of queries and generated responses with automatically inserted ads, and evaluate whether LLMs or fine-tuned sentence transformers can detect the ads. In our experiments, the investigated LLMs struggle with the task but sentence transformers achieve precision and recall values above 0.9.
Supplemental Material
- Michelle A. Amazeen and Bartosz W. Wojdynski. 2020. The Effects of Disclosure Format on Native Advertising Recognition and Audience Perceptions of Legacy and Online News Publishers. Journalism, Vol. 21, 12 (2020), 1965--1984.Google ScholarCross Ref
- Yana R. Avramova, Patrick De Pelsmacker, and Nathalie Dens. 2017. Brand Placement in Text: The Short-and Long-Term Effects of Placement Modality and Need for Cognition. International Journal of Advertising, Vol. 36, 5 (2017), 682--704.Google ScholarCross Ref
- Terrence M. Barnhardt, Isabel Manzano, Maria Brito, Melissa Myrick, and Steven M. Smith. 2016. The Effects of Product Placement in Fictitious Literature on Consumer Purchase Intention. Psychology & marketing, Vol. 33, 11 (2016), 883--898.Google Scholar
- Ahmet Bulut and Abdelrahman Mahmoud. 2023. Generating Campaign Ads & Keywords for Programmatic Advertising. IEEE Access , Vol. 11 (2023), 43557--43565. https://doi.org/10.1109/ACCESS.2023.3269505Google ScholarCross Ref
- Colin Campbell and Pamela E. Grimm. 2019. The Challenges Native Advertising Poses: Exploring Potential Federal Trade Commission Responses and Identifying Research Needs. Journal of Public Policy & Marketing, Vol. 38, 1 (2019), 110--123.Google ScholarCross Ref
- Siyu Duan, Wei Li, Jing Cai, Yancheng He, and Yunfang Wu. 2021. Query-Variant Advertisement Text Generation with Association Knowledge. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management (Virtual Event, Queensland, Australia) (CIKM '21). Association for Computing Machinery, New York, NY, USA, 412--421. https://doi.org/10.1145/3459637.3482290Google ScholarDigital Library
- Bassant Eyada and Asli Milla. 2020. Native Advertising: Challenges and Perspectives. Journal of Design Sciences and Applied Arts, Vol. 1, 1 (2020), 67--77.Google ScholarCross Ref
- J. Weston Hughes, Keng-hao Chang, and Ruofei Zhang. 2019. Generating Better Search Engine Text Advertisements with Deep Reinforcement Learning. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (Anchorage, AK, USA) (KDD '19). Association for Computing Machinery, New York, NY, USA, 2269--2277. https://doi.org/10.1145/3292500.3330754Google ScholarDigital Library
- Hamed Jafarzadeh, Aybüke Aurum, John D'Ambra, and Amir Ghapanchi. 2015. A Systematic Review on Search Engine Advertising. Pacific Asia Journal of the Association for Information Systems, Vol. 7, 3 (2015), 1--32.Google ScholarCross Ref
- Hidetaka Kamigaito, Peinan Zhang, Hiroya Takamura, and Manabu Okumura. 2021. An Empirical Study of Generating Texts for Search Engine Advertising. In Proceedings of NAACL-HLT 2021. Association for Computational Linguistics, Online, 255--262. https://aclanthology.org/2021.naacl-industry.32Google ScholarCross Ref
- Dirk Lewandowski. 2017. Users' Understanding of Search Engine Advertisements. Journal of Information Science Theory and Practice, Vol. 5, 4 (2017), 6--25.Google Scholar
- Dirk Lewandowski, Friederike Kerkmann, Sandra Rümmele, and Sebastian Sünkler. 2018. An Empirical Investigation on Search Engine Ad Disclosure. Journal of the Association for Information Science and Technology, Vol. 69, 3 (2018), 420--437.Google ScholarDigital Library
- Lars Erling Olsen and Even Johan Lanseng. 2012. Brands in texts: Attitudinal Effects of Brand Placements in Narrative Fiction. Journal of Brand Management, Vol. 19, 8 (2012), 702--711.Google ScholarCross Ref
- Elliott L. Post and Chandra N. Sekharan. 2015. Comparative Study and Evaluation of Online Ad-Blockers. In Proceedings of ICISS 2015. IEEE, 1--4.Google Scholar
- Enric Pujol, Oliver Hohlfeld, and Anja Feldmann. 2015. Annoyed users: Ads and ad-block usage in the wild. In Proceedings of the 2015 Internet Measurement Conference. 93--106.Google ScholarDigital Library
- Erin E. Schauster, Patrick Ferrucci, and Marlene S. Neill. 2016. Native Advertising is the New Journalism: How Deception Affects Social Responsibility. American Behavioral Scientist, Vol. 60, 12 (2016), 1408--1424.Google ScholarCross Ref
- Benjamin Shiller, Joel Waldfogel, and JohnnyRyan. 2018. The Effect of AdBlocking on Website Traffic and Quality. The RAND Journal of Economics, Vol. 49, 1 (2018), 43--63.Google ScholarCross Ref
- Sofia Eleni Spatharioti, David M. Rothschild, Daniel G. Goldstein, and Jake M. Hofman. 2023. Comparing Traditional and LLM-based Search for Consumer Choice: A Randomized Experiment. CoRR , Vol. abs/2307.03744 (2023), bibinfonumpages21 pages.Google Scholar
- Benjamin C. Storm and Eve Stoller. 2015. Exposure to Product Placement in Text can Influence Consumer Judgments. Applied Cognitive Psychology, Vol. 29, 1 (2015), 20--31.Google ScholarCross Ref
- Bartosz W. Wojdynski and Nathaniel J. Evans. 2016. Going Native: Effects of Disclosure Position and Language onthe Recognition and Evaluation of Online Native Advertising. Journal of Advertising, Vol. 45, 2 (2016), 157--168.Google ScholarCross Ref
- Ines Zelch, Matthias Hagen, and Martin Potthast. 2024. A User Study on the Acceptance of Native Advertising in Generative IR. In ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR 2024). ACM, bibinfonumpages11 pages. https://doi.org/10.1145/3627508.3638316Google ScholarDigital Library
Index Terms
- Detecting Generated Native Ads in Conversational Search
Recommendations
Ads Keyword Rewriting Using Search Engine Results
WWW '15 Companion: Proceedings of the 24th International Conference on World Wide WebPaid Search (PS) ads are one of the main revenue sources of online advertising companies where the goal is returning a set of relevant ads for a searched query in search engine websites such as Bing. Typical PS algorithms, return the ads which their ...
Improving ad relevance in sponsored search
WSDM '10: Proceedings of the third ACM international conference on Web search and data miningWe describe a machine learning approach for predicting sponsored search ad relevance. Our baseline model incorporates basic features of text overlap and we then extend the model to learn from past user clicks on advertisements. We present a novel ...
Fighting online click-fraud using bluff ads
Online advertising is currently the richest source of revenue for many Internet giants. The increased number of online businesses, specialized websites and modern profiling techniques have all contributed to an explosion of the income of ad brokers from ...
Comments