research-article

Chatbots Language Design: The Influence of Language Variation on User Experience with Tourist Assistant Chatbots

Authors:
Ana Paula Chaves

Northern Arizona University, and Federal University of Technology–Paraná, Campo Mourão, Brazil

Northern Arizona University, and Federal University of Technology–Paraná, Campo Mourão, Brazil

0000-0002-2307-3099
View Profile

,
Jesse Egbert

Department of English, Northern Arizona University, Flagstaff, Arizona

Department of English, Northern Arizona University, Flagstaff, Arizona
View Profile

,
Toby Hocking

School of Informatics, Computing, and Cyber Systems, Northern Arizona University, Flagstaff, Arizona

School of Informatics, Computing, and Cyber Systems, Northern Arizona University, Flagstaff, Arizona
View Profile

,
Eck Doerry

School of Informatics, Computing, and Cyber Systems, Northern Arizona University, Flagstaff, Arizona

School of Informatics, Computing, and Cyber Systems, Northern Arizona University, Flagstaff, Arizona
View Profile

,
Marco Aurelio Gerosa

School of Informatics, Computing, and Cyber Systems, Northern Arizona University, Flagstaff, Arizona

School of Informatics, Computing, and Cyber Systems, Northern Arizona University, Flagstaff, Arizona
View Profile

Authors Info & Claims

ACM Transactions on Computer-Human Interaction Volume 29 Issue 2Article No.: 13pp 1–38https://doi.org/10.1145/3487193

Published:16 January 2022Publication History

ACM Transactions on Computer-Human Interaction

Abstract

Chatbots are often designed to mimic social roles attributed to humans. However, little is known about the impact of using language that fails to conform to the associated social role. Our research draws on sociolinguistic to investigate how a chatbot’s language choices can adhere to the expected social role the agent performs within a context. We seek to understand whether chatbots design should account for linguistic register. This research analyzes how register differences play a role in shaping the user’s perception of the human-chatbot interaction. We produced parallel corpora of conversations in the tourism domain with similar content and varying register characteristics and evaluated users’ preferences of chatbot’s linguistic choices in terms of appropriateness, credibility, and user experience. Our results show that register characteristics are strong predictors of user’s preferences, which points to the needs of designing chatbots with register-appropriate language to improve acceptance and users’ perceptions of chatbot interactions.

Supplemental Material

Available for Download

zip

chaves.zip (1.3 MB)

Supplemental movie, appendix, image and software files for, Chatbots Language Design: The Influence of Language Variation on User Experience with Tourist Assistant Chatbots

REFERENCES

[1] Shawar B. Abu and Atwell E.. 2004. Evaluation of chatbot information system. In Proceedings of the 8th Maghrebian Conference on Software Engineering and Artificial Intelligence. Centre de Publication Universitaire, Tunis, 12.Google Scholar
[2] Alexis Papathanassis. 2017. R-Tourism: Introducing the potential impact of robotics and service automation in tourism. Ovidius University Annals, Series Economic Sciences 17, 1 (2017), 211–216.Google Scholar
[3] Anthony Laurence. 2005. AntConc: Design and development of a freeware corpus analysis toolkit for the technical writing classroom. In Proceedings of the International Professional Communication Conference, 2005.IEEE, 729–737.Google ScholarCross Ref
[4] Appel Markus, Izydorczyk David, Weber Silvana, Mara Martina, and Lischetzke Tanja. 2020. The uncanny of mind in a machine: Humanoid robots as tools, agents, and experiencers. Computers in Human Behavior 102 (2020), 274–286.Google ScholarDigital Library
[5] Araujo Theo. 2018. Living up to the chatbot hype: The influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Computers in Human Behavior 85 (2018), 183–189.Google ScholarCross Ref
[6] Argamon Shlomo. 2019. Register in computational language research. Register Studies 1, 1 (2019), 100–135.Google ScholarCross Ref
[7] Argamon Shlomo, Koppel Moshe, and Avneri Galit. 1998. Routing documents according to style. In Proceedings of the 1st International Workshop on Innovative Information Systems. Citeseer, 85–92.Google Scholar
[8] Bakhtin Mikhail Mikhaĭlovich. 2010. Speech Genres and Other Late Essays. University of Texas Press, Austin, TX.Google Scholar
[9] Balaji Divyaa. 2019. Assessing User Satisfaction with Information Chatbots: A Preliminary Investigation. Master’s thesis. University of Twente. Retrieved from https://essay.utwente.nl/79785/1/Balaji_MA_BMS.pdf.Google Scholar
[10] Bates Douglas, Mächler Martin, Bolker Ben, and Walker Steve. 2015. Fitting linear mixed-effects models using lme4. Journal of Statistical Software 67, 1 (2015), 1–48. DOI: https://doi.org/10.18637/jss.v067.i01Google ScholarCross Ref
[11] Biber Douglas. 1988. Variation across Speech and Writing. Cambridge University Press, Cambridge.Google ScholarCross Ref
[12] Biber Douglas. 1995. Dimensions of Register Variation: A Cross-Linguistic Comparison. Cambridge University Press, New York, NY.Google ScholarCross Ref
[13] Biber Douglas. 2012. Register as a predictor of linguistic variation. Corpus Linguistics and Linguistic Theory 8, 1 (2012), 9–37.Google ScholarCross Ref
[14] Biber Douglas. 2017. MAT–Multidimensional Analysis Tagger. Retrieved 27 August, 2021 from https://goo.gl/u7h9gb.Google Scholar
[15] Biber Douglas. 2019. Text-linguistic approaches to register variation. Register Studies 1, 1 (2019), 42–75.Google ScholarCross Ref
[16] Biber Douglas and Conrad Susan. 2019. Register, Genre, and Style (2nd ed.). Cambridge University Press, New York, NY.Google ScholarCross Ref
[17] Biber Douglas, Conrad Susan, and Cortes Viviana. 2004. If you look at...: Lexical bundles in university teaching and textbooks. Applied Linguistics 25, 3 (2004), 371–405.Google ScholarCross Ref
[18] Biber Douglas and Egbert Jesse. 2016. Using multi-dimensional analysis to study register variation on the searchable web. Corpus Linguistics Research 2, 21 (2016), 1–23.Google ScholarCross Ref
[19] Biber D. and Egbert J.. 2018. Register Variation Online. Cambridge University Academic Press, Cambridge.Google ScholarCross Ref
[20] Biber Douglas and Gray Bethany. 2010. Challenging stereotypes about academic writing: Complexity, elaboration, explicitness. Journal of English for Academic Purposes 9, 1 (2010), 2–20.Google ScholarCross Ref
[21] Biber Douglas, Gray Bethany, and Poonpon Kornwipa. 2011. Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? Tesol Quarterly 45, 1 (2011), 5–35.Google ScholarCross Ref
[22] Biber Douglas, Johansson Stig, Leech Geoffrey, Conrad Susan, Finegan Edward, and Quirk Randolph. 1999. Longman Grammar of Spoken and Written English. Vol. 2, Pearson Longman, London.Google Scholar
[23] Böcker Nina. 2019. Usability of Information-Retrieval Chatbots and the Effects of Avatars on Trust. B.S. thesis. University of Twente.Google Scholar
[24] Bosher Susan and Bowles Melissa. 2008. The effects of linguistic modification on ESL students’ comprehension of nursing course test items-a collaborative process is used to modify multiple-choice questions for comprehensibility without damaging the integrity of the item. Nursing Education Perspectives 29, 4 (2008), 174.Google Scholar
[25] Brandtzaeg Petter Bae and Følstad Asbjørn. 2018. Chatbots: Changing user needs and motivations. Interactions 25, 5 (2018), 38–43. Google ScholarDigital Library
[26] Buhalis Dimitrios and Jun Soo Hyun. 2011. E-tourism. In Contemporary Tourism Reviews. Cooper Chris (Ed.), Goodfellow Publishers Limited, Woodeaton, Oxford, 1–38.Google Scholar
[27] Cerezo Jhonny, Kubelka Juraj, Robbes Romain, and Bergel Alexandre. 2019. Building an expert recommender chatbot. In Proceedings of the 1st International Workshop on Bots in Software Engineering. IEEE, New York, NY, 59–63. Google ScholarDigital Library
[28] Chaves Ana Paula. 2020. GitHub Repository. Retrieved 27 August, 2021 from https://github.com/chavesana/chatbots-register.Google Scholar
[29] Chaves Ana Paula, Doerry Eck, Egbert Jesse, and Gerosa Marco. 2019. It’s how you say it: Identifying appropriate register for chatbot language design. In Proceedings of the 7th International Conference on Human-Agent Interaction. ACM, New York, NY, 1–8. DOI: https://doi.org/10.1145/3349537.3351901 Google ScholarDigital Library
[30] Chaves Ana Paula, Egbert Jesse, and Gerosa Marco Aurelio. 2019. Chatting like a robot: The relationship between linguistic choices and users’ experiences. In Proceedings of the ACM CHI 2019 Workshop on Conversational Agents: Acting on the Wave of Research and Development. Retrieved from https://convagents.org/.Google Scholar
[31] Chaves Ana Paula and Gerosa Marco Aurelio. 2018. Single or multiple conversational agents? An interactional coherence comparison. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 191:1–191:13. Google ScholarDigital Library
[32] Chaves Ana Paula and Gerosa Marco Aurelio. 2020. How should my chatbot interact? A survey on social characteristics in human–chatbot interaction design. International Journal of Human–Computer Interaction 0, 0 (2020), 1–30. DOI: https://doi.org/10.1080/10447318.2020.1841438Google Scholar
[33] Chen Tianqi and Guestrin Carlos. 2016. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . ACM, New York, NY, 785–794. DOI: https://doi.org/10.1145/2939672.2939785 Google ScholarDigital Library
[34] Chen Zhifa, Lu Yichen, Nieminen Mika P., and Lucero Andrés. 2020. Creating a chatbot for and with migrants: Chatbot personality drives co-design activities. In Proceedings of the 2020 ACM Designing Interactive Systems Conference. ACM, New York, NY, 219–230. Google ScholarDigital Library
[35] Christensen R. H. B.. 2019. ordinal—Regression models for ordinal data. R package version 2019.12-10. Retrieved 27 August, 2021 from https://CRAN.R-project.org/package=ordinal.Google Scholar
[36] Ciechanowski Leon, Przegalinska Aleksandra, Magnuski Mikolaj, and Gloor Peter. 2018. In the shades of the uncanny valley: An experimental study of human–chatbot interaction. Future Generation Computer Systems 92 (2018), 539–548.Google ScholarCross Ref
[37] Conrad Susan and Biber Douglas. 2014. Multi-Dimensional Studies of Register Variation in English. Routledge, New York, NY.Google Scholar
[38] Corritore Cynthia L., Marble Robert P., Wiedenbeck Susan, Kracher Beverly, and Chandran Ashwin. 2005. Measuring online trust of websites: Credibility, perceived ease of use, and risk. In Proceedings of the Americas Conference on Information Systems. Association for Information Systems, Atlanta, GA, 370.Google Scholar
[39] Cowan Benjamin R., Doyle Philip, Edwards Justin, Garaialde Diego, Hayes-Brady Ali, Branigan Holly P., Cabral João, and Clark Leigh. 2019. What’s in an accent? The impact of accented synthetic speech on lexical choice in human-machine dialogue. In Proceedings of the 1st International Conference on Conversational User Interfaces. ACM, New York, NY, 1–8. Google ScholarDigital Library
[40] Do Hyo Jin, Yang Seon Hye, Choi Boo-Gyoung, Fu Wayne T., and Bailey Brian P.. 2021. Do you have time for a quick chat? Designing a conversational interface for sexual harassment prevention training. In Proceedings of the 26th International Conference on Intelligent User Interfaces. ACM, New York, NY, 542–552. Google ScholarDigital Library
[41] Doan AnHai, Halevy Alon, and Ives Zachary. 2012. String matching. In Principles of Data Integration. Doan AnHai, Halevy Alon, and Ives Zachary (Eds.), Morgan Kaufmann, Boston, Chapter 4, 95–119. DOI: https://doi.org/10.1016/B978-0-12-416044-6.00004-1 Google ScholarDigital Library
[42] Duijst Daniëlle. 2017. Can We Improve the User Experience of Chatbots with Personalisation. Master’s thesis. University of Amsterdam.Google Scholar
[43] Duijvelshoff Willem. 2017. Use-cases and ethics of chatbots on Plek: A social intranet for organizations. In Proceedings of the Workshop on Chatbots and Artificial Intelligence.Google Scholar
[44] Dyke Gregory, Howley Iris, Adamson David, Kumar Rohit, and Rosé Carolyn Penstein. 2013. Towards academically productive talk supported by conversational agents. In Intelligent Tutoring Systems. Cerri Stefano A., Clancey William J., Papadourakis Giorgos, and Panourgia Kitty (Eds.), Springer, Berlin, Heidelberg, 531–540. Google ScholarDigital Library
[45] Edwards Justin and Sanoubari Elaheh. 2019. A need for trust in conversational interface research. In Proceedings of the 1st International Conference on Conversational User Interfaces. ACM, New York, NY, 1–3. Google ScholarDigital Library
[46] Elsholz Ela, Chamberlain Jon, and Kruschwitz Udo. 2019. Exploring language style in chatbots to increase perceived product value and user engagement. In Proceedings of the 2019 Conference on Human Information Interaction and Retrieval. ACM, New York, NY, 301–305. Google ScholarDigital Library
[47] Facebook. 2018. Tech for Tourism-Rresearch Page. Retrieved 27 August, 2021 from https://www.facebook.com/VisitFlagstaff/.Google Scholar
[48] Feine Jasper, Gnewuch Ulrich, Morana Stefan, and Maedche Alexander. 2019. A taxonomy of social cues for conversational agents. International Journal of Human-Computer Studies 132 (2019), 138–161.Google ScholarDigital Library
[49] Finstad Kraig. 2010. The usability metric for user experience. Interacting with Computers 22, 5 (2010), 323–327. Google ScholarDigital Library
[50] Fitzpatrick Kathleen Kara, Darcy Alison, and Vierhile Molly. 2017. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): A randomized controlled trial. JMIR Mental Health 4, 2 (2017), e7785.Google ScholarCross Ref
[51] Flagstaff. 2019. The City of Flagstaff–Arizona. Retrieved 27 August, 2021 from http://www.flagstaff.az.gov/2/Community-Profile.Google Scholar
[52] Fogg B. J.. 2003. Computers as persuasive social actors. In Persuasive Technology. Fogg B. J. (Ed.), Morgan Kaufmann, San Francisco, Chapter 5, 89–120.Google ScholarCross Ref
[53] Følstad Asbjørn and Brandtzæg Petter Bae. 2017. Chatbots and the new world of HCI. Interactions 24, 4 (2017), 38–42. Google ScholarDigital Library
[54] Følstad Asbjørn and Skjuve Marita. 2019. Chatbots for customer service: User experience and motivation. In Proceedings of the 1st International Conference on Conversational User Interfaces. ACM, New York, NY, 1–9. Google ScholarDigital Library
[55] Forlizzi Jodi, Zimmerman John, Mancuso Vince, and Kwak Sonya. 2007. How interface agents affect interaction between humans and computers. In Proceedings of the 2007 Conference on Designing Pleasurable Products and Interfaces. ACM, New York, NY, 209–221. Google ScholarDigital Library
[56] Friedman Jerome, Hastie Trevor, and Tibshirani Rob. 2010. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software 33, 1 (2010), 1–22. DOI: https://doi.org/10.18637/jss.v033.i01Google ScholarCross Ref
[57] Galitsky Boris, Ilvovsky Dmitry, and Goncharova Elizaveta. 2019. On a chatbot providing virtual dialogues. In Proceedings of the International Conference on Recent Advances in Natural Language Processing. INCOMA Ltd., 382–387.Google ScholarCross Ref
[58] Gavalas Damianos and Kenteris Michael. 2011. A web-based pervasive recommendation system for mobile tourist guides. Personal and Ubiquitous Computing 15, 7 (2011), 759–770. Google ScholarDigital Library
[59] Gnewuch Ulrich, Morana Stefan, and Maedche Alexander. 2017. Towards designing cooperative and social conversational agents for customer service. In Proceedings of the International Conference on Information Systems 2017. Association for Information Systems, 13.Google Scholar
[60] Go Eun and Sundar S. Shyam. 2019. Humanizing Chatbots: The effects of visual, identity and conversational cues on humanness perceptions. Computers in Human Behavior 97 (Aug. 2019), 304–316.Google ScholarDigital Library
[61] Developers Google. 2021. Conversation Design by Google. Retrieved March 3, 2021 from https://developers.google.com/assistant/conversation-design/what-is-conversation-design.Google Scholar
[62] Research Grand View. 2017. Chatbot market size to reach $1.25 Billion by 2025. CAGR: 24.3%. Retrieved 27 August, 2021 from https://www.grandviewresearch.com/press-release/global-chatbot-market.Google Scholar
[63] Grudin Jonathan and Jacques Richard. 2019. Chatbots, humbots, and the quest for artificial general intelligence. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 1–11. Google ScholarDigital Library
[64] Gu Xiaodong, Cho Kyunghyun, Ha Jung-Woo, and Kim Sunghun. 2018. Dialogwae: Multimodal response generation with conditional wasserstein auto-encoder. In International Conference on Learning Representations. OpenReview. https://openreview.net/forum?id=BkgBvsC9FQ.Google Scholar
[65] Halliday Michael A. K. and Hasan Ruqaiya. 1992. Cohesion in English. Penguin, London.Google Scholar
[66] Hayashi Yugo. 2016. The effect of “mood”: Group-based collaborative problem solving by taking different perspectives. In Proceedings of the 38th Annual Conference of the Cognitive Science Society. The Cognitive Science Society, 818–823.Google Scholar
[67] Heyselaar E. S. and Bosse T.. 2019. Using theory of mind to assess users’ sense of agency in social chatbots. In Proceedings of the Conversations 2019: 3rd International Workshop on Chatbot Research. Springer, Cham, 1–13.Google Scholar
[68] Hill Jennifer, Ford W. Randolph, and Farreras Ingrid G.. 2015. Real conversations with artificial intelligence: A comparison between human–human online conversations and human–chatbot conversations. Computers in Human Behavior 49, C (2015), 245–250. Google ScholarDigital Library
[69] Hoegen Rens, Aneja Deepali, McDuff Daniel, and Czerwinski Mary. 2019. An end-to-end conversational style matching agent. In Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents . ACM, New York, NY, 111–118. DOI: https://doi.org/10.1145/3308532.3329473 Google ScholarDigital Library
[70] Hothorn Torsten, Buehlmann Peter, Dudoit Sandrine, Molinaro Annette, and Laan Mark Van Der. 2006. Survival ensembles. Biostatistics 7, 3 (2006), 355–373.Google ScholarCross Ref
[71] Hunston Susan. 2010. Corpus Approaches to Evaluation: Phraseology and Evaluative Language. Routledge, New York, NY.Google ScholarCross Ref
[72] Hwang Shinhee, Kim Beomjun, and Lee Keeheon. 2019. A data-driven design framework for customer service chatbot. In Proceedings of the International Conference on Human–Computer Interaction. Springer, Cham, 222–236.Google ScholarCross Ref
[73] Hyland Ken. 2012. Bundles in academic discourse. Annual Review of Applied Linguistics 32, 1 (2012), 150–169.Google ScholarCross Ref
[74] 9241-11 ISO. 2018. Ergonomics of Human-System Interaction: Part 11: Usability: Definitions and Concepts. International Organization for Standardization, ISO.Google Scholar
[75] Ivanov S. and Webster C.. 2017. Adoption of robots, artificial intelligence and service automation by travel, tourism and hospitality companies–a cost-benefit analysis. In Proceedings of the International Scientific Conference “Contemporary Tourism–Traditions and Innovations. Sofia University, 19–21.Google Scholar
[76] Jabri Muayyad, Adrian Allyson D., and Boje David. 2008. Reconsidering the role of conversations in change communication: A contribution based on Bakhtin. Journal of Organizational Change Management 21, 6 (2008), 667–685.Google ScholarCross Ref
[77] Jain Mohit, Kumar Pratyush, Kota Ramachandra, and Patel Shwetak N.. 2018. Evaluating and informing the design of chatbots. In Proceedings of the 2018 Designing Interactive Systems Conference. ACM, New York, NY, 895–906. Google ScholarDigital Library
[78] Jakic Ana, Wagner Maximilian Oskar, and Meyer Anton. 2017. The impact of language style accommodation during social media interactions on brand trust. Journal of Service Management 28, 3 (2017), 418–441.Google ScholarCross Ref
[79] Jenkins Marie-Claire, Churchill Richard, Cox Stephen, and Smith Dan. 2007. Analysis of user interaction with service oriented chatbot systems. In Human–Computer Interaction. HCI Intelligent Multimodal Interaction Environments. Jacko Julie A. (Ed.), Springer, Berlin, 76–83. Google ScholarDigital Library
[80] Jiang Ridong and Banchs Rafael E.. 2017. Towards improving the performance of chat oriented dialogue system. In Proceedings of the 2017 International Conference on Asian Language Processing. IEEE, New York, NY, 23–26.Google ScholarCross Ref
[81] Joos Martin. 1967. The Five Clocks. Vol. 58, Harcourt, Brace & World, New York, NY.Google Scholar
[82] Kamberelis George. 1995. Genre as institutionally informed social practice. Journal of Contemporary Legal Issues 6 (1995), 115.Google Scholar
[83] Keijsers Merel and Bartneck Christoph. 2018. Mindless Robots get bullied. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. ACM, New York, NY, 205–214. Google ScholarDigital Library
[84] Kilgarriff Adam. 2005. Language is never, ever, ever, random. Corpus Linguistics and Linguistic Theory 1, 2 (2005), 263–276.Google ScholarCross Ref
[85] Kirakowski Jurek, Yiu Anthony, and P. O’Donnell. 2009. Establishing the hallmarks of a convincing chatbot-human dialogue. In Human-Computer Interaction, Inaki Maurtua. InTech, London.Google Scholar
[86] Kiseleva Julia, Williams Kyle, Awadallah Ahmed Hassan, Crook Aidan C., Zitouni Imed, and Anastasakos Tasos. 2016. Predicting user satisfaction with intelligent assistants. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, NY, 45–54. Google ScholarDigital Library
[87] Komatsu Takanori, Kurosawa Rie, and Yamada Seiji. 2012. How does the difference between users’ expectations and perceptions about a robotic agent affect their behavior? International Journal of Social Robotics 4, 2 (2012), 109–116.Google ScholarCross Ref
[88] Komiak Sherrie Y. X. and Benbasat Izak. 2006. The effects of personalization and familiarity on trust and adoption of recommendation agents. MIS Quarterly 30, 4 (2006), 941–960. Google ScholarDigital Library
[89] Krauss Robert M. and Chiu Chi-Yue. 1998. Language and social behavior. In The Handbook of Social Psychology. Gilbert D. T., Fiske S. T., and Lindzey G. (Eds.), McGraw-Hill, New York, NY, 41–88.Google Scholar
[90] Labov William, Ash Sharon, and Boberg Charles. 2005. The Atlas of North American English: Phonetics, Phonology and Sound Change. Walter de Gruyter, Boston, MA.Google ScholarCross Ref
[91] Lang Tania C.. 2000. The effect of the Internet on travel consumer purchasing behaviour and implications for travel agencies. Journal of Vacation Marketing 6, 4 (2000), 368–385.Google ScholarCross Ref
[92] Lasek Mirosława and Jessa Szymon. 2013. Chatbots for customer service on Hotels’ websites. Information Systems in Management 2, 2 (2013), 146–158.Google Scholar
[93] Lee Minha, Lucas Gale, Mell Johnathan, Johnson Emmanuel, and Gratch Jonathan. 2019. What’s on your virtual mind?: Mind perception in human-agent negotiations. In Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents . ACM, New York, NY, 38–45. DOI: https://doi.org/10.1145/3308532.3329465 Google ScholarDigital Library
[94] Lee Min Kyung, Kiesler Sara, and Forlizzi Jodi. 2010. Receptionist or information kiosk: How do people talk with a robot? In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work. ACM, New York, NY, 31–40. Google ScholarDigital Library
[95] Lee SeoYoung and Choi Junho. 2017. Enhancing user experience with conversational agent for movie recommendation: Effects of self-disclosure and reciprocity. International Journal of Human-Computer Studies 103, C (2017), 95–105. Google ScholarDigital Library
[96] Leech Geoffrey N. and Short Mick. 2007. Style in Fiction: A Linguistic Introduction to English Fictional Prose.. Pearson Education, London.Google Scholar
[97] Li Yanran, Su Hui, Shen Xiaoyu, Li Wenjie, Cao Ziqiang, and Niu Shuzi. 2017. DailyDialog: A manually labelled multi-turn dialogue dataset. In Proceedings of the International Joint Conference on Natural Language Processing. Asian Federation of Natural Language Processing, 986–995.Google Scholar
[98] Liao Vera Q., Davis Matthew, Geyer Werner, Muller Michael, and Shami N. Sadat. 2016. What can you do?: Studying social-agent orientation and agent proactive interactions with an agent for employees. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems . ACM, New York, NY, 264–275. Google ScholarDigital Library
[99] Lin Grace I. and Walker Marilyn A.. 2017. Stylistic variation in television dialogue for natural language generation. In Proceedings of the Workshop on Stylistic Variation. Association for Computational Linguistics, 85–93.Google ScholarCross Ref
[100] Linden Greg, Hanks Steve, and Lesh Neal. 1997. Interactive assessment of user preference models: The automated travel assistant. In User Modeling. A. Jameson, C. Paris, and C. Tasso (Eds.), Springer, Vienna, 67–78.Google ScholarCross Ref
[101] Long Michael H. and Ross Steven. 1993. Modifications That Preserve Language and Content.Technical Report. ERIC.Google Scholar
[102] Loo Jaclyn. 2017. The future of travel: New consumer behavior and the technology giving it flight. Google/Phocuswright Travel Study 2017. Retrieved on 27 August 2021 from https://www.thinkwithgoogle.com/consumer-insights/consumer-trends/new-consumer-travel-assistance/?utm_source=HospitalityTrends.Google Scholar
[103] Luger Ewa and Sellen Abigail. 2016. Like having a really bad PA: The gulf between user expectation and experience of conversational agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 5286–5297. Google ScholarDigital Library
[104] Mack Rhonda W., Blose Julia E., and Pan Bing. 2008. Believe it or not: Credibility of blogs in tourism. Journal of Vacation Marketing 14, 2 (2008), 133–144.Google ScholarCross Ref
[105] Mairesse François and Walker Marilyn A.. 2009. Can Conversational Agents Express Big Five Personality Traits through Language?:Evaluating a Psychologically-Informed Language Generator. Cambridge & Sheffield, Cambridge University Engineering Department & Department of Computer Science, University of Sheffield.Google Scholar
[106] Malinowski Bronislaw. 1923. The Problem of Meaning in Primitive Languages. Harcourt, Brace & World, Inc, New York, Chapter Supplement to C.K., 296–336.Google Scholar
[107] Maslowski Irina, Lagarde Delphine, and Clavel Chloé. 2017. In-the-wild chatbot corpus: From opinion analysis to interaction problem detection. In Proceedings of the International Conference on Natural Language, Signal and Speech Processing. International Science and General Applications, 115–120.Google Scholar
[108] Massaro Dominic W., Cohen Michael M., Daniel Sharon, and Cole Ronald A.. 1999. Developing and evaluating conversational agents. In Human Performance and Ergonomics (2nd ed.). Hancock P. A. (Ed.), Academic Press, Cambridge, Chapter 7, 173–194.Google Scholar
[109] McNamara Niamh and Kirakowski Jurek. 2006. Functionality, usability, and user experience. Interactions 13, 6 (2006), 26–28. Google ScholarDigital Library
[110] Montenegro Joao Luis Zeni, Costa Cristiano André da, and Righi Rodrigo da Rosa. 2019. Survey of conversational agents in health. Expert Systems with Applications 129 (2019), 56–67.Google ScholarDigital Library
[111] Mori Junko. 1992. About What and What about: The Distinction between Topic and Issue in Conversations. Ph.D. Dissertation. University of Delaware.Google Scholar
[112] Morris Thomas William. 2002. Conversational agents for game-like virtual environments. In Artificial Intelligence and Interactive Entertainment, Ken Forbus and Magy Seif El-Nasr. AAAI Press, Palo Alto, CA, 82–86.Google Scholar
[113] Morrissey Kellie and Kirakowski Jurek. 2013. ‘Realness’ in Chatbots: Establishing quantifiable criteria. In Proceedings of the International Conference on Human-Computer Interaction. Springer, Berlin, Heidelberg, 87–96. Google ScholarDigital Library
[114] Nass Clifford, Steuer Jonathan, and Tauber Ellen R.. 1994. Computers are social actors. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 72–78. Google ScholarDigital Library
[115] Neururer Mario, Schlögl Stephan, Brinkschulte Luisa, and Groth Aleksander. 2018. Perceptions on authenticity in chat bots. Multimodal Technologies and Interaction 2, 3 (2018), 60.Google ScholarCross Ref
[116] Nevill Craig and Bell Timothy. 1992. Compression of parallel texts. Information Processing & Management 28, 6 (1992), 781–793. Google ScholarDigital Library
[117] Niederhoffer Kate G. and Pennebaker James W.. 2002. Linguistic style matching in social interaction. Journal of Language and Social Psychology 21, 4 (2002), 337–360.Google ScholarCross Ref
[118] Palan Stefan and Schitter Christian. 2018. Prolific. ac—A subject pool for online experiments. Journal of Behavioral and Experimental Finance 17, C (2018), 22–27.Google ScholarCross Ref
[119] Paltridge Brian. 1994. Genre analysis and the identification of textual boundaries. Applied Linguistics 15, 3 (1994), 288–299.Google ScholarCross Ref
[120] Pawłowska Aneta. 2016. Tourists and social media: Already inseparable marriage or still a long-distance relationship? Analysis of focus group study results conducted among tourists using social media. World Scientific News 57 (2016), 106–115.Google Scholar
[121] Peras Dijana. 2018. Chatbot evaluation metrics: Review paper. In Proceedings of the 36th International Scientific Conference on Economic and Social Development. Varazdin Development and Entrepreneurship Agency, Varazdin, 89.Google Scholar
[122] Pinhanez Claudio S.. 2020. HCI research challenges for the next generation of conversational systems. In Proceedings of the 2nd Conference on Conversational User Interfaces. ACM, New York, NY, 1–4. Google ScholarDigital Library
[123] Radlinski Filip and Craswell Nick. 2017. A theoretical framework for conversational search. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval. Google ScholarDigital Library
[124] Radziwill Nicole M. and Benton Morgan C.. 2017. Evaluating quality of chatbots and intelligent conversational agents. Software Quality Professional 19, 3 (2017), 25–36.Google Scholar
[125] Reid Thomas B. W.. 1956. Linguistics, structuralism and philology. Archivum Linguisticum 8, 1 (1956), 28–37.Google Scholar
[126] Reiter Ehud and Dale Robert. 2000. Building Natural Language Generation Systems. Cambridge University Press, New York, NY. Google ScholarCross Ref
[127] Resendez Valeria. 2020. A Very Formal Agent: How Culture, Mode of Dressing and Linguistic Style Influence the Perceptions Toward an Embodied Conversational Agent?Master’s thesis. University of Twente.Google Scholar
[128] Ricci Francesco, Rokach Lior, and Shapira Bracha. 2011. Introduction to recommender systems handbook. In Recommender Systems Handbook. F. Ricci, L. Rokach, B. Shapira, and P. Kantor (Eds.), Springer, Boston, MA, 1–35.Google ScholarCross Ref
[129] Robb David A., Lopes José, Padilla Stefano, Laskov Atanas, Garcia Francisco J. Chiyah, Liu Xingkun, Willners Jonatan Scharff, Valeyrie Nicolas, Lohan Katrin, Lane David, Pedro Patron, Yvan Petillot, Mike J. Chantler, and Helen Hastie. 2019. Exploring interaction with remote autonomous systems using conversational agents. In Proceedings of the 2019 on Designing Interactive Systems Conference. ACM, New York, NY, 1543–1556. Google ScholarDigital Library
[130] Sato E.. 2007. A guide to linguistic modification: Strategies for increasing English language learner access to academic content. LEP Partnership. Retrieved on 27 August 2021 from https://ncela.ed.gov/files/uploads/11/abedi_sato.pdf.Google Scholar
[131] Shen Xiaoyu, Su Hui, Niu Shuzi, and Demberg Vera. 2018. Improving variational encoder-decoders in dialogue generation. In 32nd AAAI Conference on Artificial Intelligence. AAAI Press, Palo Alto, California, 5456–5463. Google ScholarDigital Library
[132] Shum Heung-yeung, He Xiao-dong, and Li Di. 2018. From Eliza to XiaoIce: Challenges and opportunities with social chatbots. Frontiers of Information Technology & Electronic Engineering 19, 1 (2018), 10–26.Google ScholarCross Ref
[133] Stoyanchev Svetlana, Liu Alex, and Hirschberg Julia. 2014. Towards natural clarification questions in dialogue systems. In Proceedings of the AISB Symposium on Questions, Discourse and Dialogue. Vol. 20, Society for the Study of Artificial Intelligence and Simulation of Behaviour, Brighton, 8.Google Scholar
[134] Strobl Carolin, Boulesteix Anne-Laure, Kneib Thomas, Augustin Thomas, and Zeileis Achim. 2008. Conditional variable importance for random forests. BMC Bioinformatics 9, 307 (2008), 11. Retrieved from http://www.biomedcentral.com/1471-2105/9/307.Google Scholar
[135] Strobl Carolin, Boulesteix Anne-Laure, Zeileis Achim, and Hothorn Torsten. 2007. Bias in random forest variable importance measures: Illustrations, sources and a solution. BMC Bioinformatics 8, 25 (2007), 21. Retrieved from http://www.biomedcentral.com/1471-2105/8/25.Google Scholar
[136] Svikhnushina Ekaterina, Placinta Alexandru, and Pu Pearl. 2021. User expectations of conversational chatbots based on online reviews. In Proceedings of the Designing Interactive Systems Conference 2021. ACM, New York, NY, 1481–1491. Google ScholarDigital Library
[137] Svikhnushina Ekaterina and Pu Pearl. 2021. Key qualities of conversational chatbots–the PEACE model. In Proceedings of the 26th International Conference on Intelligent User Interfaces. ACM, New York, NY, 520–530. Google ScholarDigital Library
[138] Sweeney Jill and Swait Joffre. 2008. The effects of brand credibility on customer loyalty. Journal of Retailing and Consumer Services 15, 3 (2008), 179–193.Google ScholarCross Ref
[139] Syed Bakhtiyar Hussain. 2020. Adapting Language Models for Style Transfer. Master’s thesis. International Institute of Information Technology Hyderabad.Google Scholar
[140] Szmrecsanyi Benedikt. 2011. Corpus-based dialectometry: A methodological sketch. Corpora 6, 1 (2011), 45–76.Google ScholarCross Ref
[141] Tallyn Ella, Fried Hector, Gianni Rory, Isard Amy, and Speed Chris. 2018. The ethnobot: Gathering ethnographies in the age of IoT. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 604. Google ScholarDigital Library
[142] Tariverdiyeva Gunay. 2019. Chatbots’ Perceived Usability in Information Retrieval Tasks: An Exploratory Analysis.Master’s thesis. University of Twente.Google Scholar
[143] Tegos Stergios, Demetriadis Stavros, and Tsiatsos Thrasyvoulos. 2016. An Investigation of conversational agent interventions supporting historical reasoning in primary education. In Proceedings of the International Conference on Intelligent Tutoring Systems. Micarelli Alessandro, Stamper John, and Panourgia Kitty (Eds.), Springer International Publishing, Cham, 260–266. Google ScholarDigital Library
[144] Thies Indrani Medhi, Menon Nandita, Magapu Sneha, Subramony Manisha, and O’neill Jacki. 2017. How do you want your chatbot? An exploratory Wizard-of-Oz study with young, urban Indians. In Proceedings of the Human–Computer Interaction–INTERACT . R. Bernhaupt, G. Dalvi, A. Joshi, D. K. Balkrishan, J. O’Neill, and M. Winckler (Eds.). Lecture Notes in Computer Science, Vol. 10513. Springer, Cham, 441–459.Google Scholar
[145] Google Think with. 2016. How Micro-Moments are Reshaping the Travel Customer Journey. Retrieved 27 August, 2021 from https://www.thinkwithgoogle.com/marketing-resources/micro-moments/micro-moments-travel-customer-journey/.Google Scholar
[146] Thomas Paul, Czerwinski Mary, McDuff Daniel, Craswell Nick, and Mark Gloria. 2018. Style and alignment in information-seeking conversation. In Proceedings of the 2018 Conference on Human Information Interaction&Retrieval. ACM, New York, NY, 42–51. Google ScholarDigital Library
[147] Bradford Rebecca Ruiz Thomas Combrink, Melinda. 2018. 2017-2018 Flagstaff Visitor Survey. Technical Report. Alliance Bank Eonomic Policy Institute, The W. A. Franke College of Business, Northern Arizona University. Prepared for the Flagstaff Convention and Visitors Bureau, Arizona Office of Tourism.Google Scholar
[148] Tikhonov Alexey and Yamshchikov Ivan P.. 2018. What is wrong with style transfer for texts? CoRRGoogle Scholar
[149] Trudgill Peter. 2000. Sociolinguistics: An Introduction to Language and Society. Penguin Books Limited, London.Google Scholar
[150] Tullis Thomas and Albert William. 2013. Measuring the User Experience, Second Edition: Collecting, Analyzing, and Presenting Usability Metrics (2nd ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA. Google ScholarDigital Library
[151] Tussyadiah Iis. 2020. A review of research into automation in tourism: Launching the Annals of Tourism Research Curated Collection on Artificial Intelligence and Robotics in Tourism. Annals of Tourism Research 81, C (2020), 102883.Google ScholarCross Ref
[152] UNWTO. 2017. UNWTO Tourism Highlights, 2017 Edition. World Tourism Organization, Madrid.Google Scholar
[153] Walgama M. S. and Hettige B.. 2017. Chatbots: The next generation in computer interfacing–A review. In Proceedings of the KDU International Research Conference. General Sir John Kotelawala Defence University, 7.Google Scholar
[154] Wallis Peter and Edmonds Bruce. 2019. How language works & what machines can do about it. In Proceedings of the 1st International Conference on Conversational User Interfaces. ACM, New York, NY, 1–3. Google ScholarDigital Library
[155] Wang Dan, Xiang Zheng, and Fesenmaier Daniel R.. 2016. Smartphone use in everyday life and travel. Journal of Travel Research 55, 1 (2016), 52–63.Google ScholarCross Ref
[156] Wanner Leo, André Elisabeth, Blat Josep, Dasiopoulou Stamatia, Farrús Mireia, Fraga Thiago, Kamateri Eleni, Lingenfelser Florian, Llorach Gerard, Martínez Oriol, and G. Meditskos. 2017. Design of a knowledge-based agent as a social companion. Procedia Computer Science 121, C (2017), 920–926. Google ScholarDigital Library
[157] Weisz Justin D., Jain Mohit, Joshi Narendra Nath, Johnson James, and Lange Ingrid. 2019. BigBlueBot: Teaching strategies for successful human-agent interactions. In Proceedings of the 24th International Conference on Intelligent User Interfaces. ACM, New York, NY, 448–459. Google ScholarDigital Library
[158] Wikibooks. 2020. Algorithm Implementation/Strings/Levenshtein distance—Wikibooks, The Free Textbook Project. Retrieved May 8, 2020 from https://en.wikibooks.org/w/index.php?title=Algorithm_Implementation/Strings/Levenshtein_distance&oldid=3678144.Google Scholar
[159] Zdravkova Katerina. 2000. Conceptual framework for an intelligent chatterbot. In Proceedings of the 22nd International Conference on Information Technology Interfaces. IEEE, New York, NY, 189–194.Google Scholar
[160] Zhao Tiancheng, Lee Kyusong, and Eskenazi Maxine. 2018. Unsupervised discrete sentence representation learning for interpretable neural dialog generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Vol. 1: Long Papers. Association for Computational Linguistics, 1098–1107. DOI: https://doi.org/10.18653/v1/P18-1101Google ScholarCross Ref
[161] Zhu Pengfei, Zhang Zhuosheng, Li Jiangtong, Huang Yafang, and Zhao Hai. 2018. Lingke: A fine-grained multi-turn chatbot for customer service. In Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations. Association for Computational Linguistics, 108–112.Google Scholar
[162] Zumstein Darius and Hundertmark Sophie. 2017. Chatbots-An interactive technology for personalized communication, transactions and services. IADIS International Journal on WWW/Internet 15, 1 (2017), 96–109.Google Scholar

Index Terms

Chatbots Language Design: The Influence of Language Variation on User Experience with Tourist Assistant Chatbots
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. Interaction paradigms
      1. Natural language interfaces

Recommendations

Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data
CSCW

Large language models (LLMs) provide a new way to build chatbots by accepting natural language prompts. Yet, it is unclear how to design prompts to power chatbots to carry on naturalistic conversations while pursuing a given goal such as collecting self-...
Read More
User Engagement with Chatbots: A Discursive Psychology Approach
CUI '20: Proceedings of the 2nd Conference on Conversational User Interfaces

Conversational agents have transcended into multiple industries with increased ability for user engagement in intelligent conversation. Conversations with chatbots are different from interpersonal communication in terms of turn-taking, intentions, and ...
Read More
Exploring how politeness impacts the user experience of chatbots for mental health support
Abstract
Politeness is important in human–human interaction when asking people to engage in sensitive conversations. If politeness manifests similarly in human–chatbot interaction, it may play an important role in the design of sensitive chatbot ...
Highlights
- Politeness can both positively and negatively impact the chatbot user experience.
- The Personal politeness chatbot was experienced as caring and encouraging.
- The Passive politeness chatbot was experienced as too apologetic and ...
Read More

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Published in
ACM Transactions on Computer-Human Interaction Volume 29, Issue 2
April 2022
347 pages
ISSN:1073-0516
EISSN:1557-7325
DOI:10.1145/3505202
Editor:
Kristina Höök
KTH Royal Institute of Technology, Sweden
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
- Published: 16 January 2022
- Accepted: 1 September 2021
- Revised: 1 July 2021
- Received: 1 April 2021
Published in tochi Volume 29, Issue 2

Permissions
Request permissions about this article.
Request Permissions

Check for updates
Author Tags
Chatbots
conversational agents
language design
register
user perceptions
Qualifiers
- research-article
- Refereed
Conference
Funding Sources
Other Metrics
View Article Metrics

Article Metrics
- 13
  Total Citations
  View Citations
- 2,497
  Total Downloads
- Downloads (Last 12 months)988
- Downloads (Last 6 weeks)115
Other Metrics
View Author Metrics
Cited By
View all

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

View Full Text

HTML Format

View this article in HTML Format .

View HTML Format

Chatbots Language Design: The Influence of Language Variation on User Experience with Tourist Assistant Chatbots

ACM Transactions on Computer-Human Interaction

Abstract

Supplemental Material

Available for Download

REFERENCES

Cited By

Index Terms

Recommendations

Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data

User Engagement with Chatbots: A Discursive Psychology Approach

Exploring how politeness impacts the user experience of chatbots for mental health support