Abstract
In this paper, a virtual assistant framework called Alice is presented. This virtual assistant is a combination of 3D avatar, face detection, face recognition and face expression recognition with a voice assistant that similar to Amazon’s Alexa. The 3D avatar (Alice) is a female character animated using Unity and the lip is animated to sync with the speech to make it looks like speaking. Besides that, the 3D avatar can display different facial expressions such as happy, sad and upset. Face detection and recognition makes the system aware of the human user’s identity. Whereas, face expression recognition enables the system to detect the facial expression of the human user. Whenever there is a question being asked, the system will use Speech-to-Text system to convert human speech to text and Natural Language Processing to interpret the intent behind the text. Based on the result of interpretation, the system decides which audio file to be used as response. Then, a realistic artificial voice is generated as response to the human user. The system can access database based on user’s identity to retrieve information about that user. This may create a personalized experience for the human user. This framework can be customized for other applications for different fields. For this Alice framework, two applications have been developed namely a question answering chatbot and a customer service agent.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adam M, Wessel M, Benlian A (2020) AI-based chatbots in customer service and their effects on user compliance. Electron markets
Luo X, Tong S, Fang Z, Qu Z (2019) Frontiers: machines versus humans: the impact of artificial intelligence chatbot disclosure on customer purchases. Mark Sci 38(6):937–947
Herrera A, Yaguachi L, Piedra N (2019) Building conversational interface for customer support applied to open campus an open online course provider. In: 2019 IEEE 19th international conference on advanced learning technologies (ICALT), pp. 11–13
Patel NP, Parikh DR, Patel DA, Patel RR (2019) AI and web-based human-like interactive university chatbot (UNIBOT). In: 2019 3rd international conference on electronics, communication and aerospace technology (ICECA), pp. 148–150
Wu EH, Lin C, Ou Y, Liu C, Wang W, Chao C (2020) Advantages and constraints of a hybrid model K-12 e-learning assistant chatbot. IEEE Acc 8:77788–77801
Hoy MB (2018) Alexa, siri, cortana, and more: an introduction to voice assistants. Med Ref Ser Q 37(1):81–88
Këpuska V, Bohouta G (2018) Next-generation of virtual personal assistants (microsoft cortana, apple siri, amazon alexa and google home). In: 2018 IEEE 8th annual computing and communication workshop and conference (CCWC), pp. 99–103
Gelbukh A (2005) Natural language processing. In: Fifth international conference on hybrid intelligent systems (HIS’05). Rio de Janeiro, Brazil
Yu AW, Dohan D, Luong M, Zhao R, Chen K, Norouzi M, Le QV (2018) Qanet Combining local convolution with global self-attention for reading comprehension. In: Proc ICLR
Hofmann S, Reinecke M (2009) Cognitive–behavioral therapy with adults. Cambridge University Press
Fitzpatrick KK, Darcy A, Vierhile M (2017) Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR Mental Health 4(2)
Wan Y, Chiu C, Liang K, Chang P (2019) Midoriko chatbot: LSTM-based emotional 3D avatar. In: 2019 IEEE 8th global conference on consumer electronics (GCCE). Osaka, Japan, pp. 937–940
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Angga PA, Fachri WE, Elevanita A, Suryadi, Agushinta RD (2015) Design of chatbot with 3D avatar, voice interface, and facial expression. In: 2015 international conference on science in information technology (ICSITech), pp. 326–330
Arsenijevic U, Jovic M (2019) Artificial intelligence marketing: chatbots. In: 2019 international conference on artificial intelligence: applications and innovations (IC-AIAI), pp. 19–193
Rhubarb Lip Sync. https://github.com/DanielSWolf/rhubarb-lip-sync
King DE (2009) Dlib-ml: a machine learning toolkit. J Mach Learn Res 10:1755–1758
Cao Q, Shen L, Xie W, Parkhi OM, Zisserman A (2018) VGGFace2: a dataset for recognising faces across pose and age. In: 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018). Xi’an, pp. 67–74
Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size. https://arxiv.org/abs/1602.07360
Taniai H (2018), Keras-facenet. https://github.com/nyoki-mtl/keras-facenet
Serengil SI (2018) Deep face recognition with keras. https://sefiks.com/2018/08/06/deep-face-recognition-with-keras/
Knyazev B, Shvetsov R, Efremova N, Kuharenko A (2018) Leveraging large face recognition data for emotion classification. In: IEEE international conference on automatic face and gesture recognition (FG 2018). Xi’an, pp. 692–696
Kim JY, Liu C, Calvo RA, McCabe K, Taylor SCR, Schuller BW, Wu K (2019) A comparison of online automatic speech recognition systems and the nonverbal responses to unintelligible speech. arXiv:1904.12403
Acknowledgements
This project was financially funded by Telekom Malaysia Research and Development (TM R&D) Grant.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Poh, SC. et al. (2021). Alice: A General-Purpose Virtual Assistant Framework. In: Alfred, R., Iida, H., Haviluddin, H., Anthony, P. (eds) Computational Science and Technology. Lecture Notes in Electrical Engineering, vol 724. Springer, Singapore. https://doi.org/10.1007/978-981-33-4069-5_31
Download citation
DOI: https://doi.org/10.1007/978-981-33-4069-5_31
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-33-4068-8
Online ISBN: 978-981-33-4069-5
eBook Packages: Computer ScienceComputer Science (R0)