Abstract
Stress during public speaking is common and adversely affects performance and self-confidence of individuals in various professional contexts. Virtual reality (VR) has been used in psychology and human-computer interaction (HCI) to induce and measure barriers to good public speaking skills, such as anxiety and fear. We reviewed the most significant papers published on major indexed journals across the past 20 years. We outline how minimal research has been conducted to detect stress via auditory means in real time during public speaking. As a measurement criterion, the use of questionnaires and physiological parameters remains prevalent. Furthermore, to induce and provide feedback, past experiments relied on simulated audiences that are either scripted or controlled by outside human agents. In this context, we explore the current complexities, limitations and opportunities for novel systems that can provide more engaging and immersive experiences. Hence, we propose a conceptual framework for the development of a voice analysis-based stress-detection computational algorithmic model that can be integrated into a virtual reality simulation. The implementation of the proposed model would ultimately help users to gradually learn how to overcome their stress in real-time and improve their public speaking performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Agarwalla, S., & Sarma, K. K. (2016). Machine learning based sample extraction for automatic speech recognition using dialectal Assamese speech. Neural Networks, 78, 97–111.
Albornoz, E. M., Milone, D. H., & Rufiner, H. L. (2011). Spoken emotion recognition using hierarchical classifiers. Computer Speech & Language, 25(3), 25.
American Psychological Association. (2022a). Anxiety. In APA dictionary. Retrieved 7/2/2022, from https://dictionary.apa.org/anxiety
American Psychological Association. (2022b). Emotion. In APA dictionary. https://dictionary.apa.org/emotion
American Psychological Association. (2022c). Stress. In APA dictionary https://dictionary.apa.org/stress
Anagnostopoulos, C.-N., Iliou, T., & Giannoukos, I. (2015). Features and classifiers for emotion recognition from speech: A survey from 2000 to 2011. Artificial Intelligence Review, 43(2), 155–177.
Ang, J., Dhillon, R., Krupski, A., Shriberg, E., & Stolcke, A. (2002). Prosody-based automatic detection of annoyance and frustration in human-computer dialog. Interspeech.
Azwar, H., Alam, N., Kazmi, B., Zain-ul-abidin, S., & Khan, S. A. (2016). Virtual reality based immersion systems. International Journal of Technology and Research, 4(4), 82–84. http://jcu.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV07T8MwED71scDAo4B4FJSNKciuX8mEUkhFpRahJCB1qhLHQWJIefX_c6aOWhhYWLx4SHSXfL7vfHcfABtcEf8XJmimCDWhYbJCj1chzQNd6UDjUvKwrH42zK2KC21rjHN3g5Lf0F0utM2aI22XAjlVIMn165tvdaTsfasT1WhD16otIR3rDuP7h2Sdd0HEpk5SroHeFmfuQBntwnPzbNOk_V70crMtZz2q8V9vuAc7Lub0otVHsg8tU_esXLMr7ejB9sZUwgO4fBon2WM08ZI4moyzmTeM0vjWG0-ncWLN7KWzNIun6SFkozi7ufOdnoKPhzKSRKRWouTFIBBKEU5JTgskL7kQduqYDhUzVBotJM8ly_WgwsWUghZVKZQ2lB1Bp17U5hg8rQjJjRIFwwgMA6gwrwJuZw8ybQqu5An0G4PM3T_xMV9b4_Tv7TPYwrBErkrm-tD5fF-ac2ijGy6cN78AJO2vbg
Badshah, A. M., Rahim, N., Ullah, N., Ahmad, J., Muhammad, K., Lee, M. Y., Kwon, S., & Baik, S. W. (2019). Deep features-based speech emotion recognition for smart affective services. Multimedia Tools and Applications, 78(5), 5571–5589. https://doi.org/10.1007/s11042-017-5292-7
Bandela, S. R., & Kumar, T. K. (2019). Speech emotion recognition using semi-NMF feature optimization. Turkish Journal of Electrical Engineering and Computer Sciences, 27(5), 3741–3757. https://doi.org/10.3906/elk-1903-121
Banse, R., & Scherer, K. R. (1996). Acoustic profiles in vocal emotion expression. Journal of Personality and Social Psychology, 70(3), 614.
Barreto, A., Zhai, J., & Adjouadi, M. (2007). Non-intrusive physiological monitoring for automated stress detection in human-computer interaction. Springer.
Batrinca, L., Stratou, G., Shapiro, A., Morency, L.-P., & Scherer, S. (2013). Cicero-towards a multimodal virtual audience platform for public speaking training. In International workshop on intelligent virtual agents.
Beatty, M. J., & Behnke, R. R. (1991). Effects of public speaking trait anxiety and intensity of speaking task on heart rate during performance. Human Communication Research, 18(2), 147–176. https://doi.org/10.1111/j.1468-2958.1991.tb00542.x
Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798–1828. https://doi.org/10.1109/TPAMI.2013.50
Bhagat, K. K., Liou, W.-K., & Chang, C.-Y. (2016). A cost-effective interactive 3D virtual reality system applied to military live firing training. Virtual Reality, 20(2), 127–140. https://doi.org/10.1007/s10055-016-0284-x
Bhavan, A., Chauhan, P., Hitkul, & Shah, R. R. (2019). Bagged Support Vector Machines For Emotion Recognition From Speech. Knowledge-Based Systems, 184, 104886.
Bitouk, D., Verma, R., & Nenkova, A. (2010). Class-level spectral features for emotion recognition. Speech Communication, 52(7-8), 613–625.
Bouchlaghem, D., Shang, H., Whyte, J., & Ganah, A. (2005). Visualisation in architecture, engineering and construction (AEC). Automation in Construction, 14(3), 287–295. https://doi.org/10.1016/j.autcon.2004.08.012
Brundage, S. B., & Hancock, A. B. (2015). Real enough: Using virtual public speaking environments to evoke feelings and behaviors targeted in stuttering assessment and treatment. American Journal of Speech-Language Pathology, 24(2), 139–149. https://doi.org/10.1044/2014_AJSLP-14-0087
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., & Weiss, B. (2005). A database of German emotional speech. Interspeech.
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008). IEMOCAP: Interactive emotional dyadic motion capture database. Language Resources and Evaluation, 42(4), 335–359.
Busso, C., Lee, S., & Narayanan, S. (2009). Analysis of emotionally salient aspects of fundamental frequency for emotion detection. IEEE Transactions on Audio, Speech, and Language Processing, 17(4), 582–596. https://doi.org/10.1109/TASL.2008.2009578
Carl, E., Stein, A. T., Levihn-Coon, A., Pogue, J. R., Rothbaum, B., Emmelkamp, P., Asmundson, G. J. G., Carlbring, P., Powers, M. B., & Stockholms, u., Samhällsvetenskapliga, f., Psykologiska, i., & Klinisk, p. (2019). Virtual reality exposure therapy for anxiety and related disorders: A meta-analysis of randomized controlled trials. Journal of Anxiety Disorders, 61, 27–36. https://doi.org/10.1016/j.janxdis.2018.08.003
Casale, S., Russo, A., & Serrano, S. (2007). Multistyle classification of speech under stress using feature subset selection based on genetic algorithms. Speech Communication, 49(10–11), 801–810. https://doi.org/10.1016/j.specom.2007.04.012
Chittaro, L., & Sioni, R. (2015). Serious games for emergency preparedness: Evaluation of an interactive vs. a non-interactive simulation of a terror attack. Computers in Human Behavior, 50, 508–519. https://doi.org/10.1016/j.chb.2015.03.074
Chollet, M., Stefanov, K., Prendinger, H., & Scherer, S. (2015a). Public speaking training with a multimodal interactive virtual audience framework – Demonstration.
Chollet, M., Wörtwein, T., Morency, L.-P., Shapiro, A., & Scherer, S. (2015b). Exploring feedback strategies to improve public speaking: An interactive virtual audience framework. In Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing. Osaka, Japan. https://doi-org.elibrary.jcu.edu.au/10.1145/2750858.2806060
Clinton, E., Cookson, G., Brown, S., & Woods, D. (2017). The stress of public speaking increases cortisol levels in undergraduates: Is increased preparation really the best remedy? In Vivo, 38(2).
Cowen, A. S., Elfenbein, H. A., Laukka, P., & Keltner, D. (2019). Mapping 24 emotions conveyed by brief human vocalization. American Psychologist, 74(6), 698.
Cox, T., & Mackay, C. (1985). The measurement of self-reported stress and arousal. British journal of psychology, 76(2), 183–186.
Crocetta, T. B., de Araújo, L. V., Guarnieri, R., Massetti, T., Ferreira, F. H. I. B., de Abreu, L. C., & de Mello Monteiro, C. B. (2018). Virtual reality software package for implementing motor learning and rehabilitation experiments. Virtual Reality, 22(3), 199–209. https://doi.org/10.1007/s10055-017-0323-2
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2011). Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE transactions on audio, speech, and language processing, 20(1), 30–42.
Darwin, C., & Prodger, P. (1998). The expression of the emotions in man and animals. Oxford University Press.
Dascalu, M.-I., Bagis, S., Nitu, M., Ferche, O.-M., & Alin Dragos Bogdan, M. (2017). Experiential learning VR system for studying computer architecture. Romanian Journal of Human – Computer Interaction, 10(3), 197–215. http://jcu.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LawIxEB50vXhpFVv6sDLQ82qSjVv3VLQonqRI6VXc2awgZe3L_2_GTVB78NJzIAyZybznG4BIdUX4RyfoJOsnudCUajkwgkQqU2OjryRfmphInw7Mlc2FPBrj2O215F51ZxvirHmPu30kGzzx_PkV8h4prre6pRpVqNm_KFUAtdF49jr3MsboK9ZnYa-YN52ENhoSp2pYOdsyuYSVJ8P4DOCatscTOgfUxn8R24AL537isJSXJlRM0YKRxzu2v_0DHeLqCt_nWOKZo3VskRsOeSQK_R4IHB7VIK7gcTJ-e5mGnqyFE9KfxYGm6BqCYlOYG8CItI2AVJzJp1QvBQ3ymHIyfeJALSFxC-1zN92dP76HumK7uO93bUPw-701D1C1z9pxjNoB6Cuspg
Dechant, M., Trimpl, S., Wolff, C., Mühlberger, A., & Shiban, Y. (2017). Potential of virtual reality as a diagnostic tool for social anxiety: A pilot study. Computers in Human Behavior, 76, 128–134. https://doi.org/10.1016/j.chb.2017.07.005
Deng, J., Zhang, Z., Marchi, E., & Schuller, B. (2013). Sparse autoencoder-based feature transfer learning for speech emotion recognition. In 2013 humaine association conference on affective computing and intelligent interaction.
Dillon, R. (2001). Extracting audio cues in real time to understand musical expressiveness. In Proceedings “Current research directions in computer music”, MOSART Workshop. Barcelona, Spain.
Dillon, R. (2003). A statistical approach to expressive intention recognition in violin performances. In Proceedings of the Stockholm Music Acoustics Conference (SMAC’03) Stockholm.
Dillon, R., Wong, G., & Ang, R. (2006). Virtual orchestra: An immersive computer game for fun and education. In Proceedings of the 2006 International conference on Game research and development.
Docan-Morgan, T., & Nelson, L. L. (2015). Chapter 11: The benefits and necessity of public speaking education – Tony Docan-Morgan & Laura L. Nelson. In Public Speaking for the Curious. https://www.researchgate.net/publication/332974615_The_Benefits_and_Necessity_of_Public_Speaking_Education
Dongrui, W., Courtney, C. G., Lance, B. J., Narayanan, S. S., Dawson, M. E., Oie, K. S., & Parsons, T. D. (2010). Optimal arousal identification and classification for affective computing using physiological signals: Virtual reality stroop task. IEEE Transactions on Affective Computing, 1(2), 109–118. https://doi.org/10.1109/T-AFFC.2010.12
Douglas-Cowie, E., Cowie, R., Sneddon, I., Cox, C., Lowry, O., Mcrorie, M., Martin, J.-C., Devillers, L., Abrilian, S., & Batliner, A. (2007). The HUMAINE database: Addressing the collection and annotation of naturalistic and induced emotional data. In International conference on affective computing and intelligent interaction.
Dowd, H., Dowd, H., Zautra, A., Zautra, A., Hogan, M., & Hogan, M. (2010). Emotion, stress, and cardiovascular response: An experimental test of models of positive and negative affect. International Journal of Behavioral Medicine, 17(3), 189–194. https://doi.org/10.1007/s12529-009-9063-3
Droppleman, L. F., & McNair, D. M. (1971). An experimental analog of public speaking. Journal of Consulting and Clinical Psychology, 36(1), 91.
El Ayadi, M., Kamel, M. S., & Karray, F. (2011). Survey on speech emotion recognition: Features, classification schemes, and databases. Pattern Recognition, 44(3), 572–587. https://doi.org/10.1016/j.patcog.2010.09.020
Elfering, A., & Grebner, S. (2011). Ambulatory assessment of skin conductivity during first thesis presentation: Lower self-confidence predicts prolonged stress response. Applied Psychophysiology and Biofeedback, 36(2), 93–99. https://doi.org/10.1007/s10484-011-9152-3
Elfering, A., & Grebner, S. (2012). Getting used to academic public speaking: Global self-esteem predicts habituation in blood pressure response to repeated thesis presentations. Applied Psychophysiology and Biofeedback, 37(2), 109–120. https://doi.org/10.1007/s10484-012-9184-3
El-Yamri, M., Romero-Hernandez, A., Gonzalez-Riojo, M., & Manero, B. (2019). Designing a VR game for public speaking based on speakers features: A case study. Smart Learning Environments, 6(1), 1–15. https://doi.org/10.1186/s40561-019-0094-1
Feldman, P. J., Cohen, S., Hamrick, N., & Lepore, S. J. (2004). Psychological stress, appraisal, emotion and cardiovascular response in a public speaking task. Psychology & Health, 19(3), 353–368. https://doi.org/10.1080/0887044042000193497
Fredrick, S. S., Demaray, M. K., Malecki, C. K., & Dorio, N. B. (2018). Can social support buffer the association between depression and suicidal ideation in adolescent boys and girls? Psychology in the Schools, 55(5), 490–505.
Fruchter, R., Reidsma, D., op den Akker, H. J. A., Nishida, T., Rienks, R. J., Rosenberg, D., Poppe, R. W., Nijholt, A., Heylen, D. K. J., & Zwiers, J. (2007). Virtual meeting rooms: From observation to simulation. AI & Society, 22(1, suppl./2), 133–144. https://doi.org/10.1007/s00146-007-0129-y
Gandhi, R. (2018). Support vector machine—Introduction to machine learning algorithms. Retrieved 7/2/2022, from https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47
Garcia-Palacios, A., Hoffman, H., Carlin, A., Furness, T. A., & Botella, C. (2002). Virtual reality in the treatment of Spider Phobia: A controlled study. Behaviour research and therapy, 40(9), 983–993. https://doi.org/10.1016/S0005-7967(01)00068-7
Gavish, N., Gutiérrez, T., Webel, S., Rodríguez, J., Peveri, M., Bockholt, U., & Tecchia, F. (2015). Evaluating virtual reality and augmented reality training for industrial maintenance and assembly tasks. Interactive Learning Environments, 23(6), 778–798.
Gebara, C. M., Barros-Neto, T. P. D., Gertsenchtein, L., & Lotufo-Neto, F. (2016). Virtual reality exposure using three-dimensional images for the treatment of social Phobia. Revista brasileira de psiquiatria (Sao Paulo, Brazil: 1999), 38(1), 24–29. https://doi.org/10.1590/1516-4446-2014-1560
Georgiades, A., Sherwood, A., Gullette, E. C. D., Babyak, M. A., Hinderliter, A., Waugh, R., Tweedy, D., Craighead, L., Bloomer, R., & Blumenthal, J. A. (2000). Effects of exercise and weight loss on mental stress–induced cardiovascular responses in individuals with high blood pressure. Hypertension: Journal of the American Heart Association, 36(2), 171–176. https://doi.org/10.1161/01.HYP.36.2.171
Gerardi, M., Cukor, J., Difede, J., Rizzo, A., & Rothbaum, B. O. (2010). Virtual reality exposure therapy for post-traumatic stress disorder and other anxiety disorders. Current Psychiatry Reports, 12(4), 298–305. https://doi.org/10.1007/s11920-010-0128-4
Giannakopoulos, T. (2015). pyAudioAnalysis: An open-source python library for audio signal analysis. PLoS One, 10(12), e0144610. https://doi.org/10.1371/journal.pone.0144610
Gillespie, S., Moore, E., Laures-Gore, J., Farina, M., Russell, S., Logan, Y. Y., & Ieee. (2017). Detecting stress and depression in adults with aphasia through speech analysis. In 2017 Ieee International conference on acoustics, speech and signal processing (pp. 5140–5144). <Go to ISI>://WOS:000414286205060.
Gramer, M., & Saria, K. (2007). Effects of social anxiety and evaluative threat on cardiovascular responses to active performance situations. Biological Psychology, 74(1), 67–74. https://doi.org/10.1016/j.biopsycho.2006.07.004
Gramer, M., & Sprintschnik, E. (2008). Social anxiety and cardiovascular responses to an evaluative speaking task: The role of stressor anticipation. Personality and Individual Differences, 44(2), 371–381. https://doi.org/10.1016/j.paid.2007.08.016
Hamid, O. K. (2018). Frame blocking and windowing speech signal. Journal of Information, Communication, and Intelligence Systems (JICIS), 4, 87–94.
Han, K., Yu, D., & Tashev, I. (2014). Speech emotion recognition using deep neural network and extreme learning machine. In Interspeech 2014.
Han, H., Byun, K., & Kang, H.-G. (2018). A deep learning-based stress detection algorithm with speech signal. In AVSU’18 International Multimedia Conference.
Hansen, J. H. L., & Liu, G. (2016). Unsupervised accent classification for deep data fusion of accent and language information. Speech Communication, 78, 19–33. https://doi.org/10.1016/j.specom.2015.12.004
Hansen, J. H., Bou-Ghazale, S. E., Sarikaya, R., & Pellom, B. (1997). Getting started with SUSAS: A speech under simulated and actual stress database. Eurospeech.
Hao, M., Cao, W.-H., Liu, Z.-T., Wu, M., & Xiao, P. (2020). Visual-audio emotion recognition based on multi-task and ensemble learning with multiple features. Neurocomputing, 391, 42–51.
Hartanto, D., Kampmann, I. L., Morina, N., Emmelkamp, P. G. M., Neerincx, M. A., & Brinkman, W. P. (2014). Controlling social stress in virtual reality environments. PLoS One, 9(3), e92804. https://doi.org/10.1371/journal.pone.0092804
Hilfert, T., & König, M. (2016). Low-cost virtual reality environment for engineering and construction. Visualization in Engineering, 4(1), 1–18. https://doi.org/10.1186/s40327-015-0031-5
Hu, H., Xu, M., & Wu, W. (2007, April 15–20). GMM Supervector Based SVM with Spectral Features for Speech Emotion Recognition. In 2007 IEEE International conference on acoustics, speech and signal processing – ICASSP ‘07.
IBM. (2020). Recurrent neural networks. Retrieved 7/2/2022, from https://www.ibm.com/cloud/learn/recurrent-neural-networks
Issa, D., Fatih Demirci, M., & Yazici, A. (2020). Speech emotion recognition with deep convolutional neural networks. Biomedical Signal Processing and Control, 59, 101894. https://doi.org/10.1016/j.bspc.2020.101894
Jain, M., Narayan, S., Balaji, P., Bhowmick, A., & Muthu, R. K. (2020). Speech emotion recognition using support vector machine. arXiv preprint arXiv:2002.07590.
Jezova, D., Hlavacova, N., Dicko, I., Solarikova, P., & Brezina, I. (2016). Psychosocial stress based on public speech in humans: Is there a real life/laboratory setting cross-adaptation? Stress, 19(4), 429–433. https://doi.org/10.1080/10253890.2016.1203416
Jönsson, P., Wallergård, M., Österberg, K., Hansen, Å. M., Johansson, G., & Karlson, B. (2010). Cardiovascular and cortisol reactivity and habituation to a virtual reality version of the Trier Social Stress Test: A pilot study. Psychoneuroendocrinology, 35(9), 1397–1403. https://doi.org/10.1016/j.psyneuen.2010.04.003
Julião, M., Silva, J., Aguiar, A., Moniz, H., & Batista, F. (2015). Speech features for discriminating stress using branch and bound wrapper search. In International Symposium on Languages, Applications and Technologies.
Junek, W. (2007). Mind reading: The interactive guide to emotions. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 16(4), 182.
Juslin, P. N., & Laukka, P. (2003). Communication of emotions in vocal expression and music performance: different channels, same code? Psychological Bulletin, 129(5), 770–814. https://doi.org/10.1037/0033-2909.129.5.770
Kaiser, J. F. (1993). Some useful properties of Teager’s energy operators. In 1993 IEEE international conference on acoustics, speech, and signal processing.
Kamiloğlu, R. G., Fischer, A. H., & Sauter, D. A. (2020). Good vibrations: A review of vocal expressions of positive emotions. Psychonomic Bulletin & Review, 27(2), 237–265. https://doi.org/10.3758/s13423-019-01701-x
Kappas, A., Hess, U., & Scherer, K. R. (1991). Voice and emotion. In Fundamentals of nonverbal behavior (p. 200). Cambridge University Press.
Kassem, M., Benomran, L., & Teizer, J. (2017). Virtual environments for safety learning in construction and engineering: Seeking evidence and identifying gaps for future research. Visualization in Engineering, 5(1), 1–15. https://doi.org/10.1186/s40327-017-0054-1
Kim, K., Kim, C.-H., Kim, S.-Y., Roh, D., & Kim, S. I. (2009). Virtual reality for obsessive-compulsive disorder: Past and the future. Psychiatry Investigation, 6(3), 115–121. https://doi.org/10.4306/pi.2009.6.3.115
Kimani, E., & Bickmore, T. (2019). Addressing public speaking anxiety in real-time using a virtual public speaking coach and physiological sensors. In Proceedings of the 19th ACM international conference on intelligent virtual agents. Paris, France. https://doi-org.elibrary.jcu.edu.au/10.1145/3308532.3329409
Koolagudi, S. G., & Rao, K. S. (2012). Emotion recognition from speech: A review. International Journal of Speech Technology, 15(2), 99–117.
Koolagudi, S. G., Maity, S., Kumar, V. A., Chakrabarti, S., & Rao, K. S. (2009). IITKGP-SESC: Speech database for emotion analysis. In International conference on contemporary computing.
Koroleva, M., Bakhchina, A., Shyshalov, I., Parin, S. B., & Polevaia, S. A. (2014). Influence of the context of public speaking on human functional state. International Journal of Psychophysiology, 94(2), 230–231. https://doi.org/10.1016/j.ijpsycho.2014.08.901
Kothgassner, O. D., Felnhofer, A., Hlavacs, H., Beutl, L., Palme, R., Kryspin-Exner, I., & Glenk, L. M. (2016). Salivary cortisol and cardiovascular reactivity to a public speaking task in a virtual and real-life environment. Computers in Human Behavior, 62, 124–135. https://doi.org/10.1016/j.chb.2016.03.081
Kothgassner, O. D., Goreis, A., Kafka, J. X., Kaufmann, M., Atteneder, K., Beutl, L., Hennig-Fast, K., Hlavacs, H., & Felnhofer, A. (2019). Virtual social support buffers stress response: An experimental comparison of real-life and virtual support prior to a social stressor. Journal of Behavior Therapy and Experimental Psychiatry, 63, 57–65.
Kotlyar, M., Donahue, C., Thuras, P., Kushner, M. G., O’Gorman, N., Smith, E. A., & Adson, D. E. (2008). Physiological response to a speech stressor presented in a virtual reality environment. Psychophysiology, 45(6), 1034–1037. https://doi.org/10.1111/j.1469-8986.2008.00690.x
Kuchibhotla, S., Vankayalapati, H. D., Vaddi, R., & Anne, K. R. (2014). A comparative analysis of classifiers in emotion recognition through acoustic features. International Journal of Speech Technology, 17(4), 401–408.
Kwon, O.-W., Chan, K., Hao, J., & Lee, T.-W. (2003). Emotion recognition by speech signals. In Eighth European conference on speech communication and technology.
Laver, K. E., George, S., Thomas, S., Deutsch, J. E., & Crotty, M. (2015). Virtual reality for stroke rehabilitation. The Cochrane Database of Systematic Reviews, (2), CD008349. https://doi.org/10.1002/14651858.CD008349.pub3
Le, Q. T., Le, Q. T., Pedro, A., Pedro, A., Park, C. S., & Park, C. S. (2015). A social virtual reality based construction safety education system for experiential learning. Journal of Intelligent & Robotic Systems, 79(3), 487–506. https://doi.org/10.1007/s10846-014-0112-z
Levy, F., Leboucher, P., Rautureau, G., & Jouvent, R. (2016). E-virtual reality exposure therapy in Acrophobia: A pilot study. Journal of Telemedicine and Telecare, 22(4), 215–220. https://doi.org/10.1177/1357633X15598243
Li, X., Tao, J., Johnson, M. T., Soltis, J., Savage, A., Leong, K. M., & Newman, J. D. (2007). Stress and emotion classification using jitter and shimmer features. In 2007 IEEE International conference on acoustics, speech and signal processing-ICASSP’07.
Lin, Y.-L., & Wei, G. (2005). Speech emotion recognition based on HMM and SVM. In 2005 international conference on machine learning and cybernetics.
Lindner, P., Miloff, A., Fagernäs, S., Andersen, J., Sigeman, M., Andersson, G., Furmark, T., Carlbring, P., & Linköpings, u., Institutionen för beteendevetenskap och, l., Filosofiska, f., & Psykologi. (2019). Therapist-led and self-led one-session virtual reality exposure therapy for public speaking anxiety with consumer hardware and software: A randomized controlled trial. Journal of Anxiety Disorders, 61, 45–54. https://doi.org/10.1016/j.janxdis.2018.07.003
Liscombe, J., Riccardi, G., & Hakkani-Tur, D. (2005). Using context to improve emotion detection in spoken dialog systems. In The Proceedings of EUROSPEECH 2005.
Lister, H. A., Piercey, C. D., & Joordens, C. (2010). The effectiveness of 3-D video virtual reality for the treatment of fear of public speaking. Journal of Cyber Therapy and Rehabilitation, 3(4), 375. http://jcu.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LSwMxEB5avYjgAxXf5OZpbTY7yW4OIkVb6sWC9OahJLOJUKQtffx_k-22Vk-Cl-SSQEgmk2_CfPMBZOKeJ798AupSas-RLKaF48Rtal2IvrQ3ThHhT8IcdNfUmCr5PbrKymmb6XTeKifUagspMEqw5a12_7XzuCWmsXwYxfTSIQWM2oRdoSTGkvov7f7m6yWEhFJUwoJpXmASXmisHfI-NDCr35buIXyul-HWP4AjWm4zdL6rNv5_sUdwUGNQ1l4ZzTE03PgE3oPBsFV2R-0A2cSzLHlmkak3Ce0sUk1YAJkRubMAdlkAj2yTqR6H-3BvYr8qns3mU1eJXZ3CoNsZPPWSWnkh-VAZT7yRKuepUzqAFV5aFMJxnxYcS2eMLpTUWkjDrXIhPvFaWlLOKasRc-JoszPYGU_G7hxYRhgCKqHKNLdoOBVekScnKcZ9mvgF3MUdG8brtJgZMjUrIMyOhamG35t3-eeRV7An6qQTIa5hZzFbuhtohrO7rQwhtr23L_APxyU
Livingstone, S. R., & Russo, F. A. (2018). The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English. PloS One, 13(5), e0196391.
Low, L.-S. A., Maddage, N. C., Lech, M., Sheeber, L. B., & Allen, N. B. (2010). Detection of clinical depression in adolescents’ speech during family interactions. IEEE Transactions on Biomedical Engineering, 58(3), 574–586.
Lugger, M., & Yang, B. (2007). The relevance of voice quality features in speaker independent emotion recognition. In 2007 IEEE International conference on acoustics, speech and signal processing-ICASSP’07.
Manju, T., Padmavathi, S., & Tamilselvi, D. (2017). A rehabilitation therapy for autism spectrum disorder using virtual reality. In International Conference On Intelligent Information Technologies.
Marsella, S., Xu, Y., Lhommet, M., Feng, A., Scherer, S., & Shapiro, A. (2013). Virtual character performance from speech. In Proceedings of the 12th ACM SIGGRAPH/Eurographics symposium on computer animation.
Martin, D. J. J. H. (2021). Hidden Markov Models. https://web.stanford.edu/~jurafsky/slp3/A.pdf
Martin, O., Kotsia, I., Macq, B., & Pitas, I. (2006). The eNTERFACE’05 audio-visual emotion database. In 22nd International Conference on Data Engineering Workshops (ICDEW’06).
Maskey, M., Lowry, J., Rodgers, J., McConachie, H., & Parr, J. R. (2014). Reducing specific phobia/fear in young people with autism spectrum disorders (ASDs) through a virtual reality environment intervention. PloS One, 9(7), e100374. https://doi.org/10.1371/journal.pone.0100374
McFee, B., Raffel, C., Liang, D., Ellis, D. P., McVicar, M., Battenberg, E., & Nieto, O. (2015). librosa: Audio and music signal analysis in python. In Proceedings of the 14th python in science conference.
Miloff, A., Lindner, P., Hamilton, W., Reuterskiöld, L., Andersson, G., Carlbring, P., Stockholms, U., Samhällsvetenskapliga, F., Psykologiska, I., & Klinisk, P. (2016). Single-session gamified virtual reality exposure therapy for spider phobia vs. traditional exposure therapy: Study protocol for a randomized controlled non-inferiority trial. Trials, 17(1), 60. https://doi.org/10.1186/s13063-016-1171-1
Mitchell, R. L., & Ross, E. D. (2013). Attitudinal prosody: What we know and directions for future study. Neuroscience & Biobehavioral Reviews, 37(3), 471–479.
Molina, K. I., Ricci, N. A., de Moraes, S. A., & Perracini, M. R. (2014). Virtual reality using games for improving physical functioning in older adults: A systematic review. Journal of Neuro Engineering and Rehabilitation, 11(1), 156. https://doi.org/10.1186/1743-0003-11-156
Motraghi, T. E., Seim, R. W., Meyer, E. C., & Morissette, S. B. (2014). Virtual reality exposure therapy for the treatment of posttraumatic stress disorder: A methodological review using CONSORT guidelines. Journal of Clinical Psychology, 70(3), 197–208. https://doi.org/10.1002/jclp.22051
Mujber, T. S., Szecsi, T., & Hashmi, M. S. (2004). Virtual reality applications in manufacturing process simulation. Journal of materials processing technology, 155, 1834–1838.
Neiberg, D., Elenius, K., Laskowski, K., & Isca. (2006). Emotion recognition in spontaneous speech using GMMs. <Go to ISI>://WOS:000269965900203.
Nijholt, A. (2014). Breaking fresh ground in human-media interaction research. Frontiers in ICT, 2014.
Nogueiras, A., Moreno, A., Bonafonte, A., & Mariño, J. B. (2001). Speech emotion recognition using hidden Markov models. In Seventh European Conference on Speech Communication and Technology.
North, M. M., North, S. M., & Coble, J. R. (1998). Virtual reality therapy: An effective treatment for the fear of public speaking. International Journal of Virtual Reality, 3(3), 1–6. https://doi.org/10.20870/IJVR.1998.3.3.2625
Nwe, T. L., Foo, S. W., & De Silva, L. C. (2003). Detection of stress and emotion in speech using traditional and FFT based log energy features. In Fourth international conference on information, communications and signal processing, 2003 and the Fourth Pacific Rim conference on multimedia. Proceedings of the 2003 Joint.
Ooi, C. S., Seng, K. P., Ang, L.-M., & Chew, L. W. (2014). A new approach of audio emotion recognition. Expert Systems with Applications, 41(13), 5858–5869. https://doi.org/10.1016/j.eswa.2014.03.026
Opriş, D., Pintea, S., García-Palacios, A., Botella, C., Szamosközi, Ş., & David, D. (2012). Virtual reality exposure therapy in anxiety disorders: A quantitative meta-analysis. Depression and Anxiety, 29(2), 85–93. https://doi.org/10.1002/da.20910
Ordaz, N., Romero, D., Gorecky, D., & Siller, H. R. (2015). Serious games and virtual simulator for automotive manufacturing education & training. Procedia Computer Science, 75, 267–274. https://doi.org/10.1016/j.procs.2015.12.247
Owens, M. E., Owens, M. E., Beidel, D. C., & Beidel, D. C. (2015). Can virtual reality effectively elicit distress associated with social anxiety disorder? Journal of Psychopathology and Behavioral Assessment, 37(2), 296–305. https://doi.org/10.1007/s10862-014-9454-x
Palo, H., Mohanty, M. N., & Chandra, M. (2015). Design of neural network model for emotional speech recognition. In Artificial intelligence and evolutionary algorithms in engineering systems (pp. 291–300). Springer.
Pan, X., Gillies, M., Barker, C., Clark, D. M., & Slater, M. (2012). Socially anxious and confident men interact with a forward virtual woman: An experimental study. PloS One, 7(4), e32931. https://doi.org/10.1371/journal.pone.0032931
Parsons, T. D., & Rizzo, A. A. (2008). Affective outcomes of virtual reality exposure therapy for anxiety and specific phobias: A meta-analysis. Journal of Behavior Therapy and Experimental Psychiatry, 39(3), 250–261.
Pedro, A., Le, Q. T., & Park, C. S. (2016). Framework for integrating safety into construction methods education through interactive virtual reality. Journal of Professional Issues in Engineering Education and Practice, 142(2), 4015011. https://doi.org/10.1061/(ASCE)EI.1943-5541.0000261
Pellett, K., & Zaidi, S. F. M. (2019). A framework for virtual reality training to improve public speaking. In 25th ACM symposium on virtual reality software and technology. Parramatta, NSW, Australia. https://doi-org.elibrary.jcu.edu.au/10.1145/3359996.3364727
Pertaub, D., Slater, M., & Barker, C. (2001). An experiment on fear of public speaking in virtual reality. Studies in Health Technology and Informatics, 81, 372–378.
Pfister, T., & Robinson, P. (2011). Real-time recognition of affective states from nonverbal features of speech and its application for public speaking skill analysis. IEEE Transactions on Affective Computing, 2(2), 66–78. https://doi.org/10.1109/T-AFFC.2011.8
Pisanski, K., Nowak, J., & Sorokowski, P. (2016). Individual differences in cortisol stress response predict increases in voice pitch during exam stress. Physiology & Behavior, 163, 234–238. https://doi.org/10.1016/j.physbeh.2016.05.018
Poorna, S. S., Anuraj, K., & Nair, G. J. (2018). A weight based approach for emotion recognition from speech: An analysis using South Indian languages. In I. Zelinka, R. Senkerik, G. Panda, & P. S. L. Kanthan (Eds.), Soft computing systems, Icscs 2018 (Vol. 837, pp. 14–24). https://doi.org/10.1007/978-981-13-1936-5_2
Powers, M. B., Briceno, N. F., Gresham, R., Jouriles, E. N., Emmelkamp, P. M. G., & Smits, J. A. J. (2013). Do conversations with virtual avatars increase feelings of social anxiety? Journal of Anxiety Disorders, 27(4), 398–403. https://doi.org/10.1016/j.janxdis.2013.03.003
Přibil, J., & Přibilová, A. (2013). Evaluation of influence of spectral and prosodic features on gmm classification of Czech and Slovak emotional speech. EURASIP Journal on Audio, Speech, and Music Processing, 2013(1), 1–22. https://doi.org/10.1186/1687-4722-2013-8
Rao, K. S., Rao, K. S., Koolagudi, S. G., Koolagudi, S. G., Vempada, R. R., & Vempada, R. R. (2013). Emotion recognition from speech using global and local prosodic features. International Journal of Speech Technology, 16(2), 143–160. https://doi.org/10.1007/s10772-012-9172-2
Rong, J., Li, G., & Chen, Y.-P. P. (2009). Acoustic feature selection for automatic emotion recognition from speech. Information Processing and Management, 45(3), 315–328. https://doi.org/10.1016/j.ipm.2008.09.003
Ruscio, A. M., Brown, T. A., Chiu, W. T., Sareen, J., Stein, M. B., & Kessler, R. C. (2008). Social fears and social Phobia in the USA: Results from the national comorbidity survey replication. Psychological medicine, 38(1), 15–28. https://doi.org/10.1017/S0033291707001699
Ryerson Multimedia Research Lab., R. U. (2017). RML emotion database. http://shachi.org/resources/4965
Sanchez, M. H., Tur, G., Ferrer, L., & Hakkani-Tür, D. (2010). Domain adaptation and compensation for emotion detection. Eleventh Annual Conference of the International Speech Communication Association.
Scherer, K. R. (1995). Expression of emotion in voice and music. Journal of Voice, 9(3), 235–248.
Scherer, K. R., Banse, R., Wallbott, H. G., & Goldbeck, T. (1991). Vocal cues in emotion encoding and decoding. Motivation and Emotion, 15(2), 123–148. https://doi.org/10.1007/BF00995674
Schmid Mast, M., Kleinlogel, E. P., Tur, B., & Bachmann, M. (2018). The future of interpersonal skills development: Immersive virtual reality training with virtual humans. Human Resource Development Quarterly, 29(2), 125–141. https://doi.org/10.1002/hrdq.21307
Schneider, J., Börner, D., Rosmalen, P. V., & Specht, M. (2015). Presentation trainer, your public speaking multimodal coach. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. Seattle, Washington, USA. https://doi-org.elibrary.jcu.edu.au/10.1145/2818346.2830603
Schreiber, L. (2011). Informative speaking. In The Public Speaking Project (Ed.), Public speaking. The virtual text (pp. 15–11).
Schuller, B., Rigoll, G., & Lang, M. (2003a). Hidden Markov model-based speech emotion recognition (Vol. 2). https://doi.org/10.1109/ICME.2003.1220939
Schuller, B., Rigoll, G., & Lang, M. (2003b). Hmm-based music retrieval using stereophonic feature information and framelength adaptation. In 2003 International Conference on Multimedia and Expo. ICME’03. Proceedings (Cat. No. 03TH8698).
Schuller, B., Seppi, D., Batliner, A., Maier, A., & Steidl, S. (2007). Towards more reality in the recognition of emotional speech. In 2007 IEEE international conference on acoustics, speech and signal processing-ICASSP’07.
Schuller, B., Batliner, A., Steidl, S., & Seppi, D. (2011). Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge. Speech Communication, 53(9), 1062–1087. https://doi.org/10.1016/j.specom.2011.01.011
Scikit Learn. (2022). Support vector machines. Retrieved 7/2/2022, from https://scikit-learn.org/stable/modules/svm.html#:~:text=Support%20vector%20machines%20(SVMs)%20are,classification%2C%20regression%20and%20outliers%20detection.&text=Uses%20a%20subset%20of%20training,it%20is%20also%20memory%20efficient
Semwal, N., Kumar, A., & Narayanan, S. (2017). Automatic speech emotion detection system using multi-domain acoustic feature selection and classification models. <Go to ISI>://WOS:000411739900003.
Shabahi, S. (2020). my-voice-analysis 0.7. https://pypi.org/project/my-voice-analysis/
Slater, M., Pertaub, D.-P., & Steed, A. (1999). Public speaking in virtual reality: Facing an audience of avatars. IEEE Computer Graphics and Applications, 19(2), 6–9.
Slater, M., Pertaub, D.-P., Barker, C., & Clark, D. M. (2006). An experimental study on fear of public speaking using a virtual environment. Cyberpsychology & Behavior, 9(5), 627–633.
Soltani, K., & Ainon, R. N. (2007). Speech emotion detection based on neural networks. <Go to ISI>://WOS:000259439900203.
Sun, R., & Moore, E. (2011). Investigating glottal parameters and teager energy operators in emotion recognition. International Conference on Affective Computing and Intelligent Interaction.
Takac, M., Collett, J., Blom, K. J., Conduit, R., Rehm, I., & De Foe, A. (2019). Public speaking anxiety decreases within repeated virtual reality training sessions. PloS One, 14(5), e0216288. https://doi.org/10.1371/journal.pone.0216288
Tanveer, M. I., Lin, E., & Hoque, M. (2015). Rhema: A real-time in-situ intelligent interface to help people with public speaking. In Proceedings of the 20th international conference on intelligent user interfaces. Atlanta, Georgia, USA. https://doi-org.elibrary.jcu.edu.au/10.1145/2678025.2701386
Tashev, I. J., Wang, Z. Q., Godin, K., & Ieee. (2017). Speech emotion recognition based on gaussian mixture models and deep neural networks 2017 Information Theory and Applications Workshop, <Go to ISI>://WOS:000426450600036.
Teoh, A. N., & Hilmert, C. (2018). Social support as a comfort or an encouragement: A systematic review on the contrasting effects of social support on cardiovascular reactivity. British Journal of Health Psychology, 23(4), 1040–1065. https://doi.org/10.1111/bjhp.12337
The EndNote Team. (2013). EndNote. In (Version EndNote X9) [64 bit]. Clarivate.
Thorsteinsson, E. B., & James, J. E. (1999). A meta-analysis of the effects of experimental manipulations of social support during laboratory stress. Psychology & Health, 14(5), 869–886. https://doi.org/10.1080/08870449908407353
Ticknor, B. (2018). Using virtual reality to treat offenders: An examination. International Journal of Criminal Justice Sciences, 13(2), 316–325. http://jcu.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV07T8MwED6VdunCo4B4lMoTWyCt09hmQQW1YgJUFRgrxw_EQwmkrQT_nnPiUMrAwpIhGWL5ku8-2_fdB0B7J2HwCxMUZWHXCENjixG3oiu5soorvOhIaLsqmCuLC500xoe7QskCunWm3K75KRIH51GAGfv87T1wPlLuvNWbaqxBw7kt4XKscTG8vh1X6Iy4UMjjXJeaAMmJWMHgGqU-s4w24LEahKn2_57V4qc-Z9mz8V9D3YR1Tz7JoPxatqBm0hY0vzHwswXtUq5LHsyrlbkhx6S6keUv28CLCgNy_5Q72QkZm4LFk3lGJo58khtrC2u62RkZpGT4IV2hjQv9DtyNhpPLq8B7LwRIERGakfYlth9aG2vMcyo0PXdEiOtDpkXsiqyZ4EmfSxkxZAEaaRW-wCQ81kZIai3dhXqapWYPiGJhKA3rJxTpGrItIS2PXKNCqkwSsXgf2tWkTf0PNJsuZ-zg78eH0EQOw8sK2jbU5_nCHMEahqrjQ98p1tlf4p7B7A
Ticknor, B. (2019). Virtual reality and correctional rehabilitation: A game changer. Criminal Justice and Behavior, 46(9), 1319–1336. https://doi.org/10.1177/0093854819842588
Truong, K. P., & van Leeuwen, D. A. (2007). Automatic discrimination between laughter and speech. Speech Communication, 49(2), 144–158. https://doi.org/10.1016/j.specom.2007.01.001
Truong, K. P., Van Leeuwen, D. A., & De Jong, F. M. (2012). Speech-based recognition of self-reported and observed emotion in a dimensional space. Speech Communication, 54(9), 1049–1063.
Uchino, B. N., & Garvey, T. S. (1997). The availability of social support reduces cardiovascular reactivity to acute psychological stress. Journal of Behavioral Medicine, 20(1), 15–27. https://doi.org/10.1023/A:1025583012283
Valmaggia, L. R., Latif, L., Kempton, M. J., & Rus-Calafell, M. (2016). Virtual reality in the psychological treatment for mental health problems: An systematic review of recent evidence. Psychiatry Research, 236, 189–195. https://doi.org/10.1016/j.psychres.2016.01.015
VanderPlas, J. (2016). Python data science handbook. In Python data science handbook. O’Reilly Media, Inc.
Wallbott, H. G., & Scherer, K. R. (1986). How universal and specific is emotional experience? Evidence from 27 countries on five continents. Information (International Social Science Council), 25(4), 763–795.
Wang, K., An, N., Li, B. N., Zhang, Y., & Li, L. (2015). Speech emotion recognition using fourier parameters. IEEE Transactions on affective computing, 6(1), 69–75.
Westenberg, P. M., Bokhorst, C. L., Miers, A. C., Sumter, S. R., Kallen, V. L., van Pelt, J., & Blöte, A. W. (2009). A prepared speech in front of a pre-recorded audience: Subjective, physiological, and neuroendocrine responses to the leiden public speaking task. Biological Psychology, 82(2), 116–124. https://doi.org/10.1016/j.biopsycho.2009.06.005
Wiemers, U. S., Schultheiss, O. C., & Wolf, O. T. (2015). Public speaking in front of an unreceptive audience increases implicit power motivation and its endocrine arousal signature. Hormones and Behavior, 71, 69–74. https://doi.org/10.1016/j.yhbeh.2015.04.007
Wirtz, P. H., Ehlert, U., Kottwitz, M. U., La Marca, R., & Semmer, N. K. (2013). Occupational role stress is associated with higher cortisol reactivity to acute stress. Journal of Occupational Health Psychology, 18(2), 121–131. https://doi.org/10.1037/a0031802
Wörtwein, T., Chollet, M., Schauerte, B., Morency, L.-P., Stiefelhagen, R., & Scherer, S. (2015). Multimodal public speaking performance assessment. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. Seattle, Washington, USA. https://doi-org.elibrary.jcu.edu.au/10.1145/2818346.2820762
Zeng, Y., Mao, H., Peng, D., & Yi, Z. (2019). Spectrogram based multi-task audio classification. Multimedia Tools and Applications, 78(3), 3705–3722. https://doi.org/10.1007/s11042-017-5539-3
Zhou, G., Hansen, J. H., & Kaiser, J. F. (2001). Nonlinear feature based classification of speech under stress. IEEE Transactions on Speech and Audio Processing, 9(3), 201–216.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Arushi, Dillon, R., Teoh, A.N., Dillon, D. (2023). Detecting Public Speaking Stress via Real-Time Voice Analysis in Virtual Reality: A Review. In: Eijdenberg, E.L., Mukherjee, M., Wood, J. (eds) Innovation-Driven Business and Sustainability in the Tropics. SEIGOP 2023. Springer, Singapore. https://doi.org/10.1007/978-981-99-2909-2_7
Download citation
DOI: https://doi.org/10.1007/978-981-99-2909-2_7
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-2908-5
Online ISBN: 978-981-99-2909-2
eBook Packages: Business and ManagementBusiness and Management (R0)