ABSTRACT
Speech-to-text services aim to transcribe input audio as accurately as possible. They increasingly play a role in everyday life, for example in personal voice assistants or in customer-company interactions. We evaluate Open AI’s Whisper, a state-of-the-art automated speech recognition service outperforming industry competitors, as of 2023. While many of Whisper’s transcriptions were highly accurate, we find that roughly 1% of audio transcriptions contained entire hallucinated phrases or sentences which did not exist in any form in the underlying audio. We thematically analyze the Whisper-hallucinated content, finding that 38% of hallucinations include explicit harms such as perpetuating violence, making up inaccurate associations, or implying false authority. We then study why hallucinations occur by observing the disparities in hallucination rates between speakers with aphasia (who have a lowered ability to express themselves using speech and voice) and a control group. We find that hallucinations disproportionately occur for individuals who speak with longer shares of non-vocal durations—a common symptom of aphasia. We call on industry practitioners to ameliorate these language-model-based hallucinations in Whisper, and to raise awareness of potential biases amplified by hallucinations in downstream applications of speech-to-text models.
- Kirrie J Ballard, Nicole M Etter, Songjia Shen, Penelope Monroe, and Chek Tien Tan. 2019. Feasibility of automatic speech recognition for providing feedback during tablet-based treatment for apraxia of speech plus aphasia. American journal of speech-language pathology 28, 2S (2019), 818–834.Google Scholar
- Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2017. Fairness in machine learning. Nips tutorial 1 (2017), 2017.Google Scholar
- Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2023. Fairness and Machine Learning: Limitations and Opportunities. MIT Press.Google Scholar
- David Frank Benson and Alfredo Ardila. 1996. Aphasia: A clinical perspective. Oxford University Press, USA.Google ScholarCross Ref
- Hervé Bredin and Antoine Laurent. 2021. End-to-end speaker segmentation for overlap-aware resegmentation. In Proc. Interspeech 2021. Brno, Czech Republic.Google ScholarCross Ref
- Chris Code and Brian Petheram. 2011. Delivering for aphasia. International Journal of Speech-Language Pathology 13, 1 (Feb. 2011), 3–10. https://doi.org/10.3109/17549507.2010.520090Google ScholarCross Ref
- Antonio R. Damasio. 1992. Aphasia. New England Journal of Medicine 326, 8 (Feb. 1992), 531–539. https://doi.org/10.1056/nejm199202203260806Google ScholarCross Ref
- Charles Ellis and Stephanie Urban. 2016. Age and aphasia: a review of presence, type, recovery and clinical outcomes. Topics in Stroke Rehabilitation 23, 6 (2016), 430–439. https://doi.org/10.1080/10749357.2016.1150412 arXiv:https://doi.org/10.1080/10749357.2016.1150412PMID: 26916396.Google ScholarCross Ref
- Gunther Eysenbach. 2023. The Role of ChatGPT, Generative Language Models, and Artificial Intelligence in Medical Education: A Conversation With ChatGPT and a Call for Papers. JMIR Medical Education 9 (March 2023), e46885. https://doi.org/10.2196/46885Google ScholarCross Ref
- Rita Frieske and Bertram E Shi. 2024. Hallucinations in Neural Automatic Speech Recognition: Identifying Errors and Hallucinatory Models. arXiv preprint arXiv:2401.01572 (2024).Google Scholar
- Graham R Gibbs. 2007. Thematic coding and categorizing. Analyzing qualitative data 703 (2007), 38–56.Google ScholarCross Ref
- Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. Comput. Surveys 55, 12 (2023), 1–38.Google ScholarDigital Library
- Dietrich Klakow and Jochen Peters. 2002. Testing the correlation of word error rate and perplexity. Speech Communication 38, 1-2 (Sept. 2002), 19–28. https://doi.org/10.1016/s0167-6393(01)00041-3Google ScholarDigital Library
- Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John R Rickford, Dan Jurafsky, and Sharad Goel. 2020. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences 117, 14 (2020), 7684–7689.Google ScholarCross Ref
- Duc Le, Keli Licata, and Emily Mower Provost. 2018. Automatic quantitative analysis of spontaneous aphasic speech. Speech Communication 100 (2018), 1–12.Google ScholarCross Ref
- Duc Le and Emily Mower Provost. 2016. Improving Automatic Recognition of Aphasic Speech with AphasiaBank. In Proc. Interspeech 2016. 2681–2685. https://doi.org/10.21437/Interspeech.2016-213Google ScholarCross Ref
- Debbie Loakes. 2022. Does Automatic Speech Recognition (ASR) Have a Role in the Transcription of Indistinct Covert Recordings for Forensic Purposes?Frontiers in Communication 7 (2022), 803452.Google Scholar
- Julio C. Hidalgo Lopez, Shelly Sandeep, MaKayla Wright, Grace M. Wandell, and Anthony B. Law. 2023. Quantifying and Improving the Performance of Speech Recognition Systems on Dysphonic Speech. Otolaryngology–Head and Neck Surgery 168, 5 (Jan. 2023), 1130–1138. https://doi.org/10.1002/ohn.170Google ScholarCross Ref
- B. MacWhinney, D. Fromm, M. Forbes, and A. Holland. 2011. AphasiaBank: Methods for studying discourse. Aphasiology 25 (2011), 1286–1307.Google ScholarCross Ref
- John Markoff. 2019. From Your Mouth to Your Screen, Transcribing Takes the Next Step. New York Times (October 2019).Google Scholar
- Tara McAllister and Kirrie J Ballard. 2018. Bringing advanced speech processing technology to the clinical management of speech disorders., 581–582 pages.Google Scholar
- Robert McMillan. 2023. With AI, Hackers Can Simply Talk Computers Into Misbehaving. Wall Street Journal (August 2023).Google Scholar
- Cade Metz, Cecilia Kang, Sheera Frenkel, Stuart A. Thompson, and Nico Grant. 2024. How Tech Giants Cut Corners to Harvest Data for A.I.New York Times (April 2024).Google Scholar
- OpenAI. 2023. GPT 3.5. https://platform.openai.com/docs/models/gpt-3-5. Accessed: 2023-11-25.Google Scholar
- OpenAI. 2023. Speech to text. https://platform.openai.com/docs/guides/speech-to-text. Accessed: 2023-11-25.Google Scholar
- Orestis Papakyriakopoulos, Anna Seo Gyeong Choi, William Thong, Dora Zhao, Jerone Andrews, Rebecca Bourke, Alice Xiang, and Allison Koenecke. 2023. Augmented Datasheets for Speech Datasets and Ethical Decision-Making. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 881–904.Google ScholarDigital Library
- Alexis Plaquet and Hervé Bredin. 2023. Powerset multi-class cross entropy loss for neural speaker diarization. In Proc. INTERSPEECH 2023.Google ScholarCross Ref
- Ying Qin, Tan Lee, Siyuan Feng, and Anthony Pak-Hin Kong. 2018. Automatic Speech Assessment for People with Aphasia Using TDNN-BLSTM with Multi-Task Learning.. In Interspeech. 3418–3422.Google Scholar
- Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust Speech Recognition via Large-Scale Weak Supervision. arXiv (2022). arXiv:arXiv:2212.04356Google Scholar
- Donald B Rubin. 1980. Bias reduction using Mahalanobis-metric matching. Biometrics (1980), 293–298.Google Scholar
- Prashant Serai, Vishal Sunder, and Eric Fosler-Lussier. 2022. Hallucination of Speech Recognition Errors With Sequence to Sequence Learning. IEEE/ACM Transactions on Audio, Speech, and Language Processing 30 (2022), 890–900. https://doi.org/10.1109/taslp.2022.3145313Google ScholarDigital Library
- David Sherfinski and Avi Asher-Schapiro. 2021. U.S. prisons mull AI to analyze inmate phone calls. Thomson Reuters Foundation News (August 2021).Google Scholar
- Silero Team. 2021. Silero VAD: pre-trained enterprise-grade Voice Activity Detector (VAD), Number Detector and Language Classifier. https://github.com/snakers4/silero-vad.Google Scholar
- The New York City Council. 2021. A Local Law to amend the administrative code of the city of New York, in relation to automated employment decision tools. https://www.nyc.gov/site/dca/about/automated-employment-decision-tools.page.Google Scholar
- US Department of Labor. 1990. Americans with Disabilities Act. https://www.dol.gov/general/topic/disability/ada.Google Scholar
- US Equal Employment Opportunity Commission. 2008. The ADA: Your Responsibilities as an Employer. https://www.eeoc.gov/publications/ada-your-responsibilities-employer.Google Scholar
- Carolina Paula Vargas, Alejandro Gaiera, Andres Brandán, Alejandro Renato, Sonia Benitez, and Daniel Luna. 2024. Automatic Speech Recognition System to Record Progress Notes in a Mobile EHR: A Pilot Study.Studies in Health Technology and Informatics 310 (2024), 124–128.Google Scholar
Index Terms
- Careless Whisper: Speech-to-Text Hallucination Harms
Recommendations
Whisper to normal speech conversion using pitch estimated from spectrum
A systematic method to connect a whispered vowel to its perceived pitch is presented.A regression function to predict the perceived pitch from the spectral information of the whispered speech is obtained.A whispered-to-normal speech converter is ...
WESPER: Zero-shot and Realtime Whisper to Normal Voice Conversion for Whisper-based Speech Interactions
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing SystemsRecognizing whispered speech and converting it to normal speech creates many possibilities for speech interaction. Because the sound pressure of whispered speech is significantly lower than that of normal speech, it can be used as a semi-silent speech ...
Comments