Surface EMG–based alternative communication (Vojtech et al., 2021)

Purpose: This study aimed to evaluate a novel communication system designed to translate surface electromyographic (sEMG) signals from articulatory muscles into speech using a personalized, digital voice. The system was evaluated for word recognition, prosodic classification, and listener perception of synthesized speech.

Method: sEMG signals were recorded from the face and neck as speakers with (n = 4) and without (n = 4) laryngectomy subvocally recited (silently mouthed) a speech corpus comprising 750 phrases (150 phrases with variable phrase-level stress). Corpus tokens were then translated into speech via personalized voice synthesis (n = 8 synthetic voices) and compared against phrases produced by each speaker when using their typical mode of communication (n = 4 natural voices, n = 4 electrolaryngeal [EL] voices). Naïve listeners (n = 12) evaluated synthetic, natural, and EL speech for acceptability and intelligibility in a visual sort-and-rate task, as well as phrasal stress discriminability via a classification mechanism.

Results: Recorded sEMG signals were processed to translate sEMG muscle activity into lexical content and categorize variations in phrase-level stress, achieving a mean accuracy of 96.3% (SD = 3.10%) and 91.2% (SD = 4.46%), respectively. Synthetic speech was significantly higher in acceptability and intelligibility than EL speech, also leading to greater phrasal stress classification accuracy, whereas natural speech was rated as the most acceptable and intelligible, with the greatest phrasal stress classification accuracy.

Conclusion: This proof-of-concept study establishes the feasibility of using subvocal sEMG-based alternative communication not only for lexical recognition but also for prosodic communication in healthy individuals, as well as those living with vocal impairments and residual articulatory function.

Supplemental Material S1. Voice sample of the phrase "chestnuts are starchy" for one EL speaker containing first word stress.

Supplemental Material S2. Voice sample of the phrase "chestnuts are starchy" for one EL speaker containing last word stress.

Supplemental Material S3. Voice sample of the phrase "chestnuts are starchy" for one EL speaker containing no phrasal stress.

Supplemental Material S4. Voice sample of the phrase "chestnuts are starchy" for one EL speaker's matched synthetic voice containing first word stress.

Supplemental Material S5. Voice sample of the phrase "chestnuts are starchy" for one EL speaker's matched synthetic voice containing last word stress.

Supplemental Material S6. Voice sample of the phrase "chestnuts are starchy" for one EL speaker's matched synthetic voice containing no phrasal stress.

Vojtech, J. M., Chan, M. D., Shiwani, B., Roy, S. H., Heaton, J. T., Meltzner, G. S., Contessa, P., De Luca, G., Patel, R., & Kline, J. C. (2021). Surface electromyography–based recognition, synthesis, and perception of prosodic subvocal speech. Journal of Speech, Language, and Hearing Research. Advance online publication. https://doi.org/10.1044/2021_JSLHR-20-00257

Publisher Note: This article is part of the Special Issue: Selected Papers From the 2020 Conference on Motor Speech—Basic Science and Clinical Innovation.