Skip to main content
SpringerLink
Log in

Speech Recognition from MEG Data Using Covariance Filters

  • Conference paper
  • First Online:
Biologically Inspired Cognitive Architectures 2023 (BICA 2023)

Part of the book series: Studies in Computational Intelligence ((SCI,volume 1130))

Included in the following conference series:

  • 183 Accesses

Abstract

Speech recognition from EEG and MEG data is the first step in the development of BCI and AI systems for further use in the decoding of inner speech. Great achievements in this direction have been made with the use of ECoG and stereo-EEG. At the same time, there are few works on this topic on the analysis of data obtained by noninvasive methods of brain activity registration. Our approach is based on the evaluation of connections in the sensor space with the extraction of the MEG connectivity pattern specific to a given segment of speech. We tested our method on 7 subjects. In all cases, our processing pipeline was sufficiently reliable and worked either without recognition errors or with few errors. After “training” the algorithm is able to recognize a fragment of spoken speech in a single presentation. For recognition, we used MEG recording segments of 50–1200 ms from the beginning of the word. A segment of at least 600 ms was required for high-quality recognition. Intervals longer than 1200 ms degraded the quality of recognition. Bandpass filtering of MEG showed that the quality of recognition is higher when using the gamma frequency range compared to the low-frequency range of the analyzed signal.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 279.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Anumanchipalli, G.K., Chartier, J., Chang, E.F.: Speech synthesis from neural decoding of spoken sentences. Nature 568(7753), 493–498 (2019). https://doi.org/10.1038/s41586-019-1119-1

    Article  Google Scholar 

  2. Norman-Haignere, S.V., Long, L.K., Devinsky, O., Doyle, W., Irobunda, I., Merricks, E.M., Mesgarani, N.: Multiscale temporal integration organizes hierarchical computation in human auditory cortex. Nat. Hum. Behav. 6(3), 455–469 (2022). https://doi.org/10.1038/s41562-021-01261-y

    Article  Google Scholar 

  3. Tang, J., LeBel, A., Jain, S., Huth, A.G.: Semantic reconstruction of continuous language from non-invasive brain recordings. Nat. Neurosci. (2023). https://doi.org/10.1038/s41593-023-01304-9

    Article  Google Scholar 

  4. Vvedensky, V., Filatov, I., Gurtovoy, K., Sokolov, M.: Alpha rhythm dynamics during spoken word recognition. Stud. Comput. Intell. 1064, 65–70 (2023). https://doi.org/10.1007/978-3-031-19032-2_7

    Article  MathSciNet  Google Scholar 

  5. Lizarazu, M., Carreiras, M., Molinaro, N.: Theta-gamma phase-amplitude coupling in auditory cortex is modulated by language proficiency. Hum. Brain Mapp. 44(7), 2862–2872 (2023). https://doi.org/10.1002/hbm.26250

    Article  Google Scholar 

  6. Neymotin, S.A., Tal, I., Barczak, A., O’Connell, M.N., McGinnis, T., Markowitz, N., Lakatos, P.: Detecting spontaneous neural oscillation events in primate auditory cortex. Eneuro 9(4), ENEURO.0281–21 (2022). https://doi.org/10.1523/ENEURO.0281-21.2022

  7. Anurova, I., Vetchinnikova, S., Dobrego, A., Williams, N., Mikusova, N., Suni, A., Palva, S.: Event-related responses reflect chunk boundaries in natural speech. NeuroImage 255, 119203 (2022). https://doi.org/10.1016/j.neuroimage.2022.119203

    Article  Google Scholar 

  8. Dash, D., Ferrari, P., Wang, J.: Decoding imagined and spoken phrases from non-invasive neural (MEG) signals. Front. Neurosci. 14, 290 (2020). https://doi.org/10.3389/fnins.2020.00290

  9. Verkhlyutov, V.: MEG data during the presentation of Gabor patterns and word sets. ZENODO, 7458233 (2022). https://zenodo.org/record/7458233

  10. https://github.com/BrainTravelingWaves/22SpeechRecognition

  11. Defossez, A., Caucheteux, C., Rapin, J., Kabeli, O., King, J.-R. Decoding speech from non-invasive brain recordings. ArXiv, 2208.12266, 1-15. (2022). http://arxiv.org/abs/2208.12266

  12. Sato, N.: Cortical traveling waves reflect state-dependent hierarchical sequencing of local regions in the human connectome network. Sci. Rep. 12(1), 334 (2022). https://doi.org/10.1038/s41598-021-04169-9

  13. Rolls, E.T., Deco, G., Huang, C.-C., Feng, J.: The human language effective connectome. NeuroImage 258, 119352 (2022). https://doi.org/10.1016/j.neuroimage.2022.119352

  14. Liaukovich, K., Ukraintseva, Y., Martynova, O.: Implicit auditory perception of local and global irregularities in passive listening condition. Neuropsychologia 165(July 2020), 108129 (2022). https://doi.org/10.1016/j.neuropsychologia.2021.108129

  15. Proix, T., Delgado Saa, J., Christen, A., Martin, S., Pasley, B.N., Knight, R.T., Giraud, A.-L.: Imagined speech can be decoded from low- and cross-frequency intracranial EEG features. Nat. Commun. 13(1), 48 (2022). https://doi.org/10.1038/s41467-021-27725-3

  16. Arnulfo, G., Wang, S.H., Myrov, V., Toselli, B., Hirvonen, J., Fato, M.M., Palva, J.M.: Long-range phase synchronization of high-frequency oscillations in human cortex. Nat. Commun. 11(1), 5363 (2020). https://doi.org/10.1038/s41467-020-18975-8

Download references

Acknowledgments

This study was supported by the Russian Science Foundation - Grant no. 23-78-00011. The authors are grateful to Chernyshev B. V. and Prokofyev A. O. from the Center for Neurocognitive Research (MEG Center) at the Moscow City University of Psychology and Education.

Author information

Authors and Affiliations

  1. Institute of Higher Nervous Activity and Neurophysiology of RAS, Moscow, Russia

    Vitaly Verkhlyutov & Olga Martynova

  2. RNC Kurchatov Institute, Moscow, Russia

    Victor Vvedensky & Konstantin Gurtovoy

  3. Derzhavin Tambov State University, Tambov, Russia

    Evgenii Burlakov

Authors
  1. Vitaly Verkhlyutov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Victor Vvedensky
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Konstantin Gurtovoy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Evgenii Burlakov
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Olga Martynova
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vitaly Verkhlyutov .

Editor information

Editors and Affiliations

  1. Cybernetics Department, Moscow Engineering Physics Institute, Moscow, Russia

    Alexei V. Samsonovich

  2. College of Science and Technology, Ningbo University, Ningbo, China

    Tingting Liu

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Verkhlyutov, V., Vvedensky, V., Gurtovoy, K., Burlakov, E., Martynova, O. (2024). Speech Recognition from MEG Data Using Covariance Filters. In: Samsonovich, A.V., Liu, T. (eds) Biologically Inspired Cognitive Architectures 2023. BICA 2023. Studies in Computational Intelligence, vol 1130. Springer, Cham. https://doi.org/10.1007/978-3-031-50381-8_98

Download citation

Publish with us

Policies and ethics

Search

Navigation