Abstract
Neuroimaging studies have demonstrated that mental imagery and perception share similar neural substrates, however, there are still ambiguities according to different auditory imagery content. In addition, there is still a lack of information regarding the underlying neural correlation between the two modalities. In the present study, we adopted functional magnetic resonance imaging to explore the neural representation during imagery and perception of actual sounds in our surroundings. Univariate analysis was used to assess the differences between the modalities of average activation intensity, and stronger imagery activation was found in sensorimotor regions but weaker activation in auditory association cortices. Additionally, multi-voxel pattern analysis with a support vector machine classifier was implemented to decode environmental sounds within- or cross-modality. Significant above-chance accuracies were found in all overlapping regions in the classification of within-modality, while successful cross-modality classification only was found in sensorimotor regions. Both univariate and multivariate analyses found distinct representation between auditory imagery and perception in the overlapping regions, including superior temporal gyrus and inferior frontal sulcus as well as the precentral cortex and pre-supplementary motor area. Our results confirm the overlapping activation regions between auditory imagery and perception reported by previous studies and suggest that activation regions showed dissociable representation pattern in imagery and perception of sound categories.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Addis DR, Pan L, Vu MA, Laiser N, Schacter DL (2009) Constructive episodic simulation of the future and the past: distinct subsystems of a core brain network mediate imagining and remembering. Neuropsychologia 47(11):2222–2238. https://doi.org/10.1016/j.neuropsychologia.2008.10.026
Albers AM, Kok P, Toni I, Dijkerman HC, de Lange FP (2013) Shared representations for working memory and mental imagery in early visual cortex. Curr Biol 23(15):1427–1431. https://doi.org/10.1016/j.cub.2013.05.065
Amaro E, Barker GJ (2006) Study design in fMRI: basic principles. Brain Cogn 60(3):220–232
Amunts K, Morosan P, Hilbig H, Zilles K (2012) Auditory system. Hum Nerv Syst. https://doi.org/10.1016/b978-0-12-374236-0.10036-7
Ariani G, Oosterhof NN, Lingnau A (2018) Time-resolved decoding of planned delayed and immediate prehension movements. Cortex 99:330–345. https://doi.org/10.1016/j.cortex.2017.12.007
Arnott SR, Grady CL, Hevenor SJ, Graham S, Alain C (2005) The functional organization of auditory working memory as revealed by fMRI. J Cogn Neurosci 17(5):819–831. https://doi.org/10.1162/0898929053747612
Boccia M, Sulpizio V, Palermo L, Piccardi L, Guariglia C, Galati G (2017) I can see where you would be: patterns of fMRI activity reveal imagined landmarks. NeuroImage 144:174–182
Bunzeck N, Wuestenberg T, Lutz K, Heinze HJ, Jancke L (2005) Scanning silence: mental imagery of complex sounds. NeuroImage 26(4):1119–1127. https://doi.org/10.1016/j.neuroimage.2005.03.013
Cichy RM, Heinzle J, Haynes JD (2012) Imagery and perception share cortical representations of content and location. Cereb Cortex 22:372–380
Copolov DL, Seal ML, Maruff P, Ulusoy R, Wong MT, Tochon-Danguy HJ, Egan GF (2003) Cortical activation associated with the experience of auditory hallucinations and perception of human speech in schizophrenia: a PET correlation study. Psychiatry Res 122(3):139–152
Davis T, LaRocque KF, Mumford JA, Norman KA, Wagner AD, Poldrack RA (2014) What do differences between multi-voxel and univariate analysis mean? How subject-, voxel-, and trial-level variance impact fMRI analysis. NeuroImage 97:271–283
de Borst AW, de Gelder B (2018) Mental imagery follows similar cortical reorganization as perception: intra-modal and cross-modal plasticity in congenitally blind. Cereb Cortex. https://doi.org/10.1093/cercor/bhy151
de Borst AW, Valente G, Jaaskelainen IP, Tikka P (2016) Brain-based decoding of mentally imagined film clips and sounds reveals experience-based information patterns in film professionals. NeuroImage 129:428–438. https://doi.org/10.1016/j.neuroimage.2016.01.043
Dierks T, Linden DE, Jandl M, Formisano E, Goebel R, Lanfermann H, Singer W (1999) Activation of Heschl’s gyrus during auditory hallucinations. Neuron 22(3):615–621
Emmerling TC, Zimmermann J, Sorger B, Frost MA, Goebel R (2016) Decoding the direction of imagined visual motion using 7T ultra-high field fMRI. NeuroImage 125:61–73. https://doi.org/10.1016/j.neuroimage.2015.10.022
Engel LR, Frum C, Puce A, Walker NA, Lewis JW (2009) Different categories of living and non-living sound-sources activate distinct cortical networks. NeuroImage 47:1778–1791
Ganis G, Thompson WL, Kosslyn SM (2004) Brain areas underlying visual mental imagery and visual perception: an fMRI study. Brain Res Cogn Brain Res 20(2):226–241. https://doi.org/10.1016/j.cogbrainres.2004.02.012
Gilbert SJ, Fung H (2018) Decoding intentions of self and others from fMRI activity patterns. NeuroImage 172:278–290. https://doi.org/10.1016/j.neuroimage.2017.12.090
Halpern AR, Zatorre RJ (1999) When that tune runs through your head: a pet investigation of auditory imagery for familiar melodies [J]. Cereb Cortex 9(7):697–704
Halpern AR, Zatorre RJ, Bouffard M, Johnson JA (2004) Behavioral and neural correlates of perceived and imagined musical timbre. Neuropsychologia 42(9):1281–1292. https://doi.org/10.1016/j.neuropsychologia.2003.12.017
Hampshire A, Chamberlain SR, Monti MM, Duncan J, Owen AM (2010) The role of the right inferior frontal gyrus: inhibition and attentional control. NeuroImage 20:1313–1319
Herholz SC, Halpern AR, Zatorre RJ (2012) Neuronal correlates of perception, imagery, and memory for familiar tunes. J Cogn Neurosci 24(6):1382–1397. https://doi.org/10.1162/jocn_a_00216
Hickok G, Poeppel D (2007) The cortical organization of speech processing. Nat Rev Neurosci 8(5):393
Huang S, Seidman LJ, Rossi S, Ahveninen J (2013) Distinct cortical networks activated by auditory attention and working memory load. NeuroImage 83:1098–1108. https://doi.org/10.1016/j.neuroimage.2013.07.074
Hubbard TL (2010) Auditory imagery: empirical findings. Psychol Bull 136(2):302–329. https://doi.org/10.1037/a0018436
Huijbers W, Pennartz CM, Rubin DC, Daselaar SM (2011) Imagery and retrieval of auditory and visual information: neural correlates of successful and unsuccessful performance. Neuropsychologia 49(7):1730–1740. https://doi.org/10.1016/j.neuropsychologia.2011.02.051
Jäncke Lutz, Shah NJ (2004) ‘Hearing’ syllables by ‘seeing’ visual stimuli. Eur J Neurosci 19(9):2603–2608
Jardri R, Pouchet A, Pins D, Thomas P (2011) Cortical activations during auditory verbal hallucinations in schizophrenia: a coordinate-based meta-analysis. Am J Psychiatry 168(1):73–81. https://doi.org/10.1176/appi.ajp.2010.09101522
Koelsch S, Schulze K, Sammler D, Fritz T, Muller K, Gruber O (2009) Functional architecture of verbal and tonal working memory: an FMRI study. Hum Brain Mapp 30(3):859–873. https://doi.org/10.1002/hbm.20550
Kompus K, Westerhausen R, Hugdahl K (2011) The “paradoxical” engagement of the primary auditory cortex in patients with auditory verbal hallucinations: a meta-analysis of functional neuroimaging studies. Neuropsychologia 49(12):3361–3369. https://doi.org/10.1016/j.neuropsychologia.2011.08.010
Kraemer DJ, Macrae CN, Green AE, Kelley WM (2005) Musical imagery: sound of silence activates auditory cortex. Nature 434(7030):158
Kriegeskorte N, Simmons WK, Bellgowan PS, Baker CI (2009) Circular analysis in systems neuroscience: the dangers of double dipping. Nat Neurosci 12(5):535–540. https://doi.org/10.1038/nn.2303
Kumar S, Sedley W, Barnes GR, Teki S, Friston KJ, Griffiths TD (2014) A brain basis for musical hallucinations. Cortex 52:86–97. https://doi.org/10.1016/j.cortex.2013.12.002
Le Bihan D, Turner R, Zeffiro TA, Cuenod CA, Jezzard P, Bonnerot V (1993) Activation of human primary visual cortex during visual recall: a magnetic resonance imaging study. Proc Natl Acad Sci USA 90(24):11802–11805
Lee SH, Kravitz DJ, Baker CI (2012) Disentangling visual imagery and perception of real-world objects. NeuroImage 59(4):4064–4073
Leung AW, Alain C (2011) Working memory load modulates the auditory “What” and “Where” neural networks. NeuroImage 55(3):1260–1269. https://doi.org/10.1016/j.neuroimage.2010.12.055
Lewis JW, Brefczynski JA, Phinney RE, Janik JJ, DeYoe EA (2005) Distinct cortical pathways for processing tool versus animal sounds. J Neurosci 25:5148–5158
Lima CF, Lavan N, Evans S (2015) Feel the noise: relating individual differences in auditory imagery to the structure and function of sensorimotor systems. Cereb Cortex 25:4638–4650
Lima CF, Krishnan S, Scott SK (2016) Roles of supplementary motor areas in auditory processing and auditory imagery. Trends Neurosci 39(8):527–542. https://doi.org/10.1016/j.tins.2016.06.003
Linke AC, Cusack R (2015) Flexible information coding in human auditory cortex during perception, imagery, and STM of complex sounds. J Cogn Neurosci 27(7):1322–1333. https://doi.org/10.1162/jocn_a_00780
Maeder PP, Meuli RA, Adriani M, Bellmann A, Fornari E, Thiran JP, Pittet A, Clarke S (2001) Distinct pathways involved in sound recognition and localization: a human fMRI study. NeuroImage 14:802–816
Mahmoudi A, Takerkart S, Regragui F, Boussaoud D, Brovelli A (2012) Multivoxel pattern analysis for FMRI data: a review. Comput Math Methods Med 2012:1–14
McGettigan C, Walsh E, Jessop R, Agnew ZK, Sauter DA, Warren JE, Scott SK (2015) Individual differences in laughter perception reveal roles for mentalizing and sensorimotor systems in the evaluation of emotional authenticity. Cereb Cortex 25:246–257
Meteyard L, Cuadrado SR, Bahrami B, Vigliocco G (2012) Coming of age: a review of embodiment and the neuroscience of semantics. Cortex 48:788–804
Norman KA, Polyn SM, Detre GJ, Haxby JV (2006) Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cogn Sci 10:424–430
Oh J, Kwon JH, Yang PS, Jeong J (2013) Auditory imagery modulates frequency-specific areas in the human auditory cortex. J Cogn Neurosci 25(2):175–187. https://doi.org/10.1162/jocn_a_00280
Oldfield RC (1971) The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9(1):97–113
Raij TT, Riekki TJ (2012) Poor supplementary motor area activation differentiates auditory verbal hallucination from imagining the hallucination. Neuroimage Clin 1(1):75–80
Rauschecker JP, Scott SK (2009) Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing. Nat Neurosci 12(6):718–724. https://doi.org/10.1038/nn.2331
Shergill SS, Bullmore ET, Brammer MJ (2001) A functional study of auditory verbal imagery. Psychol Med 31(2):241. https://doi.org/10.1017/S003329170100335X
Stelzer J, Chen Y, Turner R (2013) Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (MVPA): random permutations and cluster size control. NeuroImage 65:69–82. https://doi.org/10.1016/j.neuroimage.2012.09.063
Stuart GP, Jones DM (1996) From auditory image to auditory percept: facilitation through common processes? Mem Cognit 24:296–304
Tian B, Reser D, Amy D (2001) Functional specialization in rhesus monkey auditory cortex. Science 292(5515):290–293
Tian X, Zarate JM, Poeppel D (2016) Mental imagery of speech implicates two mechanisms of perceptual reactivation. Cortex 77:1–12. https://doi.org/10.1016/j.cortex.2016.01.002
Tong Y, Chen Q, Nichols TE, Rasetti R, Callicott JH, Berman KF, Weinberger DR, Mattay VS (2016) Seeking optimal region-of-interest (ROI) single-value summary measures for fmri studies in imaging genetics. PLoS One 11:e151391
Treder MS, Purwins H, Miklody D, Sturm I, Blankertz B (2014) Decoding auditory attention to instruments in polyphonic music using single-trial EEG classification. J Neural Eng 11(2):0226009
van de Ven VG, Formisano E, Roder CH, Prvulovic D, Bittner RA, Dietz MG, Hubl D, Dierks T, Federspiel A, Esposito F, Di Salle F, Jansma B, Goebel R, Linden DE (2005) The spatiotemporal pattern of auditory cortical responses during verbal hallucinations. NeuroImage 27(3):644–655. https://doi.org/10.1016/j.neuroimage.2005.04.041
Vetter P, Smith FW, Muckli L (2014) Decoding sound and imagery content in early visual cortex. Curr Biol 24(11):1256–1262. https://doi.org/10.1016/j.cub.2014.04.020
Wang X, Gu J, Xu J, Li X, Geng J, Wang B, Liu B (2018) Decoding natural scenes based on sounds of objects within scenes using multivariate pattern analysis. Neurosci Res. https://doi.org/10.1016/j.neures.2018.11.009
Wheeler ME, Petersen SE, Buckner RL (2000) Memory’s echo: Vivid remembering reactivates sensory-specific cortex. Proc Natl Acad Sci India Sect A (Phys Sci) 97(20):11125–11129
Willander J, Baraldi S (2010) Development of a new Clarity of Auditory Imagery Scale. Behav Res Methods 42(3):785–790
Wong D, Pisoni DB, Learn J (2002) PET imaging of differential cortical activation by monaural speech and nonspeech stimuli. Hear Res 166(1):9–23
Wu J, Mai X, Chan CC, Zheng Y, Luo Y (2006) Event-related potentials during mental imagery of animal sounds. Psychophysiology 43:592–597
Wu J, Yu Z, Mai X, Wei J, Luo Y (2011) Pitch and loudness information encoded in auditory imagery as revealed by event-related potentials. Psychophysiology 48(3):415–419. https://doi.org/10.1111/j.1469-8986.2010.01070.x
Yoo SS, Lee CU, Choi BG (2001) Human brain mapping of auditory imagery: event-related functional MRI study. NeuroReport 12(14):3045–3049
Yoo SS, Freeman DK, McCarthy JJ, Jolesz FA (2003) Neural substrates of tactile imagery: a functional MRI study. NeuroReport 14(4):581–585
Zafar R, Malik AS, Kamel N, Dass SC, Abdullah JM, Reza F, Abdul KA (2015) Decoding of visual information from human brain activity: a review of fMRI and EEG studies. J Integra Neurosci 14:155–168
Zald DH, Pardo JV (2002) The neural correlates of aversive auditory stimulation. NeuroImage 16:746–753
Zatorre RJ, Evans AC, Meyer E (1994) Neural mechanisms underlying melodic perception and memory for pitch. J Neurosci Methods 14:1908–1919
Zeidman P, Mullally SL, Maguire EA (2015) Constructing, perceiving, and maintaining scenes: hippocampal activity and connectivity. Cereb Cortex 25(10):3836–3855. https://doi.org/10.1093/cercor/bhu266
Zmigrod L, Garrison JR, Carr J, Simons JS (2016) The neural mechanisms of hallucinations: a quantitative meta-analysis of neuroimaging studies. Neurosci Biobehav Rev 69:113–123. https://doi.org/10.1016/j.neubiorev.2016.05.037
Acknowledgements
This work was supported by the National Key Research and Development Program of China (no. 2018YFB0204304), National Natural Science Foundation of China (no. U1736219 and no. 61571327), Shandong Provincial Natural Science Foundation of China (no. ZR2015HM081) and Project of Shandong Province Higher Educational Science and Technology Program (J15LL01).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This study was approved by the Research Ethics Committee of Yantai Affiliated Hospital of Binzhou Medical University. All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, and the applicable revisions at the time of the investigation.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Gu, J., Zhang, H., Liu, B. et al. An investigation of the neural association between auditory imagery and perception of complex sounds. Brain Struct Funct 224, 2925–2937 (2019). https://doi.org/10.1007/s00429-019-01948-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00429-019-01948-z