Abstract
Reading is a multisensory function that relies on arbitrary associations between auditory speech sounds and symbols from a second modality. Studies of bimodal phonetic perception have mostly investigated the integration of visual letters and speech sounds. Blind readers perform an analogous task by using tactile Braille letters instead of visual letters. The neural underpinnings of audiotactile phonetic processing have not been studied before. We used functional magnetic resonance imaging to reveal the neural correlates of audiotactile phonetic processing in 16 early-blind Braille readers. Braille letters and corresponding speech sounds were presented in unimodal, and congruent/incongruent bimodal configurations. We also used a behavioral task to measure the speed of blind readers in identifying letters presented via tactile and/or auditory modalities. Reaction times for tactile stimuli were faster. The reaction times for bimodal stimuli were equal to those for the slower auditory-only stimuli. fMRI analyses revealed the convergence of unimodal auditory and unimodal tactile responses in areas of the right precentral gyrus and bilateral crus I of the cerebellum. The left and right planum temporale fulfilled the ‘max criterion’ for bimodal integration, but activities of these areas were not sensitive to the phonetical congruency between sounds and Braille letters. Nevertheless, congruency effects were found in regions of frontal lobe and cerebellum. Our findings suggest that, unlike sighted readers who are assumed to have amodal phonetic representations, blind readers probably process letters and sounds separately. We discuss that this distinction might be due to mal-development of multisensory neural circuits in early blinds or it might be due to inherent differences between Braille and print reading mechanisms.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Altarelli I, Leroy F, Monzalvo K et al (2014) Planum temporale asymmetry in developmental dyslexia: revisiting an old question. Hum Brain Mapp. doi:10.1002/hbm.22579
Beauchamp MS (2005) Statistical criteria in FMRI studies of multisensory integration. Neuroinformatics 3:93–113. doi:10.1385/NI:3:2:093
Beauchamp MS, Yasar NE, Frye RE, Ro T (2008) Touch, sound and vision in human superior temporal sulcus. Neuroimage 41:1011–1020. doi:10.1016/j.neuroimage.2008.03.015
Blau V, van Atteveldt N, Ekkebus M et al (2009) Reduced neural integration of letters and speech sounds links phonological and reading deficits in adult dyslexia. Curr Biol 19:503–508. doi:10.1016/j.cub.2009.01.065
Blau V, Reithler J, Van Atteveldt N et al (2010) Deviant processing of letters and speech sounds as proximate cause of reading failure: a functional magnetic resonance imaging study of dyslexic children. Brain 133:868–879. doi:10.1093/brain/awp308
Blomert L (2011) The neural signature of orthographic-phonological binding in successful and failing reading development. Neuroimage 57:695–703. doi:10.1016/j.neuroimage.2010.11.003
Blomert L, Froyen D (2010) Multi-sensory learning and learning to read. Int J Psychophysiol 77:195–204. doi:10.1016/j.ijpsycho.2010.06.025
Brancazio L (2004) Lexical influences in audiovisual speech perception. J Exp Psychol Hum Percept Perform 30:445–463. doi:10.1037/0096-1523.30.3.445
Burr D, Gori M (2012) Multisensory integration develops late in humans. In: Murray M, Wallace M (eds) The neural bases of multisensory processes. CRC Press, Boca Raton, Chapter 18. http://www.ncbi.nlm.nih.gov/books/NBK92864. Accessed 3 Nov 2015
Burton H, Snyder AZ, Diamond JB, Raichle ME (2002) Adaptive changes in early and late blind: a FMRI study of verb generation to heard nouns. J Neurophysiol 88:3359–3371. doi:10.1152/jn.00129.2002
Burton H, Diamond JB, McDermott KB (2003) Dissociating cortical regions activated by semantic and phonological tasks: a FMRI study in blind and sighted people. J Neurophysiol 90:1965–1982. doi:10.1152/jn.00279.2003
Carriere BN, Royal DW, Perrault TJ et al (2007) Visual deprivation alters the development of cortical multisensory integration. J Neurophysiol 98:2858–2867. doi:10.1152/jn.00587.2007
Champoux F, Collignon O, Bacon BA et al (2011) Early- and late-onset blindness both curb audiotactile integration on the parchment-skin illusion. Psychol Sci J Am Psychol Soc/APS 22:19–25. doi:10.1177/0956797610391099
Daneman M (1988) How reading braille is both like and unlike reading print. Mem Cognit 16:497–504. doi:10.3758/BF03197051
Dehaene S, Cohen L (2007) Cultural recycling of cortical maps. Neuron 56:384–398. doi:10.1016/j.neuron.2007.10.004
Dijkstra T, Frauenfelder UH, Schreuder R (1993) Bidirectional grapheme-phoneme activation in a bimodal detection task. J Exp Psychol Hum Percept Perform 19:931–950. doi:10.1037/0096-1523.19.5.931
Doehrmann O, Naumer MJ (2008) Semantics and the multisensory brain: how meaning modulates processes of audio-visual integration. Brain Res 1242:136–150. doi:10.1016/j.brainres.2008.03.071
Erickson KA, Hatton D (2007) Literacy and visual impairment. Semin Speech Lang 28:58–68. doi:10.1055/s-2007-967930
Fishbein HD (1979) Braille-phonics: a new technique for aiding the reading disabled. J Learn Disabil 12:60–64. doi:10.1177/002221947901200111
Foxe JJ, Morocz IA, Murray MM et al (2000) Multisensory auditory–somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Cogn Brain Res 10:77–83. doi:10.1016/S0926-6410(00)00024-0
Foxe JJ, Wylie GR, Martinez A et al (2002) Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. J Neurophysiol 88:540–543. doi:10.1152/jn.00694.2001
Friston KJ, Worsley KJ, Frackowiak RS et al (1994) Assessing the significance of focal activations using their spatial extent. Hum Brain Mapp 1:210–220. doi:10.1002/hbm.460010306
Fulbright RK, Jenner AR, Mencl WE et al (1999) The cerebellum’s role in reading: a functional MR imaging study. Am J Neuroradiol 20:1925–1930. http://www.ajnr.org/content/20/10/1925.long. Accessed 3 Nov 2015
Gaab N, Gabrieli JDE, Glover GH (2007) Assessing the influence of scanner background noise on auditory processing. II. An fMRI study comparing auditory processing in the absence and presence of recorded scanner noise using a sparse design. Hum Brain Mapp 28:721–732. doi:10.1002/hbm.20299
Gick B, Jóhannsdóttir KM, Gibraiel D, Mühlbauer J (2008) Tactile enhancement of auditory and visual speech perception in untrained perceivers. J Acoust Soc Am 123:EL72–EL76. doi:10.1121/1.2884349
Gillon GT (2004) Phonological awareness: from research to practice. Guilford Press, New York
Gillon GT, Young AA (2002) The phonological-awareness skills of children who are blind. J Vis Impair Blind 96:38–49. https://www.afb.org/jvib/jvibabstractnew.asp?articleid=JVIB960103. Accessed 3 Nov 2015
Gizewski ER, Timmann D, Forsting M (2004) Specific cerebellar activation during Braille reading in blind subjects. Hum Brain Mapp 22:229–235. doi:10.1002/hbm.20031
Gizewski ER, Lambertz N, Ladd ME et al (2005) Cerebellar activation patterns in deaf participants for perception of sign language and written text. NeuroReport 16:1913–1917. doi:10.1097/01.wnr.0000186592.41587.3e
Gobbelé R, Schürmann M, Forss N et al (2003) Activation of the human posterior parietal and temporoparietal cortices during audiotactile interaction. Neuroimage 20:503–511. doi:10.1016/S1053-8119(03)00312-4
Gougoux F, Belin P, Voss P et al (2009) Voice perception in blind persons: a functional magnetic resonance imaging study. Neuropsychologia 47:2967–2974. doi:10.1016/j.neuropsychologia.2009.06.027
Greaney J, Reason R (1999) Phonological processing in Braille. Dyslexia 5:215–226. doi:10.1002/(SICI)1099-0909(199912)5:4<215:AID-DYS145>3.0.CO;2-G
Griffiths TD, Warren JD (2002) The planum temporale as a computational hub. Trends Neurosci 25:348–353. doi:10.1016/S0166-2236(02)02191-4
Hagoort P, Baggio G, Willems RM (2009) Semantic unification. In: Gazzaniga MS (ed) The cognitive neurosciences, 4th edn. MIT Press, Cambridge, pp 819–836. http://pubman.mpdl.mpg.de/pubman/faces/viewItemOverviewPage.jsp?itemId=escidoc:64579. Accessed 3 Nov 2015
Hamilton RH, Pascual-Leone A (1998) Cortical plasticity associated with Braille learning. Trends Cogn Sci 2:168–174. doi:10.1016/S1364-6613(98)01172-3
Hein G, Doehrmann O, Müller NG et al (2007) Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas. J Neurosci 27:7881–7887. doi:10.1523/JNEUROSCI.1740-07.2007
Highnam CL, Bleile KM (2011) Language in the cerebellum. Am J Speech Lang Pathol 20:337–347. doi:10.1044/1058-0360(2011/10-0096)
Hoefer M, Tyll S, Kanowski M et al (2013) Tactile stimulation and hemispheric asymmetries modulate auditory perception and neural responses in primary auditory cortex. Neuroimage 79:371–382. doi:10.1016/j.neuroimage.2013.04.119
Hötting K, Röder B (2004) Hearing cheats touch, but less in congenitally blind than in sighted individuals. Psychol Sci 15:60–64. doi:10.1111/j.0963-7214.2004.01501010.x
Hötting K, Rösler F, Röder B (2004) Altered auditory-tactile interactions in congenitally blind humans: an event-related potential study. Exp Brain Res 159:370–381. doi:10.1007/s00221-004-1965-3
Hugdahl K, Heiervang E, Ersland L et al (2003) Significant relation between MR measures of planum temporale area and dichotic processing of syllables in dyslexic children. Neuropsychologia 41:666–675. doi:10.1016/S0028-3932(02)00224-5
Ito T, Ostry DJ (2012) Speech sounds alter facial skin sensation. J Neurophysiol 107:442–447. doi:10.1152/jn.00029.2011
Ito T, Tiede M, Ostry DJ (2009) Somatosensory function in speech perception. Proc Natl Acad Sci U S A 106:1245–1248. doi:10.1073/pnas.0810063106
Jenkinson M, Bannister P, Brady M, Smith S (2002) Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage 17:825–841. doi:10.1006/nimg.2002.1132
Joubert SA, Lecours AR (2000) The role of sublexical graphemic processing in reading. Brain Lang 72:1–13. doi:10.1006/brln.1999.2279
Kassuba T, Klinge C, Hölig C et al (2011) The left fusiform gyrus hosts trisensory representations of manipulable objects. Neuroimage 56:1566–1577. doi:10.1016/j.neuroimage.2011.02.032
Kassuba T, Menz MM, Röder B, Siebner HR (2013) Multisensory interactions between auditory and haptic object recognition. Cereb Cortex 23:1097–1107. doi:10.1093/cercor/bhs076
Kayser C, Petkov CI, Augath M, Logothetis NK (2005) Integration of touch and sound in auditory cortex. Neuron 48:373–384. doi:10.1016/j.neuron.2005.09.018
McCoy LE (1975) Braille: a language for severe dyslexics. J Learn Disabil 8:32–36. doi:10.1177/002221947500800504
McGettigan C, Scott SK (2012) Cortical asymmetries in speech perception: what’s wrong, what’s right and what’s left? Trends Cogn Sci 16:269–276. doi:10.1016/j.tics.2012.04.006
McGurk H, MacDonald J (1976) Hearing lips and seeing voices. Nature 264:746–748. doi:10.1038/264746a0
Millar S (1975) Effects of tactual and phonological similarity on the recall of Braille letters by blind children. Br J Psychol 66:193–201. doi:10.1111/j.2044-8295.1975.tb01455.x
Murray MM, Molholm S, Michel CM et al (2005) Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb Cortex 15:963–974. doi:10.1093/cercor/bhh197
Noppeney U (2012) Characterization of multisensory integration with fMRI: experimental design, statistical analysis, and interpretation. In: Murray M, Wallace M (eds) The neural bases of multisensory processes. CRC Press, Boca Raton, Chapter 13. http://www.ncbi.nlm.nih.gov/books/NBK92833. Accessed 3 Nov 2015
Occelli V, Bruns P, Zampini M, Röder B (2012) Audiotactile integration is reduced in congenital blindness in a spatial ventriloquism task. Neuropsychologia 50:36–43. doi:10.1016/j.neuropsychologia.2011.10.019
Occelli V, Spence C, Zampini M (2013) Auditory, tactile, and audiotactile information processing following visual deprivation. Psychol Bull 139:189–212. doi:10.1037/a0028416
Price CJ (2012) A review and synthesis of the first 20 years of PET and fMRI studies of heard speech, spoken language and reading. Neuroimage 62:816–847. doi:10.1016/j.neuroimage.2012.04.062
Pring L (1982) Phonological and tactual coding of Braille by blind children. Br J Psychol 73(Pt 3):351–359. doi:10.1111/j.2044-8295.1982.tb01817.x
Raczkowski D, Kalat JW, Nebes R (1974) Reliability and validity of some handedness questionnaire items. Neuropsychologia 12:43–47. doi:10.1016/0028-3932(74)90025-6
Raij T, Uutela K, Hari R (2000) Audiovisual integration of letters in the human brain. Neuron 28:617–625. doi:10.1016/S0896-6273(00)00138-0
Ramus F (2004) The neural basis of reading acquisition. In: Gazzaniga MS (ed) The cognitive neurosciences, 3rd edn. MIT Press, Cambridge, pp 815–824. http://www.researchgate.net/profile/Franck_Ramus/publication/2841234_The_Neural_Basis_of_Reading_Acquisition/links/546caaba0cf2a80cf2e0ebb4.pdf. Accessed 3 Nov 2015
Rauschecker JP, Tian B (2000) Mechanisms and streams for processing of “what” and “where” in auditory cortex. Proc Natl Acad Sci USA 97:11800–11806. doi:10.1073/pnas.97.22.11800
Reed CM, Rubin SI, Braida LD, Durlach NI (1978) Analytic study of the tadoma method: discrimination ability of untrained observers. J Speech Lang Hear Res 21:625–637. doi:10.1044/jshr.2104.625
Sadato N (2005) How the blind “see” Braille: lessons from functional magnetic resonance imaging. Neuroscientist 11:577–582. doi:10.1177/1073858405277314
Sadato N, Pascual-Leone A, Grafman J et al (1996) Activation of the primary visual cortex by Braille reading in blind subjects. Nature 380:526–528. doi:10.1038/380526a0
Sadato N, Pascual-Leone A, Grafman J et al (1998) Neural networks for Braille reading by the blind. Brain 121:1213–1229. doi:10.1093/brain/121.7.1213
Sato M, Cavé C, Ménard L, Brasseur A (2010) Auditory-tactile speech perception in congenitally blind and sighted adults. Neuropsychologia 48:3683–3686. doi:10.1016/j.neuropsychologia.2010.08.017
Scheithauer MC, Tiger JH (2012) A computer-based program to teach braille reading to sighted individuals. J Appl Behav Anal 45:315–327. doi:10.1901/jaba.2012.45-315
Schneider TR, Lorenz S, Senkowski D, Engel AK (2011) Gamma-band activity as a signature for cross-modal priming of auditory object recognition by active haptic exploration. J Neurosci 31:2502–2510. doi:10.1523/JNEUROSCI.6447-09.2011
Schröger E, Widmann A, Schroger E, Widmann A (1998) Speeded responses to audiovisual signal changes result from bimodal integration. Psychophysiology 35:755–759. doi:10.1111/1469-8986.3560755
Schürmann M, Caetano G, Jousmäki V, Hari R (2004) Hands help hearing: facilitatory audiotactile interaction at low sound-intensity levels. J Acoust Soc Am 115:830. doi:10.1121/1.1639909
Schürmann M, Caetano G, Hlushchuk Y et al (2006) Touch activates human auditory cortex. Neuroimage 30:1325–1331. doi:10.1016/j.neuroimage.2005.11.020
Shah NJ, Steinhoff S, Mirzazade S et al (2000) The effect of sequence repeat time on auditory cortex stimulation during phonetic discrimination. Neuroimage 12:100–108. doi:10.1006/nimg.2000.0588
Smith SM (2002) Fast robust automated brain extraction. Hum Brain Mapp 17:143–155. doi:10.1002/hbm.10062
Specht K (2014) Neuronal basis of speech comprehension. Hear Res 307:121–135. doi:10.1016/j.heares.2013.09.011
Stoodley CJ, Stein JF (2011) The cerebellum and dyslexia. Cortex 47:101–116. doi:10.1016/j.cortex.2009.10.005
Teshiba TM, Ling J, Ruhl DA et al (2013) Evoked and intrinsic asymmetries during auditory attention: implications for the contralateral and neglect models of functioning. Cereb Cortex 23:560–569. doi:10.1093/cercor/bhs039
Ulrich R, Giray M (1986) Separate-activation models with variable base times: testability and checking of cross-channel dependency. Percept Psychophys 39:248–254. doi:10.3758/BF03204931
Van Atteveldt N, Formisano E, Goebel R, Blomert L (2004) Integration of letters and speech sounds in the human brain. Neuron 43:271–282. doi:10.1016/j.neuron.2004.06.025
Van Atteveldt NM, Formisano E, Blomert L, Goebel R (2007a) The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cereb Cortex 17:962–974. doi:10.1093/cercor/bhl007
Van Atteveldt NM, Formisano E, Goebel R, Blomert L (2007b) Top-down task effects overrule automatic multisensory responses to letter-sound pairs in auditory association cortex. Neuroimage 36:1345–1360. doi:10.1016/j.neuroimage.2007.03.065
Van Atteveldt N, Roebroeck A, Goebel R (2009) Interaction of speech and script in human auditory cortex: insights from neuro-imaging and effective connectivity. Hear Res 258:152–164. doi:10.1016/j.heares.2009.05.007
Veispak A, Boets B, Ghesquière P (2012a) Differential cognitive and perceptual correlates of print reading versus braille reading. Res Dev Disabil 34:372–385. doi:10.1016/j.ridd.2012.08.012
Veispak A, Boets B, Ghesquière P (2012b) Parallel versus sequential processing in print and braille reading. Res Dev Disabil 33:2153–2163. doi:10.1016/j.ridd.2012.06.012
Wallace MT, Perrault TJ, Hairston WD, Stein BE (2004) Visual experience is necessary for the development of multisensory integration. J Neurosci 24:9580–9584. doi:10.1523/JNEUROSCI.2535-04.2004
Willems RM, Ozyürek A, Hagoort P (2009) Differential roles for left inferior frontal and superior temporal cortex in multimodal integration of action and language. Neuroimage 47:1992–2004. doi:10.1016/j.neuroimage.2009.05.066
Woo C-W, Krishnan A, Wager TD (2014) Cluster-extent based thresholding in fMRI analyses: pitfalls and recommendations. Neuroimage 91:412–419. doi:10.1016/j.neuroimage.2013.12.058
Xiang HD, Fonteijn HM, Norris DG, Hagoort P (2010) Topographical functional connectivity pattern in the perisylvian language networks. Cereb Cortex 20:549–560. doi:10.1093/cercor/bhp119
Ziegler JC, Goswami U (2005) Reading acquisition, developmental dyslexia, and skilled reading across languages: a psycholinguistic grain size theory. Psychol Bull 131:3–29. doi:10.1037/0033-2909.131.1.3
Acknowledgments
We would like to thank the blind participants for their cooperation and patience. We also acknowledge the contributions of Ben Krasnow for design and construction of the Braille device (http://www.magconcept.com/MRI), Mr. Rafi’ee for operating the MR scanner, and Dr. Rouhbakhsh for his insightful ideas in early conceptualization of the current study. We also would like to thank two anonymous reviewers for their thoughtful comments and the suggestion of gathering behavioral reaction time data. This research was supported by a grant from Tehran University of Medical Sciences (Project No. 87-04-30-8111).
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Pishnamazi, M., Nojaba, Y., Ganjgahi, H. et al. Neural correlates of audiotactile phonetic processing in early-blind readers: an fMRI study. Exp Brain Res 234, 1263–1277 (2016). https://doi.org/10.1007/s00221-015-4515-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-015-4515-2