Trends in Cognitive Sciences
ReviewThe emergence of multisensory systems through perceptual narrowing
Section snippets
The world is multisensory
The objects and events that make up our everyday experience provide us with a constant flow of sensory signals in multiple modalities. Although such inputs can potentially create confusion, our ability to integrate multisensory information enables us to have coherent and meaningful perceptual experiences. Talking faces, for example, are typically specified by various spatiotemporally congruent and modality-specific attributes as well as a host of invariant amodal attributes. Modality-specific
The developmental problem and the progressive framework
Because the world is multisensory, a developing infant's task is to discover the multisensory coherence of the objects and events that constitute the infant's normal ecology. This task is difficult because the infant has an immature nervous system and is perceptually inexperienced. Nonetheless, infants gradually overcome these limitations and become capable of detecting multisensory coherence 8, 9. They are able to overcome these limitations for two reasons. First, multisensory coherence is
The progressive framework is not the whole story: perceptual narrowing
Although the evidence consistent with the progressive theoretical framework is unquestionable, other evidence suggests that various human perceptual functions undergo developmental narrowing in early life and that this is crucial for the eventual development of species-specific patterns of perceptual expertise (see Box 1 for historical note). This evidence comes from studies of speech, face and music perception, and shows that, initially, perceptual tuning is so broad that it allows young
Multisensory perceptual narrowing
If multisensory perception is the default mode of perception 40, 41 then perceptual narrowing might be a pan-sensory process. If it is, then the progressive developmental theoretical framework does not adequately account for the development of multisensory perception. The first hint that this might be so came from a study that pre-dates the various findings of narrowing in the unisensory domain. This study showed that whereas 3-week-old infants made spontaneous intensity-based audiovisual (A-V)
Is perceptual narrowing unique to humans?
The evidence for multisensory perceptual narrowing raises questions about the evolution of this developmental mechanism. Naturally, the only way to get at this issue is through comparative studies with non-human primates (hereafter, primates). Given that both humans and other extant primates use both facial and vocal expressions as communication signals, it is perhaps not surprising that many primates recognize the correspondence between the visual and auditory components of vocal signals.
Conclusions
Experience can have complementary effects on the development of perceptual functions. On the one hand, it can induce and facilitate the emergence of a particular perceptual function and it can maintain that function through continued exposure to specific sensory inputs. On the other, it also can have the seemingly opposite but, in reality, complementary effect. That is, as experience with native sensory input accumulates, the scope of initially broad perceptual abilities is narrowed to best
Acknowledgements
This work was supported by National Science Foundation grants to DJL (BCS-0751888) and AAG (BCS-0547760 CAREER award). We thank Robert Seyfarth for pointing out examples of developmental narrowing in monkeys.
References (81)
The magnificent compromise: Cortical field evolution in mammals
Neuron
(2007)Heterogeneity and heterochrony in the development of intersensory perception
Cogn. Brain Res.
(2002)- et al.
Amodal representations of speech in infants
Infant Behav. & Deve.
(1993) - et al.
Infants’ ability to match dynamic phonetic and gender information in the face and voice
J. Exp. Child Psychol.
(2002) - et al.
Cross-language speech perception: Evidence for perceptual reorganization during the first year of life
Infant Behav. & Dev.
(1984) Divergent developmental patterns for infants’ perception of two nonnative consonant contrasts
Infant Behav. & Dev.
(1995)Development of the other-race effect during infancy: evidence toward universality?
J. Exp. Child Psychol.
(2009)The decline of cross-species intersensory perception in human infants: Underlying mechanisms and its developmental persistence
Brain Res.
(2008)Monkeys match the number of voices they hear to the number of faces they see
Curr. Biol.
(2005)Vocal-tract resonances as indexical cues in rhesus monkeys
Curr. Biol.
(2007)
Vocal development in vervet monkeys
Anim. Behav.
Face processing limitation to own species in primates: A comparative study in brown capuchins, Tonkean macaques and humans
Behav. Processes
The natural statistics of audiovisual speech
PLoS Comput. Biol.
Spatial and Temporal Constraints on Audiovisual Speech Perception
Intersensory Redundancy Guides the Development of Selective Attention, Perception, and Cognition in Infancy
Curr. Directions Psychol. Sci.
The value of multimodal redundancy in the development of intersensory perception
The cortex in multidimensional space: where do cortical areas come from? Commentary
Dev. Sci.
The ontogeny of information
The development of infant intersensory perception: advantages of a comparative convergent-operations approach
Psychol. Bull.
Principles of perceptual learning and development
Limitations on input as a basis for neural organization and perceptual development: A preliminary theoretical statement
Dev. Psychobiol.
Visual Differentiation, Intersensory Integration, and Voluntary Motor Control
Monogr. Soc. Res. Child Dev.
The origins of intelligence in children
Perceptual development from the ecological approach
The bimodal perception of speech in infancy
Science
Two-month-old infants match phonetic information in lips and voice
Dev. Sci.
The role of person familiarity in young infants’ perception of emotional expressions
Child Dev.
Intermodal perception of expressive behaviors: Relation of eye and voice?
Dev. Psy.
Sleeper effects
Dev. Sci.
Visual activity and cortical rewiring: activity-dependent plasticity of cortical networks
Prog. Brain Res.
Early experience determines how the senses will interact
J. Neurophysiol.
Visual influences on auditory spatial learning
Philos. Trans. R. Soc. Lond. B Biol. Sci
Linguistic experience alters phonetic perception in infants by 6 months of age
Science
Development of language-specific phoneme representations in the infant brain
Nature Neurosci.
Foreign-language experience in infancy: effects of short-term exposure and social interaction on phonetic learning
Proc. Natl. Acad. Sci. U.S.A.
Is face processing species-specific during the first year of life?
Science
Plasticity of face processing in infancy
Proc. Natl. Acad. Sci. U.S.A.
An investigation of the contact hypothesis of the own-race bias in face recognition
Q. J. Exp. Psychol. A: Human Experimental Psychology
The Other-Race Effect Develops During Infancy: Evidence of Perceptual Narrowing
Psychol. Sci
Recognition of own-race and other-race faces by three-month-old infants
J. Child Psychol. Psychiatr.
Cited by (221)
A multisensory Interaction Framework for Human-Cyber–Physical System based on Graph Convolutional Networks
2024, Advanced Engineering InformaticsThe development of the relationship between auditory and visual neural sensitivity and autonomic arousal from 6 m to 12 m
2023, Developmental Cognitive NeuroscienceDevelopment and experience-dependence of multisensory spatial processing
2023, Trends in Cognitive SciencesThe discrimination of expressions in facial movements by infants: A study with point-light displays
2023, Journal of Experimental Child Psychology