Special issue: ViewpointAction-based language: A theory of language acquisition, comprehension, and production
Introduction
The nature of language and the evolutionary process producing it are still matters of debate. This is partly due to the complexity and multidimensional nature of language. What do we refer to when we speak about the language faculty? Viewing cognition as an embodied, situated, and social enterprise offers the possibility of a new approach. This view of language and cognition has important philosophical antecedents, especially in the phenomenological tradition (see Gallese, 2007, 2008), which argues that meaning does not inhabit a pre-given Platonic world of ideal and eternal truths to which mental representations connect and conform. Instead, phenomenology entertains a perspective compatible with many empirical results of contemporary cognitive neuroscience: meaning is the outcome of our situated interactions with the world.
With the advent of language, the notion of meaning changes, as it is amplified by freeing itself from being dependent upon specific instantiations of actual experience. Language affords the opportunity to connect all possible actions within a network, thereby expanding the meaning of individual situated experiences. Language does this by evoking the near totality of possibilities for action the world presents us, and by structuring those actions within a web of related meanings. By endorsing this perspective, it follows that if we confine language solely to its predicative use, we inappropriately reify just one part of language’s nature. Instead, our understanding of linguistic expressions is not solely an epistemic attitude; it is first and foremost a pragmatic attitude directed toward action.
Data from psychology, psycholinguistics, and neuroscience have demonstrated the importance of action systems to perception (Wilson and Knoblich, 2005), social processes such as mentalizing (Gallese and Goldman, 1998, Gallese, 2003a, Sommerville and Decety, 2006), and to language comprehension (Glenberg and Robertson, 1999, Pulvermüller, 2002, Pulvermüller, 2005, Pulvermüller, 2008, Gallese, 2007, 2008). The action-related account of language suggests that the neuroscientific investigation of what language is and how it works should begin from the domain of action. However, no formal theories of this interaction have been proposed. Here we adapt well-tested theories of motor control, the modular selection and identification for control (MOSAIC) and hierarchical modular selection and identification for control (HMOSAIC) theories (Haruno et al., 2003) to produce our theory of action-based language (ABL). We apply the theory to language acquisition, comprehension, and some aspects of production, namely gesture.
We begin with a brief review of recent work on the relation between language and action (for more complete reviews see Gallese, 2007, 2008; Glenberg, 2007, Pulvermüller, 2005, Rizzolatti and Craighero, 2007) and the neurophysiology of the connection between language and action. This review is followed by a description of the MOSAIC and HMOSAIC models and how we modify them to apply to language phenomena. In brief, we propose that (a) the neural networks underlying the HMOSAIC model include pre-motor mirror neurons and canonical neurons, and (b) Hebbian learning underlies the association of neural networks used in speech production (e.g., uttering the word “give”) and action control (e.g., the act of giving) so that the meaning of the utterance is grounded in the action and the expected outcome of the action. We then discuss how the model applies to the acquisition of nouns, verbs, and syntactic structures, how it explains simple language comprehension, and we apply the model to gesture as one component of language production.
One caveat is important. Whereas we focus on the relation between language and action, we do not claim that all language phenomena can be accommodated by action systems. Even within an embodied approach to language, there is strong evidence for contributions to language comprehension by perceptual systems (e.g., Pulvermüller, 2002, Kaschak et al., 2005, Pecher et al., 2003, Saygin et al., 2010) and emotional systems (Havas et al., 2010, Havas et al., 2007), and we address some of this work in the discussion. Our primary goal, however, is to make progress in understanding what appear to be major contributions of action to language.
Section snippets
Language and action
At first blush, action would seem to have little in common with language (conceived as a cognitive, modular system). Nonetheless, strong links between language and action have been found in analyses based on evolution (e.g., Gentilucci and Corballis, 2006, Rizzolatti and Arbib, 1998), neurophysiology (see Rizzolatti and Craighero, 2007, for a review), and behavior. Here we focus on behavioral data and briefly touch on other approaches.
The Indexical Hypothesis (Glenberg and Robertson, 1999)
Neurophysiology and the language-action connection
The neurophysiological basis for the language-based modulation of the motor system is most likely related to the properties of a set of neurons, the so-called mirror neurons, first discovered in area F5 of the monkey pre-motor cortex (Gallese et al., 1996, Rizzolatti et al., 1996). Here we review properties of mirror neurons [(and mirror neuron systems or mirror mechanisms – MMs)] that are relevant to the development of the ABL theory described later.
Mirror neurons discharge when the animal
The MOSAIC and HMOSAIC theories of action control
In this section we describe a theory of motor control developed by Wolpert and colleagues (e.g., Wolpert et al., 2003). After describing the theory in some detail, we take two novel steps. The first is to relate some operations of the theory to mirror and canonical neurons. We then discuss how a simple modification of this model can link it to language processes.
Two types of models are often invoked in theories of motor control. A controller (also referred to as a backward or inverse model)
Linking HMOSAIC to language
In this section we develop the HMOSAIC model so that it becomes a model of hierarchical control in language as well as action production, that is, the ABL model. This development is followed by a discussion of the application of ABL to selective aspects of language acquisition (learning nouns, verbs, and multi-word constructions), comprehension, and production (gesture, in particular). Each of these sections includes a description of how the ABL model applies and a brief review of supporting
Learning linguistic constructions
In this section we consider how the ABL theory can provide a unique account of several components of language acquisition. We begin with a consideration of the learning of nouns and verbs. We then demonstrate how the theory accounts for learning syntactic-like constructions such as the double-object construction that describes transfer events, such as “Give the horse [object 1] the apple [object 2]”.
Consider how an infant can associate a verbal label for an object (e.g., “bottle”) with the
Comprehension: using the motor system to guide simulation
A number of researchers have proposed that language comprehension is a process of simulation (e.g., Barsalou, 1999), and that the simulation makes use of the motor system (Gallese, 2007, Gallese, 2008; Gallese and Lakoff, 2005, Glenberg and Robertson, 2000, Zwaan and Taylor, 2006). Here we provide an example of how the ABL theory produces such a simulation, and exactly what it is about that simulation that counts as language comprehension. Consider the understanding of a sentence read to a
Gesture as one component of language production
The ABL model provides insight into several features of language production such as syntactic priming (Chang et al., 2006), interactive alignment in dialog (Pickering and Garrod, 2004), and gesture. Due to space constraints, here we only consider production of co-speech gesture, and even more specifically, representational gestures (McNeill, 1992). These gestures pertain to speech content by pointing to objects, by depicting with the hands object shapes and motion paths, and by using particular
Discussion
This discussion briefly addresses three issues that would seem to present difficulties for the ABL theory. The first is that any account of language centered on the motor system is already discredited by arguments against Behaviorism. The second issue is that language must take into account processes that are not motoric, and in particular that the representation of space plays a critical role in cognitive development (e.g., Mandler, 2008). Finally, we briefly address how a sensorimotor account
Author note
Arthur Glenberg, Department of Psychology, Arizona State University ([email protected]); Vittorio Gallese, Department of Neuroscience, University of Parma. Support for this work was provided to AMG by NSF grants BCS 0315434 and BCS 0744105 and to VG by MIUR (Ministero Italiano dell’Università e della Ricerca) and by the EU grants DISCOS, NESTCOM and ROSSI. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily
References (119)
- et al.
Visual and oculomotor selection: Links, causes and implications for spatial attention
Trends in Cognitive Sciences
(2006) - et al.
Embodied semantics and the premotor cortex: Congruent representations for visually presented actions and linguistic phrases describing actions
Current Biology
(2006) - et al.
Speech and gesture share the same communication system
Neuropsychologia
(2006) - et al.
The mirror neuron system and action recognition
Brain and Language
(2004) - et al.
Listening to action-related sentences modulates the activity of the motor system: A combined TMS and behavioral study
Cognitive Brain Research
(2005) - et al.
The motor somatotopy of speech perception
Current Biology
(2009) - et al.
A mirror up to nature
Current Biology
(2008) - et al.
Dynamic anticipatory processing of hierarchical sequential events: A common role for Broca’s area and ventral premotor cortex across domains?
Cortex
(2006) - et al.
Mirror neurons and the simulation theory of mind-reading
Trends in Cognitive Sciences
(1998) - et al.
From manual gesture to speech: A gradual transition
Neuroscience and Biobehavioral Reviews
(2006)
Symbol grounding and meaning: A comparison of high-dimensional and embodied theories of meaning
Journal of Memory and Language
Mental models contribute to foregrounding during text comprehension
Journal of Memory and Language
Use-induced motor plasticity affects the processing of abstract and concrete language
Current Biology
Language in the two-year-old: Receptive and productive stages
Cognition
Neural modeling and imaging of the cortical interactions underlying syllable production
Brain and Language
Hierarchical MOSAIC for movement generation
International Congress Series
Somatotopic representation of action words in human motor and premotor cortex
Neuron
Beyond a single area: Motor control and language within a neural architecture encompassing Broca’s area
Cortex
Constructing meaning: The role of affordances and grammatical constructions in sentence comprehension
Journal of Memory and Language
Perception of motion affects language processing
Cognition
Offering a hand to pragmatic understanding: The role of speech and gesture in comprehension and memory
Journal of Memory and Language
Corticospinal neurons in macaque ventral premotor cortex with mirror properties: A potential mechanism for action suppression?
Neuron
The essential role of premotor cortex in speech perception
Current Biology
Single-neuron responses in humans during execution and observation of actions
Current Biology
Do people use language production to make predictions during comprehension?
Trends in Cognitive Sciences
Representation, space, and Hollywood squares: Looking at things that aren’t there anymore
Cognition
Language within our grasp
Trends in Neurosciences
The cortical motor system
Neuron
Reorienting attention across the horizontal and vertical meridians: Evidence in favor of a premotor theory
Neuropsychologia
Premotor cortex and the recognition of motor actions
Cognitive Brain Research
Now you see it, now you don’t: Mediating the mapping between language and the visual world
Neural re-use as a fundamental organizational principle of the brain
Behavioral and Brain Sciences
Infant Gestures Predict Verb Production One Year Later
The effects of motor neurone disease on language: Further evidence
Brain and Language
Stored word sequences in language learning
Psychological Science
Perceptual symbol systems
The Behavioral and Brain Sciences
Predicting the consequences of our own actions: The role of sensorimotor context estimation
The Journal of Neuroscience
Ventral premotor and inferior parietal cortices make distinct contribution to action organization and intention understanding
Cerebral Cortex
Putting words in perspective
Memory and Cognition
Action observation activates premotor and parietal areas in a somatotopic manner: An fMRI study
European Journal of Neuroscience
Functions of the mirror neuron system: Implications for neurorehabilitation
Cognitive Behavioral Neurology
The roots of verbs in prelinguistic action knowledge
Wired to be social: The ontogeny of human interaction
PLoS ONE
Actions and affordances in syntactic ambiguity resolution
Journal of Experimental Psychology: Learning, Memory, and Cognition
Becoming syntactic
Psychological Review
Two-year-olds learn novel nouns, verbs, and conventional actions from massed or distributed exposures
Developmental Psychology
Rapid plasticity of human cortical movement representation induced by practice
Journal of Neurophysiology
Visuomotor priming
Visual Cognition
Levels of embodied meaning. From pointing to counterfactuals
Action representation and language in the brain
Theoretical Linguistics
Cited by (391)
Developmental changes in the visual, haptic, and bimodal perception of geometric angles
2024, Journal of Experimental Child PsychologyGesture like a kitten and you won't forget your tale: Drama-based, embodied story time supports preschoolers’ narrative skills
2024, Early Childhood Research QuarterlyNeurobiological mechanisms for language, symbols and concepts: Clues from brain-constrained deep neural networks
2023, Progress in NeurobiologyThe effect of fine motor skills, handwriting, and typing on reading development
2023, Journal of Experimental Child PsychologySchool-based physical activity interventions and language skills: a systematic review and meta-analysis of randomized controlled trials
2023, Journal of Science and Medicine in SportAltered evoked responses for motor-related words in children with upper limb motor impairments
2023, Clinical Neurophysiology