Implicit statistical learning in language processing: Word predictability is the key☆
Introduction
Understanding the role that learning and memory abilities play in language acquisition and processing remains an important challenge in the cognitive sciences. Towards this end, a major advance has been in recognizing that language consists of complex, highly variable patterns occurring in sequence, and as such can be described in terms of statistical or distributional relations among language units (Redington & Chater, 1997). Due to the probabilistic nature of language, rarely is a spoken utterance completely unpredictable; most often, the next word in a sentence will depend on the preceding context of the sentence (Rubenstein, 1973). Put another way, what a language speaker considers to be a “meaningful” sentence can be quantified in terms of how much the preceding context constrains or predicts the next spoken word (Miller & Selfridge, 1950). Due to the apparent importance of context and word predictability in language, sensitivity to such probabilistic relations among language units likely is crucial for successful language learning and understanding.
It is not surprising then, that it is now widely accepted that general abilities related to learning about complex structured patterns – i.e., implicit statistical learning1 – are important for language processing (Altmann, 2002, Conway and Christiansen, 2005, Conway and Pisoni, 2008, Gupta and Dell, 1999, Kirkham et al., 2007, Kuhl, 2004, Pothos, 2007, Reber, 1967, Saffran, 2003, Turk-Browne et al., 2005, Ullman, 2004). Implicit learning is thought to be important for word segmentation (Saffran, Aslin, & Newport, 1996), word learning (Graf Estes et al., 2007, Mirman et al., 2008), the learning of phonotactic (Chambers, Onishi, & Fisher, 2003) and orthographic (Pacton, Perruchet, Fayol, & Cleeremans, 2001) regularities, aspects of speech production (Dell, Reed, Adams, & Meyer, 2000), and the acquisition of syntax (Gomez and Gerken, 2000, Ullman, 2004). What is more surprising, however, is that despite the voluminous work on implicit learning, few if any studies have demonstrated a direct causal link between implicit learning abilities and everyday language competence. Although there is some evidence suggesting that implicit learning is disturbed in certain language-impaired clinical populations (e.g., Evans et al., 2009, Howard et al., 2006, Plante et al., 2002, Tomblin et al., 2007), other studies have revealed no such relationship between implicit learning and language processing, and the reason for the discrepancy is not entirely clear (for additional discussion, see Conway, Karpicke, & Pisoni, 2007).
We propose that if implicit learning supports language, then it ought to be possible to demonstrate an empirical association between individual differences in implicit learning abilities in healthy adults and some measure of language processing. However, a challenge lies in choosing language and implicit learning tasks that purportedly tap into the same underlying processes. Toward this end, we use Elman’s (1990) now classic paper as a theoretical foundation, in which a connectionist model – a simple recurrent network (SRN) – was shown to represent sequential order implicitly in terms of the effect it had on processing. The SRN had a context layer that served to give it a memory for previous internal states. This memory, coupled with the network’s learning algorithm, gave the SRN the ability to learn about structure in sequential input, enabling it to predict the next element in a sequence, based on the preceding context. Elman (1990) and many others since have used the SRN successfully to model both language learning and processing (Christiansen & Chater, 1999) and, interestingly enough, implicit learning (Cleeremans, 1993).
The crucial commonality between implicit (sequence) learning and language learning and processing may be the ability to encode and represent sequential input, using preceding context to implicitly predict upcoming units. To directly test this hypothesis, we explore whether individual differences in implicit learning abilities are related to how well one is able to use sentence context – i.e., word predictability – to guide spoken language perception under degraded listening conditions.
Section snippets
Word predictability in spoken language perception
Previous work has shown that knowledge of the sequential probabilities in language can enable a listener to better identify – and perhaps even implicitly predict – the next word that will be spoken (Miller et al., 1951, Onnis et al., 2008, Rubenstein, 1973; c.f., Bar, 2007). This use of top-down knowledge becomes especially apparent when the speech signal is perceptually degraded, which is the case in many real-world situations. When ambient noise degrades parts of a spoken utterance, the
Experiment 1
In the first experiment, participants engaged in a visual implicit learning task that indirectly assessed learning through improvements to immediate memory span for sequences containing redundant statistical structure (Conway et al., 2007, Karpicke and Pisoni, 2004, Miller and Selfridge, 1950). Participants also completed a speech perception in noise task that used degraded sentences varying in the predictability of the final word. If implicit learning abilities are important for acquiring
Experiment 2
Experiment 1 demonstrated a statistically significant correlation between visual implicit learning and the use of knowledge of word order predictability in auditory-only speech perception. Experiment 2 served two purposes. First, it was designed to replicate the main finding of Experiment 1 but with a change in both the sensory modalities of the two experimental tasks (see Table 3) and the type of underlying structure used to generate the input sequences in the implicit learning task. If a
Experiments 1 and 2 combined
Fig. 4 shows a scatterplot of the data from both Experiments 1 and 2 combined (using standardized z-scores for the implicit learning task). As can be seen from the plot, the overall correlation between implicit learning and the word predictability difference score is positive and statistically significant (r = .418, p < .01, 2-tailed).
Experiment 3
Experiments 1 and 2 demonstrated that implicit learning abilities are associated with the ability to use knowledge of word order predictability to aid speech perception. This association remained strong even after controlling for the common variance associated with linguistic knowledge, general intelligence, and short-term sequence memory capacity. As a final replication and extension of these findings, we next include, in addition to a visual implicit learning and auditory-only sentence
General discussion
The data from these three experiments show that performance on an implicit learning task is significantly correlated with performance on a spoken language measure that assesses sensitivity to word predictability in speech. The implicit learning tasks involved observing and reproducing visual color or auditory nonword sequences; a learning score was calculated for each individual by measuring the improvement to immediate serial recall for sequences with consistent statistical structure. The
Acknowledgment
This project was supported by the following grants from the National Institute on Deafness and Other Communication Disorders: R03DC009485 and T32DC00012.
We wish to thank Lauren Grove for her help in data collection and manuscript preparation, and Luis Hernandez for his technical assistance.
References (70)
The proactive brain: Using analogies and associations to generate predictions
Trends in Cognitive Sciences
(2007)Effects of domain-specific knowledge on memory for serial order
Cognition
(2005)The insufficiency of three types of learning to explain language acquisition
Lingua
(2008)- et al.
Infants learn phonotactic regularities from brief auditory experience
Cognition
(2003) - et al.
Toward a connectionist model of recursion in human linguistic performance
Cognitive Science
(1999) - et al.
The nationwide speech project: A new corpus of American English dialects
Speech Communication
(2006) Finding structure in time
Cognitive Science
(1990)Learning and development in neural networks: The importance of starting small
Cognition
(1993)- et al.
Correlational analyses of procedural and declarative learning performance
Intelligence
(1995) - et al.
Evaluation of the role of phonological STM in the development of vocabulary in children: A longitudinal study
Journal of Memory and Language
(1989)
Infant artificial language learning and language acquisition
Trends in Cognitive Sciences
The effects of variation on learning word order rules by adults with and without language-based learning disabilities
Journal of Communication Disorders
Dyslexics are impaired on implicit higher-order sequence learning, but not on implicit spatial context learning
Neuropsychologia
Are there interactive processes in speech perception?
Trends in Cognitive Sciences
Implicit learning deficits in dyslexic adults: An fMRI study
Neuroimage
The link between statistical segmentation and word learning in adults
Cognition
Maturational constraints on language learning
Cognitive Science
Implicit learning and statistical learning: One phenomenon, two approaches
Trends in Cognitive Sciences
Sensitivity to word order cues by normal and language/learning disabled adults
Journal of Communication Disorders
Implicit learning of artificial grammars
Journal of Verbal Learning and Verbal Behavior
Probabilistic and distributional approaches to language acquisition
Trends in Cognitive Sciences
Forward and backward serial recall
Intelligence
Contributions of memory circuits to language: The declarative/procedural model
Cognition
Implicit learning deficit in children with developmental dyslexia
Neuropsychologia
Statistical learning in infants
Proceedings of the National Academy of Sciences
Working memory
Early word-object associations and later language development
First Language
Relationships between high- and low-probability SPIN scores
The Journal of the Acoustical Society of America
Short-term memory for serial order: A recurrent neural network model
Psychological Review
Neural responses to structural incongruencies in language and statistical learning point to similar underlying mechanisms
Mechanisms of implicit learning: Connectionist models of sequence learning
Modality-constrained statistical learning of tactile, visual, and auditory sequences
Journal of Experimental Psychology
Statistical learning within and between modalities: Pitting abstract against stimulus-specific representations
Psychological Science
The importance of sound for cognitive sequencing abilities: The auditory scaffolding hypothesis
Current Directions in Psychological Science
Contribution of implicit sequence learning to spoken language processing: Some preliminary findings from normal-hearing adults
Journal of Deaf Studies and Deaf Education
Cited by (248)
Weight status impacts children's incidental statistical learning
2023, International Journal of PsychophysiologyNeural dynamics underlying the processing of implicit form-meaning connections: The dissociative roles of theta and alpha oscillations
2023, International Journal of PsychophysiologyNeural and cognitive correlates of performance in dynamic multi-modal settings
2023, Neuropsychologia
- ☆
Portions of this research was presented at the 29th Annual Meeting of the Cognitive Science Society, Nashville, TN, August, 2007.