Elsevier

Cognition

Volume 114, Issue 3, March 2010, Pages 356-371
Cognition

Implicit statistical learning in language processing: Word predictability is the key

https://doi.org/10.1016/j.cognition.2009.10.009Get rights and content

Abstract

Fundamental learning abilities related to the implicit encoding of sequential structure have been postulated to underlie language acquisition and processing. However, there is very little direct evidence to date supporting such a link between implicit statistical learning and language. In three experiments using novel methods of assessing implicit learning and language abilities, we show that sensitivity to sequential structure – as measured by improvements to immediate memory span for structurally-consistent input sequences – is significantly correlated with the ability to use knowledge of word predictability to aid speech perception under degraded listening conditions. Importantly, the association remained even after controlling for participant performance on other cognitive tasks, including short-term and working memory, intelligence, attention and inhibition, and vocabulary knowledge. Thus, the evidence suggests that implicit learning abilities are essential for acquiring long-term knowledge of the sequential structure of language – i.e., knowledge of word predictability – and that individual differences on such abilities impact speech perception in everyday situations. These findings provide a new theoretical rationale linking basic learning phenomena to specific aspects of spoken language processing in adults, and may furthermore indicate new fruitful directions for investigating both typical and atypical language development.

Introduction

Understanding the role that learning and memory abilities play in language acquisition and processing remains an important challenge in the cognitive sciences. Towards this end, a major advance has been in recognizing that language consists of complex, highly variable patterns occurring in sequence, and as such can be described in terms of statistical or distributional relations among language units (Redington & Chater, 1997). Due to the probabilistic nature of language, rarely is a spoken utterance completely unpredictable; most often, the next word in a sentence will depend on the preceding context of the sentence (Rubenstein, 1973). Put another way, what a language speaker considers to be a “meaningful” sentence can be quantified in terms of how much the preceding context constrains or predicts the next spoken word (Miller & Selfridge, 1950). Due to the apparent importance of context and word predictability in language, sensitivity to such probabilistic relations among language units likely is crucial for successful language learning and understanding.

It is not surprising then, that it is now widely accepted that general abilities related to learning about complex structured patterns – i.e., implicit statistical learning1 – are important for language processing (Altmann, 2002, Conway and Christiansen, 2005, Conway and Pisoni, 2008, Gupta and Dell, 1999, Kirkham et al., 2007, Kuhl, 2004, Pothos, 2007, Reber, 1967, Saffran, 2003, Turk-Browne et al., 2005, Ullman, 2004). Implicit learning is thought to be important for word segmentation (Saffran, Aslin, & Newport, 1996), word learning (Graf Estes et al., 2007, Mirman et al., 2008), the learning of phonotactic (Chambers, Onishi, & Fisher, 2003) and orthographic (Pacton, Perruchet, Fayol, & Cleeremans, 2001) regularities, aspects of speech production (Dell, Reed, Adams, & Meyer, 2000), and the acquisition of syntax (Gomez and Gerken, 2000, Ullman, 2004). What is more surprising, however, is that despite the voluminous work on implicit learning, few if any studies have demonstrated a direct causal link between implicit learning abilities and everyday language competence. Although there is some evidence suggesting that implicit learning is disturbed in certain language-impaired clinical populations (e.g., Evans et al., 2009, Howard et al., 2006, Plante et al., 2002, Tomblin et al., 2007), other studies have revealed no such relationship between implicit learning and language processing, and the reason for the discrepancy is not entirely clear (for additional discussion, see Conway, Karpicke, & Pisoni, 2007).

We propose that if implicit learning supports language, then it ought to be possible to demonstrate an empirical association between individual differences in implicit learning abilities in healthy adults and some measure of language processing. However, a challenge lies in choosing language and implicit learning tasks that purportedly tap into the same underlying processes. Toward this end, we use Elman’s (1990) now classic paper as a theoretical foundation, in which a connectionist model – a simple recurrent network (SRN) – was shown to represent sequential order implicitly in terms of the effect it had on processing. The SRN had a context layer that served to give it a memory for previous internal states. This memory, coupled with the network’s learning algorithm, gave the SRN the ability to learn about structure in sequential input, enabling it to predict the next element in a sequence, based on the preceding context. Elman (1990) and many others since have used the SRN successfully to model both language learning and processing (Christiansen & Chater, 1999) and, interestingly enough, implicit learning (Cleeremans, 1993).

The crucial commonality between implicit (sequence) learning and language learning and processing may be the ability to encode and represent sequential input, using preceding context to implicitly predict upcoming units. To directly test this hypothesis, we explore whether individual differences in implicit learning abilities are related to how well one is able to use sentence context – i.e., word predictability – to guide spoken language perception under degraded listening conditions.

Section snippets

Word predictability in spoken language perception

Previous work has shown that knowledge of the sequential probabilities in language can enable a listener to better identify – and perhaps even implicitly predict – the next word that will be spoken (Miller et al., 1951, Onnis et al., 2008, Rubenstein, 1973; c.f., Bar, 2007). This use of top-down knowledge becomes especially apparent when the speech signal is perceptually degraded, which is the case in many real-world situations. When ambient noise degrades parts of a spoken utterance, the

Experiment 1

In the first experiment, participants engaged in a visual implicit learning task that indirectly assessed learning through improvements to immediate memory span for sequences containing redundant statistical structure (Conway et al., 2007, Karpicke and Pisoni, 2004, Miller and Selfridge, 1950). Participants also completed a speech perception in noise task that used degraded sentences varying in the predictability of the final word. If implicit learning abilities are important for acquiring

Experiment 2

Experiment 1 demonstrated a statistically significant correlation between visual implicit learning and the use of knowledge of word order predictability in auditory-only speech perception. Experiment 2 served two purposes. First, it was designed to replicate the main finding of Experiment 1 but with a change in both the sensory modalities of the two experimental tasks (see Table 3) and the type of underlying structure used to generate the input sequences in the implicit learning task. If a

Experiments 1 and 2 combined

Fig. 4 shows a scatterplot of the data from both Experiments 1 and 2 combined (using standardized z-scores for the implicit learning task). As can be seen from the plot, the overall correlation between implicit learning and the word predictability difference score is positive and statistically significant (r = .418, p < .01, 2-tailed).

Experiment 3

Experiments 1 and 2 demonstrated that implicit learning abilities are associated with the ability to use knowledge of word order predictability to aid speech perception. This association remained strong even after controlling for the common variance associated with linguistic knowledge, general intelligence, and short-term sequence memory capacity. As a final replication and extension of these findings, we next include, in addition to a visual implicit learning and auditory-only sentence

General discussion

The data from these three experiments show that performance on an implicit learning task is significantly correlated with performance on a spoken language measure that assesses sensitivity to word predictability in speech. The implicit learning tasks involved observing and reproducing visual color or auditory nonword sequences; a learning score was calculated for each individual by measuring the improvement to immediate serial recall for sequences with consistent statistical structure. The

Acknowledgment

This project was supported by the following grants from the National Institute on Deafness and Other Communication Disorders: R03DC009485 and T32DC00012.

We wish to thank Lauren Grove for her help in data collection and manuscript preparation, and Luis Hernandez for his technical assistance.

References (70)

  • R.L. Gomez et al.

    Infant artificial language learning and language acquisition

    Trends in Cognitive Sciences

    (2000)
  • H. Grunow et al.

    The effects of variation on learning word order rules by adults with and without language-based learning disabilities

    Journal of Communication Disorders

    (2006)
  • J.H. Howard et al.

    Dyslexics are impaired on implicit higher-order sequence learning, but not on implicit spatial context learning

    Neuropsychologia

    (2006)
  • J.L. McClelland et al.

    Are there interactive processes in speech perception?

    Trends in Cognitive Sciences

    (2006)
  • D. Menghini et al.

    Implicit learning deficits in dyslexic adults: An fMRI study

    Neuroimage

    (2006)
  • D. Mirman et al.

    The link between statistical segmentation and word learning in adults

    Cognition

    (2008)
  • E.L. Newport

    Maturational constraints on language learning

    Cognitive Science

    (1990)
  • P. Perruchet et al.

    Implicit learning and statistical learning: One phenomenon, two approaches

    Trends in Cognitive Sciences

    (2006)
  • E. Plante et al.

    Sensitivity to word order cues by normal and language/learning disabled adults

    Journal of Communication Disorders

    (2002)
  • A.S. Reber

    Implicit learning of artificial grammars

    Journal of Verbal Learning and Verbal Behavior

    (1967)
  • M. Redington et al.

    Probabilistic and distributional approaches to language acquisition

    Trends in Cognitive Sciences

    (1997)
  • V.M. Rosen et al.

    Forward and backward serial recall

    Intelligence

    (1997)
  • M.T. Ullman

    Contributions of memory circuits to language: The declarative/procedural model

    Cognition

    (2004)
  • S. Vicari et al.

    Implicit learning deficit in children with developmental dyslexia

    Neuropsychologia

    (2003)
  • G.T.M. Altmann

    Statistical learning in infants

    Proceedings of the National Academy of Sciences

    (2002)
  • A.D. Baddeley

    Working memory

    (1986)
  • B.M. Bernhardt et al.

    Early word-object associations and later language development

    First Language

    (2007)
  • R.C. Bilger et al.

    Relationships between high- and low-probability SPIN scores

    The Journal of the Acoustical Society of America

    (1979)
  • M.M. Botvinick et al.

    Short-term memory for serial order: A recurrent neural network model

    Psychological Review

    (2006)
  • M.H. Christiansen et al.

    Neural responses to structural incongruencies in language and statistical learning point to similar underlying mechanisms

  • A. Cleeremans

    Mechanisms of implicit learning: Connectionist models of sequence learning

    (1993)
  • C.M. Conway et al.

    Modality-constrained statistical learning of tactile, visual, and auditory sequences

    Journal of Experimental Psychology

    (2005)
  • C.M. Conway et al.

    Statistical learning within and between modalities: Pitting abstract against stimulus-specific representations

    Psychological Science

    (2006)
  • C.M. Conway et al.

    The importance of sound for cognitive sequencing abilities: The auditory scaffolding hypothesis

    Current Directions in Psychological Science

    (2009)
  • C.M. Conway et al.

    Contribution of implicit sequence learning to spoken language processing: Some preliminary findings from normal-hearing adults

    Journal of Deaf Studies and Deaf Education

    (2007)
  • Cited by (248)

    • Weight status impacts children's incidental statistical learning

      2023, International Journal of Psychophysiology
    View all citing articles on Scopus

    Portions of this research was presented at the 29th Annual Meeting of the Cognitive Science Society, Nashville, TN, August, 2007.

    View full text