Elsevier

NeuroImage

Volume 37, Issue 3, 1 September 2007, Pages 1017-1031
NeuroImage

Dissociable but inter-related systems of cognitive control and reward during decision making: Evidence from pupillometry and event-related fMRI

https://doi.org/10.1016/j.neuroimage.2007.04.066Get rights and content

Abstract

Decision making involves the allocation of cognitive resources in response to expectations and feedback. Here we explored how frontal networks respond in a gambling paradigm in which uncertainty was manipulated to increase demands for cognitive control. In one experiment, pupil diameter covaried with uncertainty during decision making and with the degree to which subsequent outcomes violated reward expectations. In a second experiment, fMRI showed that both uncertainty and unexpected outcomes modulated activation in a network of frontal regions. Thus, the frontal network supports multiple phases of the decision-making process including information regarding reward uncertainty and reward outcome. In contrast, striatal activation only tracked reward delivery, suggesting a distinct reward pathway that might, under certain circumstances, oppose the frontal network. These results are consistent with the interpretation that reward signals may bias recruitment of frontal networks that are linked to allocation of cognitive resources.

Introduction

Decision making under uncertainty recruits a distributed network of frontal regions (Botvinick et al., 2001, Huettel et al., 2005, Volz et al., 2005). Lateral and midline frontal regions found to be active in neuroimaging studies of decision making are similar to those observed in other paradigms that require controlled processing. Activation in these frontal regions typically increases with task difficulty (Duncan and Owen, 2000, Paus et al., 1998). Moreover, frontal networks involved in cognitive control may be biased by dopamine (DA) neuromodulatory signals that may direct the allocation of cognitive resources (Cohen et al., 2002, Holroyd and Coles, 2002, Schultz, 2002). In an elegant series of studies, Schultz and colleagues demonstrated that midbrain DA neurons respond to rewards in a variety of contexts, exhibiting phasic firing patterns that code for prediction errors in response to discrepancies between reward expectation and delivery (Fiorillo et al., 2003, Hollerman and Schultz, 1998, Schultz et al., 1997).

The finding of phasic DA firing in animal studies has motivated several neuroimaging studies in humans. These imaging studies have consistently observed reward-related activity in the striatum (for reviews, see Knutson and Cooper, 2005, McClure et al., 2004), which are known to receive significant DA projections from the midbrain (Lidow et al., 1989, Lynd-Balta and Haber, 1994). Responding for rewards increases [11C] raclopride (a dopamine marker) binding in the striatum (Koepp et al., 1998, Zald et al., 2004). Moreover, striatal activity increases in response to rewarding stimuli such as cocaine and amphetamine infusion (Breiter et al., 1997, Knutson et al., 2004), beautiful faces (Aharon et al., 2001), juice (Berns et al., 2001, McClure et al., 2003, O'Doherty et al., 2002) and monetary rewards (Breiter et al., 2001, Delgado et al., 2003, Dreher et al., 2006, Elliott et al., 2000, Haruno et al., 2004, Knutson et al., 2000). Consistent with Schultz et al. (1997), imaging experiments have found reward prediction error signals in the putamen (McClure et al., 2003) and nucleus accumbens (O'Doherty et al., 2003).

However, there are little data concerning how networks involved in cognitive control interact with reward prediction and error signals during decision making. Understanding this interaction is important because it has been proposed that dopaminergic reward pathways may serve as motivating and learning signals in the dynamic allocation of resources during decision making (Cohen et al., 2002, Holroyd and Coles, 2002, Montague et al., 2004, Schultz, 2002).

Here we used pupillometry and functional MRI (fMRI) to examine the response of control networks in a gambling paradigm similar to that employed by Critchley et al. (2001). Gambling paradigms have been used to investigate human decision making (Breiter et al., 2001, Critchley et al., 2001, Delgado et al., 2000, Elliott et al., 2000, Rogers et al., 1999), in part because they enable researchers to manipulate both the demands for cognitive control and the level of reward expectation. In the current study, subjects made decisions under varying levels of reward uncertainty and received monetary feedback that confirmed or violated these expectations. This design has the advantage of being able to dissociate processes of cognitive control that are associated with decision uncertainty from reward processes associated with trial outcome (Maunsell, 2004).

We hypothesized that both the uncertainty of decisions and the degree to which outcomes violate expectations would be associated with increased activity in frontal control networks. In contrast, as has been well established in the literature, we expected to confirm that activity in subcortical reward pathways would depend primarily on reward outcome. Dissociation of the effects of reward and increased demands for cognitive control should be most apparent on trials involving an unexpected loss, which would lead to augmented frontal recruitment.

Interpretation of the results of the present study may be complicated by the fact that uncertainty and expected value covaried because the level of reward expectation was manipulated while holding the amount of reward constant. Because the expected value of a reward is its amount times its probability, it follows that increasing uncertainty by decreasing reward probability lowers expected value. However, Knutson et al. (2005) have shown that activity in medial prefrontal regions is positively correlated with overall expected value. Therefore, one would expect that if expected value is the dominant factor, then activity in these regions should be depressed on uncertain (low expected value) trials, whereas if uncertainty is the critical factor in this paradigm, as we hypothesize, then activity should increase.

To gain insight into the temporal dynamics of resource allocation during our gambling paradigm, in a separate group of subjects we monitored changes in pupil diameter over the course of each trial. Pupil diameter has long been known to provide a nonspecific measure of cognitive effort with high temporal resolution (for a review, see Beatty and Lucero-Wagoner, 2000). The temporal resolution of pupillometry was important for this study because it allowed examination of the temporal dynamics of the gambling task in a way that is not possible using fMRI alone.

Although the mechanism by which pupil diameter tracks task demands remains uncertain (but see Aston-Jones and Cohen, 2005), neuroimaging studies have demonstrated that changes in pupil diameter correlate with activity in prefrontal regions involved in cognitive control (Critchley et al., 2005, Siegle et al., 2003). In the present study, pupillometry demonstrated effects of control during temporally separate decision and feedback epochs, while event-related fMRI revealed neural correlates of cognitive control that were distinct from regions that respond to reward outcomes. These results reveal a dissociation between a striatal reward system and a frontal control network, consistent with the theory that dopaminergic reward pathways provide a modulatory gating signal to a frontal network of cognitive control (Holroyd and Coles, 2002, Montague et al., 2004).

Section snippets

Subjects

Thirty-three subjects participated in the pupillometry experiment. All had normal or contact-lens-corrected vision. One subject was excluded from the analysis due to an error in data acquisition, leaving 32 subjects (mean age 20.1 years; 16 males).

Twenty-seven subjects participated in the fMRI experiment. All were right handed, had no history of neurologic injury, and had normal or corrected vision. One subject was excluded from the analysis due to an error in data acquisition, leaving 26

Choice proportions and response times

In both the pupillometry and fMRI experiments, subjects choosing between the face-up and face-down cards nearly always selected the card with the higher probability of reward (Figs. 2A and 3A). The amount of time it took them to make their selection increased as a function of the uncertainty of the outcome (Figs. 2B and 3B, and Table 1). Response times (RTs) on uncertain trials were slower than RTs on probable trials [pupillometry: t(31) = 17.63, p < 0.001; fMRI: t(25) = 11.84, p < 0.001], which in

Discussion

The present study investigated the neural processes underlying decision making in a gambling paradigm. In one experiment, the temporal dynamics of cognitive resource allocation were studied using pupillometry. We found separate phasic responses during the decision and feedback epochs. Robust pupil dilation, indicative of increased cognitive effort, occurred in response to uncertainty early during the decision epoch and in response to unexpected losses during the feedback epoch. The fMRI results

Acknowledgments

We thank Neal Cohen and John Stern for their invaluable assistance with the cognitive pupillometry. Avi Snyder and Mark McAvoy provided support with imaging and analysis tools. We also thank Ben Shannon for discussion. This work was supported in by NIH grants MH55308, AG05886 and the Howard Hughes Medical Institute.

References (85)

  • J. Duncan et al.

    Common regions of the human frontal lobe recruited by diverse cognitive demands

    Trends Neurosci.

    (2000)
  • W. Ellermeier et al.

    Gender differences in pain ratings and pupil reactions to painful pressure stimuli

    Pain

    (1995)
  • K.J. Friston et al.

    Analysis of fMRI time-series revisited

    NeuroImage

    (1995)
  • B.T. Gold et al.

    Common prefrontal regions coactivate with dissociable posterior regions during controlled semantic and phonological tasks

    Neuron

    (2002)
  • B. Knutson et al.

    fMRI visualization of brain activity during a monetary incentive delay task

    NeuroImage

    (2000)
  • B. Knutson et al.

    Amphetamine modulates human incentive processing

    Neuron

    (2004)
  • E. Lynd-Balta et al.

    The organization of midbrain projections to the ventral striatum in the primate

    Neuroscience

    (1994)
  • L. Maccotta et al.

    Rapid self-paced event-related functional MRI: feasibility and implications of stimulus- versus response-locked timing

    NeuroImage

    (2001)
  • J.H. Maunsell

    Neuronal representations of cognitive state: reward or attention?

    Trends Cogn. Sci.

    (2004)
  • S.M. McClure et al.

    Temporal prediction errors in a passive learning task activate human striatum

    Neuron

    (2003)
  • F.M. Miezin et al.

    Characterizing the hemodynamic response: effects of presentation rate, sampling procedure, and the possibility of ordering brain activity based on relative timing

    NeuroImage

    (2000)
  • J.P. O'Doherty et al.

    Neural responses during anticipation of a primary taste reward

    Neuron

    (2002)
  • J.P. O'Doherty et al.

    Temporal difference models and reward-related learning in the human brain

    Neuron

    (2003)
  • M.P. Paulus et al.

    Error rate and outcome predictability affect neural activation in prefrontal cortex and anterior cingulate during decision-making

    NeuroImage

    (2002)
  • D.L. Schacter et al.

    Late onset of anterior prefrontal activity during true and false recognition: an event-related fMRI study

    NeuroImage

    (1997)
  • W. Schultz

    Getting formal with dopamine and reward

    Neuron

    (2002)
  • G.J. Siegle et al.

    Use of concurrent pupil dilation assessment to inform interpretation and analysis of fMRI data

    NeuroImage

    (2003)
  • A.Z. Snyder

    Difference image versus ratio image error function forms in PET-PET realignment. In Quantification of brain functioning PET

  • K.G. Volz et al.

    Predicting events of varying probability: uncertainty investigated by fMRI

    NeuroImage

    (2003)
  • K.G. Volz et al.

    Variants of uncertainty in decision-making and their neural correlates

    Brain Res. Bull.

    (2005)
  • T.D. Wager et al.

    Common and unique components of response inhibition revealed by fMRI

    NeuroImage

    (2005)
  • M.E. Wheeler et al.

    Functional–anatomic correlates of remembering and knowing

    NeuroImage

    (2004)
  • K.J. Worsley et al.

    Analysis of fMRI time-series revisited—Again

    NeuroImage

    (1995)
  • G. Aston-Jones et al.

    An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance

    Annu. Rev. Neurosci.

    (2005)
  • J. Beatty

    Task-evoked pupillary responses, processing load, and the structure of processing resources

    Psychol. Bull.

    (1982)
  • J. Beatty et al.

    The Pupillary System. In Handbook of Psychophysiology

  • G.S. Berns et al.

    Predictability modulates human brain response to reward

    J. Neurosci.

    (2001)
  • M.M. Botvinick et al.

    Conflict monitoring and cognitive control

    Psychol. Rev.

    (2001)
  • G.M. Boynton et al.

    Linear systems analysis of functional magnetic resonance imaging in human V1

    J. Neurosci.

    (1996)
  • J.W. Brown et al.

    Learned predictions of error likelihood in the anterior cingulate cortex

    Science

    (2005)
  • R.L. Buckner

    Functional–anatomic correlates of control processes in memory

    J. Neurosci.

    (2003)
  • R.L. Buckner et al.

    Functional anatomic studies of memory retrieval for auditory words and visual pictures

    J. Neurosci.

    (1996)
  • Cited by (110)

    • Need for cognition is associated with the interaction of reward and task-load on effort: A verification and extension study

      2022, International Journal of Psychophysiology
      Citation Excerpt :

      Pupillometry techniques can further help to reveal the underlying neural network of the association between individual cognitive motivation and actual effort expenditure. The pupil is innervated by brain areas associated with both cognitive and reward processing (Szabadi, 1996; Satterthwaite et al., 2007), and thus variation in pupil dilation is associated with multiple operations that reside at the intersection of cognition and motivation. This has been demonstrated for such tasks as speech in noise recognition under monetary reward (Koelewijn et al., 2018) and stress (Pielage et al., 2021), and cognitive effort towards success importance (Richter, 2016) or lower motivational negative symptoms (McGovern et al., 2020).

    View all citing articles on Scopus
    View full text