Dissociable but inter-related systems of cognitive control and reward during decision making: Evidence from pupillometry and event-related fMRI
Introduction
Decision making under uncertainty recruits a distributed network of frontal regions (Botvinick et al., 2001, Huettel et al., 2005, Volz et al., 2005). Lateral and midline frontal regions found to be active in neuroimaging studies of decision making are similar to those observed in other paradigms that require controlled processing. Activation in these frontal regions typically increases with task difficulty (Duncan and Owen, 2000, Paus et al., 1998). Moreover, frontal networks involved in cognitive control may be biased by dopamine (DA) neuromodulatory signals that may direct the allocation of cognitive resources (Cohen et al., 2002, Holroyd and Coles, 2002, Schultz, 2002). In an elegant series of studies, Schultz and colleagues demonstrated that midbrain DA neurons respond to rewards in a variety of contexts, exhibiting phasic firing patterns that code for prediction errors in response to discrepancies between reward expectation and delivery (Fiorillo et al., 2003, Hollerman and Schultz, 1998, Schultz et al., 1997).
The finding of phasic DA firing in animal studies has motivated several neuroimaging studies in humans. These imaging studies have consistently observed reward-related activity in the striatum (for reviews, see Knutson and Cooper, 2005, McClure et al., 2004), which are known to receive significant DA projections from the midbrain (Lidow et al., 1989, Lynd-Balta and Haber, 1994). Responding for rewards increases [11C] raclopride (a dopamine marker) binding in the striatum (Koepp et al., 1998, Zald et al., 2004). Moreover, striatal activity increases in response to rewarding stimuli such as cocaine and amphetamine infusion (Breiter et al., 1997, Knutson et al., 2004), beautiful faces (Aharon et al., 2001), juice (Berns et al., 2001, McClure et al., 2003, O'Doherty et al., 2002) and monetary rewards (Breiter et al., 2001, Delgado et al., 2003, Dreher et al., 2006, Elliott et al., 2000, Haruno et al., 2004, Knutson et al., 2000). Consistent with Schultz et al. (1997), imaging experiments have found reward prediction error signals in the putamen (McClure et al., 2003) and nucleus accumbens (O'Doherty et al., 2003).
However, there are little data concerning how networks involved in cognitive control interact with reward prediction and error signals during decision making. Understanding this interaction is important because it has been proposed that dopaminergic reward pathways may serve as motivating and learning signals in the dynamic allocation of resources during decision making (Cohen et al., 2002, Holroyd and Coles, 2002, Montague et al., 2004, Schultz, 2002).
Here we used pupillometry and functional MRI (fMRI) to examine the response of control networks in a gambling paradigm similar to that employed by Critchley et al. (2001). Gambling paradigms have been used to investigate human decision making (Breiter et al., 2001, Critchley et al., 2001, Delgado et al., 2000, Elliott et al., 2000, Rogers et al., 1999), in part because they enable researchers to manipulate both the demands for cognitive control and the level of reward expectation. In the current study, subjects made decisions under varying levels of reward uncertainty and received monetary feedback that confirmed or violated these expectations. This design has the advantage of being able to dissociate processes of cognitive control that are associated with decision uncertainty from reward processes associated with trial outcome (Maunsell, 2004).
We hypothesized that both the uncertainty of decisions and the degree to which outcomes violate expectations would be associated with increased activity in frontal control networks. In contrast, as has been well established in the literature, we expected to confirm that activity in subcortical reward pathways would depend primarily on reward outcome. Dissociation of the effects of reward and increased demands for cognitive control should be most apparent on trials involving an unexpected loss, which would lead to augmented frontal recruitment.
Interpretation of the results of the present study may be complicated by the fact that uncertainty and expected value covaried because the level of reward expectation was manipulated while holding the amount of reward constant. Because the expected value of a reward is its amount times its probability, it follows that increasing uncertainty by decreasing reward probability lowers expected value. However, Knutson et al. (2005) have shown that activity in medial prefrontal regions is positively correlated with overall expected value. Therefore, one would expect that if expected value is the dominant factor, then activity in these regions should be depressed on uncertain (low expected value) trials, whereas if uncertainty is the critical factor in this paradigm, as we hypothesize, then activity should increase.
To gain insight into the temporal dynamics of resource allocation during our gambling paradigm, in a separate group of subjects we monitored changes in pupil diameter over the course of each trial. Pupil diameter has long been known to provide a nonspecific measure of cognitive effort with high temporal resolution (for a review, see Beatty and Lucero-Wagoner, 2000). The temporal resolution of pupillometry was important for this study because it allowed examination of the temporal dynamics of the gambling task in a way that is not possible using fMRI alone.
Although the mechanism by which pupil diameter tracks task demands remains uncertain (but see Aston-Jones and Cohen, 2005), neuroimaging studies have demonstrated that changes in pupil diameter correlate with activity in prefrontal regions involved in cognitive control (Critchley et al., 2005, Siegle et al., 2003). In the present study, pupillometry demonstrated effects of control during temporally separate decision and feedback epochs, while event-related fMRI revealed neural correlates of cognitive control that were distinct from regions that respond to reward outcomes. These results reveal a dissociation between a striatal reward system and a frontal control network, consistent with the theory that dopaminergic reward pathways provide a modulatory gating signal to a frontal network of cognitive control (Holroyd and Coles, 2002, Montague et al., 2004).
Section snippets
Subjects
Thirty-three subjects participated in the pupillometry experiment. All had normal or contact-lens-corrected vision. One subject was excluded from the analysis due to an error in data acquisition, leaving 32 subjects (mean age 20.1 years; 16 males).
Twenty-seven subjects participated in the fMRI experiment. All were right handed, had no history of neurologic injury, and had normal or corrected vision. One subject was excluded from the analysis due to an error in data acquisition, leaving 26
Choice proportions and response times
In both the pupillometry and fMRI experiments, subjects choosing between the face-up and face-down cards nearly always selected the card with the higher probability of reward (Figs. 2A and 3A). The amount of time it took them to make their selection increased as a function of the uncertainty of the outcome (Figs. 2B and 3B, and Table 1). Response times (RTs) on uncertain trials were slower than RTs on probable trials [pupillometry: t(31) = 17.63, p < 0.001; fMRI: t(25) = 11.84, p < 0.001], which in
Discussion
The present study investigated the neural processes underlying decision making in a gambling paradigm. In one experiment, the temporal dynamics of cognitive resource allocation were studied using pupillometry. We found separate phasic responses during the decision and feedback epochs. Robust pupil dilation, indicative of increased cognitive effort, occurred in response to uncertainty early during the decision epoch and in response to unexpected losses during the feedback epoch. The fMRI results
Acknowledgments
We thank Neal Cohen and John Stern for their invaluable assistance with the cognitive pupillometry. Avi Snyder and Mark McAvoy provided support with imaging and analysis tools. We also thank Ben Shannon for discussion. This work was supported in by NIH grants MH55308, AG05886 and the Howard Hughes Medical Institute.
References (85)
- et al.
Beautiful faces have variable reward value: fMRI and behavioral evidence
Neuron
(2001) - et al.
The role of the inferior frontal junction area in cognitive control
Trends Cogn. Sci.
(2005) - et al.
Acute effects of cocaine on human brain activity and emotion
Neuron
(1997) - et al.
Functional imaging of neural responses to expectancy and experience of monetary gains and losses
Neuron
(2001) - et al.
Functional–anatomic correlates of object priming in humans revealed by rapid presentation event-related fMRI
Neuron
(1998) - et al.
Functional–anatomic study of episodic retrieval: II. Selective averaging of event-related fMRI trials to test the retrieval success hypothesis
NeuroImage
(1998) - et al.
Computational perspectives on dopamine function in prefrontal cortex
Curr. Opin. Neurobiol.
(2002) - et al.
Neural activity in the human brain relating to uncertainty and arousal during anticipation
Neuron
(2001) - et al.
Anterior cingulate activity during error and autonomic response
NeuroImage
(2005) - et al.
Cognitive control in the posterior frontolateral cortex: evidence from common activations in task coordination, interference control, and working memory
NeuroImage
(2004)
Common regions of the human frontal lobe recruited by diverse cognitive demands
Trends Neurosci.
Gender differences in pain ratings and pupil reactions to painful pressure stimuli
Pain
Analysis of fMRI time-series revisited
NeuroImage
Common prefrontal regions coactivate with dissociable posterior regions during controlled semantic and phonological tasks
Neuron
fMRI visualization of brain activity during a monetary incentive delay task
NeuroImage
Amphetamine modulates human incentive processing
Neuron
The organization of midbrain projections to the ventral striatum in the primate
Neuroscience
Rapid self-paced event-related functional MRI: feasibility and implications of stimulus- versus response-locked timing
NeuroImage
Neuronal representations of cognitive state: reward or attention?
Trends Cogn. Sci.
Temporal prediction errors in a passive learning task activate human striatum
Neuron
Characterizing the hemodynamic response: effects of presentation rate, sampling procedure, and the possibility of ordering brain activity based on relative timing
NeuroImage
Neural responses during anticipation of a primary taste reward
Neuron
Temporal difference models and reward-related learning in the human brain
Neuron
Error rate and outcome predictability affect neural activation in prefrontal cortex and anterior cingulate during decision-making
NeuroImage
Late onset of anterior prefrontal activity during true and false recognition: an event-related fMRI study
NeuroImage
Getting formal with dopamine and reward
Neuron
Use of concurrent pupil dilation assessment to inform interpretation and analysis of fMRI data
NeuroImage
Difference image versus ratio image error function forms in PET-PET realignment. In Quantification of brain functioning PET
Predicting events of varying probability: uncertainty investigated by fMRI
NeuroImage
Variants of uncertainty in decision-making and their neural correlates
Brain Res. Bull.
Common and unique components of response inhibition revealed by fMRI
NeuroImage
Functional–anatomic correlates of remembering and knowing
NeuroImage
Analysis of fMRI time-series revisited—Again
NeuroImage
An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance
Annu. Rev. Neurosci.
Task-evoked pupillary responses, processing load, and the structure of processing resources
Psychol. Bull.
The Pupillary System. In Handbook of Psychophysiology
Predictability modulates human brain response to reward
J. Neurosci.
Conflict monitoring and cognitive control
Psychol. Rev.
Linear systems analysis of functional magnetic resonance imaging in human V1
J. Neurosci.
Learned predictions of error likelihood in the anterior cingulate cortex
Science
Functional–anatomic correlates of control processes in memory
J. Neurosci.
Functional anatomic studies of memory retrieval for auditory words and visual pictures
J. Neurosci.
Cited by (110)
Classification of depression tendency from gaze patterns during sentence reading
2024, Biomedical Signal Processing and ControlPupil dilation and skin conductance as measures of prediction error in aversive learning
2022, Behaviour Research and TherapyNeed for cognition is associated with the interaction of reward and task-load on effort: A verification and extension study
2022, International Journal of PsychophysiologyCitation Excerpt :Pupillometry techniques can further help to reveal the underlying neural network of the association between individual cognitive motivation and actual effort expenditure. The pupil is innervated by brain areas associated with both cognitive and reward processing (Szabadi, 1996; Satterthwaite et al., 2007), and thus variation in pupil dilation is associated with multiple operations that reside at the intersection of cognition and motivation. This has been demonstrated for such tasks as speech in noise recognition under monetary reward (Koelewijn et al., 2018) and stress (Pielage et al., 2021), and cognitive effort towards success importance (Richter, 2016) or lower motivational negative symptoms (McGovern et al., 2020).
The influence of reward anticipation on conflict control in children and adolescents: Evidences from hierarchical drift-diffusion model and event-related potentials
2022, Developmental Cognitive Neuroscience