Key Points
-
This review surveys recent behavioural and electrophysiological studies concerning the neural basis of value-based decisions. Psychophysicists and sensory physiologists traditionally emphasize the effects of sensory stimuli on decision-making, but cognitive psychologists and economists have long known that decision-making is strongly influenced by an organism's prior experience or beliefs concerning the 'value' of alternative choices, often expressed in terms of appetitive or aversive consequences. Although the neural mechanisms that underlie the computation of value are largely unknown, the emerging field of 'neuroeconomics' has taken on the task of elucidating these computations and how they influence choice behaviour.
-
Here we outline a programme of research for the electrophysiological investigation of value-based decision making in awake, behaving monkeys. The key components of this programme are: first, showing that choice behaviour is under the control of value computations emerging from an animal's history of choices and rewards; second, the modelling of behavioural data to gain insight into the decision variables in the brain that might specify these choices; and third, electrophysiological analysis to determine whether and how the hypothesized decision variables are actually encoded within specific neural systems. The heart of the review is a comparison of three recent papers on value-based choice, with specific attention to the key components outlined above. Through a careful consideration of this series of new studies, we aim to elucidate principles that will guide the investigation of value-based choice in the future.
-
From a broader point of view, the new effort to understand value-based choice offers hope for a substantive synthesis of two areas of systems neuroscience that have traditionally existed in separate spheres — the study of cognition and the study of reward and motivation. These subjects are intrinsically linked: cognitive behaviour is typically fuelled by motivation and reward; reward and motivation in turn serve cognitive and behavioural ends. The study of value-based choice provides an ideal platform for analysing the interaction of a quintessentially cognitive behaviour — decision-making — and a quintessentially motivational drive — reward harvesting. The studies reviewed in this paper make a promising start on this ambitious agenda.
Abstract
To make adaptive decisions, animals must evaluate the costs and benefits of available options. The nascent field of neuroeconomics has set itself the ambitious goal of understanding the brain mechanisms that are responsible for these evaluative processes. A series of recent neurophysiological studies in monkeys has begun to address this challenge using novel methods to manipulate and measure an animal's internal valuation of competing alternatives. By emphasizing the behavioural mechanisms and neural signals that mediate decision making under conditions of uncertainty, these studies might lay the foundation for an emerging neurobiology of choice behaviour.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$189.00 per year
only $15.75 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Bauby, J. D. The Diving Bell and The Butterfly: A Memoir of Life in Death (Alfred A. Knopf Inc., New York, 1997).
Glimcher, P. W. & Rustichini, A. R. Neuroeconomics: the consilience of brain and decision. Science 306, 447–452 (2004).
Parker, A. J. & Newsome, W. T. Sense and the single neuron: probing the physiology of perception. Annu. Rev. Neurosci. 21, 227–277 (1998). An exploration of the link between neural activity and sensory perception, summarizing work that comprises the background to recent neurophysiological studies of perceptual decision making.
Rizzolatti, G. & Luppino, G. The cortical motor system. Neuron 31, 889–901 (2001).
Graziano, M. S., Taylor, C. S., Moore, T. & Cooke, D. F. The cortical control of movement revisited. Neuron 36, 349–362 (2002).
Andersen, R. A. & Bruneo, C. A. Intentional maps in posterior parietal cortex. Annu. Rev. Neurosci. 25, 189–220 (2002).
Leon, M. I. & Shadlen, M. N. Exploring the neurophysiology of decisions. Neuron 21, 669–672 (1998).
Georgopoulos, A. P. Neural aspects of cognitive motor control. Curr. Opin. Neurobiol. 10, 238 (2000).
Schall, J. D. Neural basis of deciding, choosing and acting. Nature Rev. Neurosci. 2, 33–42 (2001).
Gold, J. I. & Shadlen, M. N. Neural computations that underlie decisions about sensory stimuli. Trends Cogn. Sci. 5, 10–16 (2001). Proposes that neural systems implement categorical decisions about perceptual stimuli by computing a decision variable related to the logarithm of the likelihood ratio in favour of one or another alternative. Discusses how this framework can be used to interpret neural signals recorded from eye movement planning centres during perceptual discrimination tasks.
Glimcher, P. W. Making choices: the neurophysiology of visual-saccadic decision making. Trends Neurosci. 24, 654–659 (2001).
Romo, R. & Salinas, E. Touch and go: decision making mechanisms in somatosensation. Annu. Rev. Neurosci. 24, 107–137 (2001).
Romo, R., Hernandez, A., Zainos, A., Lemus, L. & Brody, C. Neural correlates of decision-making in secondary somatosensory cortex. Nature Neurosci. 5, 1217–1225 (2002).
Romo, R. & Salinas, E. Flutter discrimination: neural codes, perception, memory and decision making. Nature Rev. Neurosci. 4, 203–218 (2003).
Graham, N. V. S. Visual Pattern Analysers (Oxford Univ. Press, Oxford, 1989).
Shadlen, M. N. & Newsome, W. T. Motion perception: seeing and deciding. Proc. Natl Acad. Sci. USA 93, 628–633 (1996). One of the first demonstrations of a correlation between the activity of single cortical neurons and an evolving perceptual decision.
Shadlen, M. N. & Newsome, W. T. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J. Neurophysiol. 86, 1916–1936 (2001).
Kim, J. N. & Shadlen, M. N. Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nature Neurosci. 2, 176–185 (1999).
Horwitz, G. D. & Newsome, W. T. Target selection for saccadic eye movements: prelude activity in the superior colliculus during a direction-discrimination task. J. Neurophysiol. 86, 2543–2558 (2001).
Roitman, J. D. & Shadlen, M. N. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J. Neurosci. 22, 9475–9489 (2002).
Hanks, T. D. & Shadlen, M. N. Microstimulation of macaque area LIP affects decision making in a motion discrimination task. Soc. Neurosci. Abstr. 20.9 (2004).
Bisley, J. W. & Goldberg, M. E. Neuronal activity in the lateral intraparietal area and spatial attention. Science 299, 81–86 (2003).
Platt, M. L. & Glimcher, P. W. Neural correlates of decision variables in parietal cortex. Nature 400, 233–238 (1998).
Kahneman, D. & Tversky, A. (eds) Choices, Values, and Frames (Cambridge Univ. Press, Cambridge, 2000).
Berridge, K. C. & Robinson, T. E. Parsing reward. Trends Neurosci. 26, 507–513 (2003). Discusses the distinct psychological components of reward and their underlying neural substrates. Emphasizes the distinction between systems devoted to the affective ('liking') and motivational ('wanting') aspects of reward, and between motivational processes based on simple Pavlovian associations and those involving more cognitive representations of value.
Olds, J. & Milner, P. M. Positive reinforcement produced by electrical stimulation of septal area and other regions of the rat brain. J. Comp. Physiol. Psychol. 47, 419–427 (1954).
Shizgal, P. Neural basis of utility estimation. Curr. Opin. Neurobiol. 7, 198–208 (1997). Summarizes findings from a series of elegant experiments involving BSR in the rat that indicate the existence of a final common neural representation for reward information.
Wise, R. A. & Bozarth, M. A. A psychomotor stimulant theory of addiction. Psychol. Rev. 94, 469–492 (1987).
Hyman, S. E. & Malenka, R. C. Addiction and the brain: the neurobiology of compulsion and its persistence. Nature Rev. Neurosci. 2, 695–703 (2001).
Kelley, A. E. Memory and addiction. Shared neural circuitry and molecular mechanisms. Neuron 44, 161–167 (2004).
Berridge, K. C. & Robinson, T. E. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Rev. 28, 309–369 (1998).
Garris, P. A. et al. Dissociation of dopamine release in the nucleus accumbens from intracranial self-stimulation. Nature 398, 67–69 (1999).
Pecina, S., Cagniard, B., Berridge, K. C., Aldridge, J. W. & Zhuang, X. Hyperdopaminergic mutant mice have higher 'wanting' but not 'liking' for sweet rewards. J. Neurosci. 23, 9395–9402 (2003).
Liu, Z. et al. DNA targeting of rhinal cortex D2 receptor protein reversibly blocks learning of cues that predict reward. Proc. Natl Acad. Sci. USA 101, 12336–12341 (2004).
Schultz, W., Apicella, P. & Ljungberg, T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci. 13, 900–913 (1993).
Schultz, W. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1–27 (1998).
Hollerman, J. R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nature Neurosci. 1, 304–309 (1998).
Sutton, R. S. & Barto, A. G. Reinforcement Learning (MIT Press, Cambridge, Massachusetts, 1998).
Dayan, P. & Abbott, L. F. Theoretical Neuroscience Ch. 9 (MIT Press, Cambridge, Massachusetts, 2001).
Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997). Reviews evidence indicating that midbrain dopamine neurons signal an error in the prediction of future reward (see references 35–37). Proposes that the function of these neurons is particularly well described by a specific class of reinforcement learning algorithms, and shows how a model that uses a dopamine-like signal to implement such an algorithm can learn to predict future rewards and guide action selection.
Schultz, W. Getting formal with dopamine and reward. Neuron 36, 241–263 (2002).
Montague, P. R., Dayan, P., Person, C. & Sejnowski, T. Bee foraging in uncertain environments using predictive Hebbian learning. Nature 377, 725–728 (1995).
Barto, A. G. in Models of Information Processing in the Basal Ganglia (eds Houk, J. C., Davis, J. L. & Beiser, D. G.) 215–232 (MIT Press, Cambridge, Massachusetts, 1995).
Kawagoe, R., Takikawa, Y. & Hikosaka, O. Expectation of reward modulates cognitive signals in the basal ganglia. Nature Neurosci. 1, 411–416 (1998).
Lauwereyns, J., Watanabe, K., Coe, B. & Hikosaka, O. A neural correlate of response bias in monkey caudate nucleus. Nature 418, 413–417 (2002).
Gold, J. I. Linking reward expectation to behavior in the basal ganglia. Trends Neurosci. 26, 12–14 (2003).
Montague, P. R. & Berns, G. S. Neural economics and the biological substrates of valuation. Neuron 36, 265–284 (2002). Among the first of a series of recent attempts (see also references 41,48,49) to integrate our emerging understanding of the role of dopamine with other reward-related neural signals for the purpose of developing a coherent framework for understanding the neural basis of reward and valuation.
McClure, S. M., Daw, N. D. & Montague, P. R. A computational substrate for incentive salience. Trends Neurosci. 26, 423–428 (2003).
Montague, P. R., Hyman, S. E. & Cohen, J. D. Computational roles for dopamine in behavioral control. Nature 431, 760–767 (2004).
Nakahara, H., Itoh, H., Kawagoe, R., Takikawa, Y. & Hikosaka, O. Dopamine neurons can represent context-dependent prediction error. Neuron 41, 269–280 (2004).
Rolls, E. T., Critchley, H., Mason, R. & Wakeman, E. A. Orbitofrontal cortex neurons: role in olfactory and visual association learning. J. Neurophysiol. 75, 1970–1978 (1996).
Tremblay, L. & Schultz, W. Relative reward preference in primate orbitofrontal cortex. Nature 398, 704–708 (1999).
Tremblay, L. & Schultz, W. Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. J. Neurophysiol. 83, 1877–1885 (2000).
Watanabe, M. Reward expectancy in primate prefrontal neurons. Nature 382, 629–632 (1996).
Leon, M. I. & Shadlen, M. N. Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24, 415–425 (1999).
Kobayashi, S., Lauwereyns, J., Koizumi, M., Sakagami, M. & Hikosaka, O. Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J. Neurophysiol. 87, 1488–1498 (2002).
Wallis, J. D. & Miller, E. K. Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur. J. Neurosci. 18, 2069–2081 (2003).
Roesch, M. R. & Olson, C. R. Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. J. Neurophysiol. 90, 1766–1789 (2003).
Roesch, M. R. & Olson, C. R. Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304, 307–310 (2004).
Shidara, M. & Richmond, B. J. Anterior cingulate: single neuron signals related to the degree of reward expectancy. Science 296, 1709–1711 (2002).
McCoy, A. N., Crowley, J. C., Haghighian, G., Dean, H. L. & Platt, M. L. Saccade reward signals in posterior cingulate cortex. Neuron 40, 1031–1040 (2003).
Ito, S., Stuphorn, V., Brown, J. W. & Schall, J. D. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302, 120–122 (2003).
Fuster, J. in Cerebral Cortex (eds Peter, J. & Jones, E.) 151–177 (Plenum, New York, 1985).
Segraves, M. A. & Goldberg, M. E. Functional properties of corticotectal neurons in the monkey's frontal eye field. J. Neurophysiol. 58, 1387–1419 (1987).
Funahashi, S., Bruce, C. J. & Goldman-Rakic, P. S. Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex. J. Neurophysiol. 61, 331–349 (1989).
Romo, R., Brody, C. D., Hernandez, A. & Lemus, L. Neuronal correlates of parametric working memory in the prefrontal cortex. Nature 339, 470–473 (1999).
Gnadt, J. W. & Andersen, R. A. Memory related motor planning activity in posterior parietal cortex of macaque. Exp. Brain Res. 70, 216–220 (1988).
Barash, S., Bracewell, R. M., Fogassi, L., Gnadt, J. W. & Andersen, R. A. Saccade-related activity in the lateral intraparietal area. I. Temporal properties; comparison with area 7a. J. Neurophysiol. 66, 1095–1108 (1991).
Colby, C. L., Duhamel, J. R. & Goldberg, M. E. Visual, presaccadic, and cognitive activation of single neurons in monkey intraparietal area. J. Neurophysiol. 76, 2841–2852 (1996).
Bechara, A., Tranel, D., Damasio, H. & Damasio, A. R. Deciding advantageously before knowing the advantageous strategy. Science 275, 1293–1295 (1997).
Iversen, S. D. & Mishkin, M. Perseverative interference in monkeys following selective lesions of the inferior prefrontal convexity. Exp. Brain Res. 11, 376–386 (1970).
Rolls, E. T. The orbitofrontal cortex and reward. Cereb. Cortex 10, 284–294 (2000).
Damasio, A. R. Descartes' Error: Emotion, Reason, and the Human Brain (Grosset/Putnam, New York, 1994).
Musallam, S., Corneil, B. D., Greger, B., Scherberger, H. & Andersen, R. A. Cognitive control signals for neural prosthetics. Science 305, 258–262 (2004).
Schultz, W. Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioral ecology. Curr. Opin. Neurobiol. 14, 139–147 (2004). An up-to-date survey of the types and locations of reward-related signals that have been documented using neurophysiological techniques.
Barraclough, D. J., Conroy, M. L. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game. Nature Neurosci. 7, 404–410 (2004).
Sugrue, L. P., Corrado, G. S. & Newsome, W. T. Matching behavior and the representation of value in the parietal cortex. Science 304, 1782–1787 (2004).
Dorris, M. C. & Glimcher, P. W. Activity in posterior parietal cortex is correlated with the relative subjective desirability of action. Neuron 44, 365–378 (2004). References 76–78 describe studies that all used a novel free-choice approach to investigate the behavioural and neural basis of value-based decision making in awake monkeys. These papers are discussed at length in the main text of this review.
Erev, I. & Roth, A. E. Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria. Am. Econ. Rev. 88, 848–881 (1998).
Platt, M. L. & Glimcher, P. W. Response fields of intraparietal neurons quantified with multiple saccadic targets. Exp. Brain Res. 121, 65–75 (1998).
Gottlieb, J. P., Kusunoki, M. & Goldberg, M. E. The representation of visual salience in monkey parietal cortex. Nature 391, 481–484 (1998).
Colby, C. L. & Goldberg, M. E. Space and attention in parietal cortex. Annu. Rev. Neurosci. 22, 319–349 (1999).
Glimcher, P. W. The neurobiology of visual-saccadic decision making. Annu. Rev. Neurosci. 26, 133–179 (2003).
Glimcher, P. W., Dorris, M. C. & Bayer, H. M. Physiological utility theory and the neuroeconomics of choice. Games Econ. Behav. (in the press).
Gold, J. I. & Shadlen, M. N. Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward. Neuron 36, 299–308 (2002).
Gold, J. I. & Shadlen, M. N. Representation of a perceptual decision in developing oculomotor commands. Nature 404, 390–394 (2000).
Shadlen, M. N. Pursuing commitments. Nature Neurosci. 5, 819–821 (2002).
Horwitz, G. D., Batista, A. P. & Newsome, W. T. Representation of an abstract perceptual decision in macaque superior colliculus. J. Neurophysiol. 91, 2281–2296 (2004).
Rizzolatti, G. & Craighero, L. The mirror-neuron system. Annu. Rev. Neurosci. 27, 169–192 (2004).
Maunsell, J. H. Neuronal representations of cognitive state: reward of attention? Trends Cogn. Sci. 8, 261–265 (2004).
Koch, C. & Ullman, S. Shifts in selective visual attention: toward the underlying neural circuitry. Hum. Neurobiol. 4, 219–227 (1985).
Itti, L. & Koch, C. Computational modeling of visual attention. Nature Rev. Neurosci. 2, 194–203 (2001).
Kahneman, D. in Les Prix Nobel 2002 (ed. Frangsmyr, T.) 416–499 (2002). A clear summary of the pioneering work of Kahneman and Tversky on the psychology of valuation and choice, which emphasizes the importance of intuitive over deliberative decision making and challenges basic economic assumptions of rationality.
Dayan, P. & Balleine, B. W. Reward, motivation, and reinforcement learning. Neuron 36, 285–298 (2002).
Dickinson, A. in Animal Learning and Cognition (ed. Mackintosh, N. J.) 45–79 (Academic, Orlando, 1994).
Nevin, J. A. Analyzing Thorndike's law of effect: the question of stimulus-response bonds. J. Exp. Anal. Behav. 72, 447–450 (1999).
Colwill, R. M. & Rescorla, R. A. Instrumental responding remains sensitive to reinforcer devaluation after extensive training. J. Exp. Psychology Anim. Behav. Process. 11, 520–536 (1985).
Nash, J. F. Equilibrium points in n-person games. Proc. Natl Acad. Sci. USA 36, 48–49 (1950).
Holt, C. A. & Roth, A. E. The Nash equilibrium: a perspective. Proc. Natl Acad. Sci. USA 101, 3999–4002 (2004).
Herrnstein, R. J. Relative and absolute strength of response as a function of frequency of reinforcement. J. Exp. Anal. Behav. 4, 267–272 (1961).
Stephens, D. W. & Krebs, J. R. Foraging Theory (Princeton Univ. Press, Princeton, New Jersey, 1986).
Baum, W. M. Optimization and the matching law as accounts of instrumental behavior. J. Exp. Anal. Behav. 36, 387–401 (1981).
Gallistel, C. R. & Gibbon, J. Time, rate and conditioning. Psychol. Rev. 107, 289–344 (2000). A refreshing, if controversial, look at the subject of operant choice. Argues for distinct decision mechanisms in choice situations that involve opting for the single best alternative, compared with those settings in which an animal must decide in what proportion to allocate its time or responses between alternatives.
Heyman, G. M. A Markov model description of changeover probabilities on concurrent variable-interval schedules. J. Exp. Anal. Behav. 31, 41–51 (1979).
Green, D. M. & Swets, J. A. Signal Detection Theory and Psychophysics (Wiley, New York, 1966). A seminal work that contributed the key theoretical foundation to research on perceptual decision making.
Van Essen, D. C. in The Visual Neurosciences (eds Chalupa, L. & Werner, J. S.) 507–521 (MIT Press, Cambridge, Massachusetts, 2004).
Van Essen, D. C. et al. An integrated software suite for surface-based analyses of cerebral cortex. J. Am. Med. Inform. Assoc. 8, 443–459 (2001).
Acknowledgements
We thank several colleagues who provided critical comments and suggestions during the preparation of this review: S. Baccus, C. Fiorillo, B. Linkenhoker, J. Reppas, A. Rorie and M. Shadlen. We also thank R. Gallistel of Rutgers University who taught us a great deal about matching behaviour and helped us design the oculomotor matching task used in our own studies. L.P.S. was supported by a Stanford Graduate Fellowship and by the Medical Scientist Training Program at the Johns Hopkins University School of Medicine. G.S.C. was also supported by a Stanford Graduate Fellowship and is currently supported by a National Research Service Award predoctoral fellowship from the National Institute of Mental Health. W.T.N. is an Investigator of the Howard Hughes Medical Institute and is also support by the National Eye Institute.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Related links
Rights and permissions
About this article
Cite this article
Sugrue, L., Corrado, G. & Newsome, W. Choosing the greater of two goods: neural currencies for valuation and decision making. Nat Rev Neurosci 6, 363–375 (2005). https://doi.org/10.1038/nrn1666
Published:
Issue Date:
DOI: https://doi.org/10.1038/nrn1666
This article is cited by
-
Orbitofrontal cortex control of striatum leads economic decision-making
Nature Neuroscience (2023)
-
Uncertainty modulates visual maps during noninstrumental information demand
Nature Communications (2022)
-
The increased analgesic efficacy of cold therapy after an unsuccessful analgesic experience is associated with inferior parietal lobule activation
Scientific Reports (2022)
-
Zebrafish capable of generating future state prediction error show improved active avoidance behavior in virtual reality
Nature Communications (2021)
-
Neurocomputational mechanisms underlying the subjective value of information
Communications Biology (2021)