Skip to main content
SpringerLink
Log in

Modeling Neuromodulation as a Framework to Integrate Uncertainty in General Cognitive Architectures

  • Conference paper
  • First Online:
Book cover Artificial General Intelligence (AGI 2016)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9782))

Included in the following conference series:

Abstract

One of the most critical properties of a versatile intelligent agent is its capacity to adapt autonomously to any change in the environment without overly complexifying its cognitive architecture. In this paper, we propose that understanding the role of neuromodulation in the brain is of central interest for this purpose. More precisely, we propose that an accurate estimation of the nature of uncertainty present in the environment is performed by specific brain regions and broadcast throughout the cerebral network by neuromodulators, resulting in appropriate changes in cerebral functioning and learning modes. Better understanding the principles of these mechanisms in the brain might tremendously inspire the field of Artificial General Intelligence. The original contribution of this paper is to relate the four major neuromodulators to four fundamental dimensions of uncertainty.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Anderson, J.R., Bothell, D., Byrne, M.D., Douglass, S., Lebiere, C., Qin, Y.: An integrated theory of the mind. Psychol. Rev. 111(4), 1036–1060 (2004). http://dx.doi.org/10.1037/0033-295x.111.4.1036

    Article  Google Scholar 

  2. Aston-Jones, G., Rajkowski, J., Kubiak, P.: Conditioned responses of monkey locus coeruleus neurons anticipate acquisition of discriminative behavior in a vigilance task. Neuroscience 80(3), 697–715 (1997).http://dx.doi.org/10.1016/s0306-4522(97)00060–2

    Google Scholar 

  3. Aston-Jones, G., Cohen, J.D.: An integrative theory of Locus Coeruleus-Norepinephrine function: adaptive gain and optimal performance. Ann. Rev. Neurosci. 28(1), 403–450 (2005). http://dx.doi.org/10.1146/annurev.neuro.28.061604.135709

    Article  Google Scholar 

  4. Aston-Jones, G., Rajkowski, J., Cohen, J.: Role of locus coeruleus in attention and behavioral flexibility. Biol. Psychiatry 46(9), 1309–1320 (1999). http://dx.doi.org/10.1016/s0006-3223(99)00140–7

    Article  Google Scholar 

  5. Balasubramani, P.P., Chakravarthy, V.S., Ravindran, B., Moustafa, A.A.: An extended reinforcement learning model of basal ganglia to understand the contributions of serotonin and dopamine in risk-based decision making, reward prediction, and punishment learning. Front. Comput. Neurosci. 8 (2014)

    Google Scholar 

  6. Berger-Tal, O., Nathan, J., Meron, E., Saltz, D.: The exploration-exploitation dilemma: a multidisciplinary framework. PLoS ONE 9(4), 1–8 (2014)

    Article  Google Scholar 

  7. Berridge, K.C., Robinson, T.E.: What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Rev. 28(3), 309–369 (1998). http://dx.doi.org/10.1016/s0165-0173(98)00019–8

    Article  Google Scholar 

  8. Bouret, S., Sara, S.J.: Network reset: a simplified overarching theory of locus coeruleus noradrenaline function. Trends Neurosci. 28(11), 574–582 (2005). http://dx.doi.org/10.1016/j.tins.2005.09.002

    Article  Google Scholar 

  9. Calandreau, L., Trifilieff, P., Mons, N., Costes, L., Marien, M., Marighetto, A., Micheau, J., Jaffard, R., Desmedt, A.: Extracellular hippocampal acetylcholine level controls amygdala function and promotes adaptive conditioned emotional response. J. Neurosci. Official J. Soc. Neurosci. 26(52), 13556–13566 (2006)

    Article  Google Scholar 

  10. Carrere, M., Alexandre, F.: A pavlovian model of the amygdala and its influence within the medial temporal lobe. Front. Syst. Neurosci. 9(41) (2015)

    Google Scholar 

  11. Cohen, J.D., McClure, S.M., Yu, A.J.: Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos. Trans. R. Soc. Lond. Ser. B, Biol. Sci. 362(1481), 933–942 (2007). http://dx.doi.org/10.1098/rstb.2007.2098

    Article  Google Scholar 

  12. Cools, R., Nakamura, K., Daw, N.D.: Serotonin and dopamine: unifying affective, activational, and decision functions. Neuropsychopharmacology 36(1), 98–113 (2011). http://dx.doi.org/10.1038/npp.2010.121

    Article  Google Scholar 

  13. Daw, N.D., Kakade, S., Dayan, P.: Opponent interactions between serotonin and dopamine. Neural Netw. 15(4–6), 603–616 (2002).http://dx.doi.org/10.1016/s0893-6080(02)00052–7

    Google Scholar 

  14. Daw, N.D., O’Doherty, J.P., Dayan, P., Seymour, B., Dolan, R.J.: Cortical substrates for exploratory decisions in humans. Nature 441(7095), 876–879 (2006). http://dx.doi.org/10.1038/nature04766

    Article  Google Scholar 

  15. Dayan, P.: Twenty-five lessons from computational neuromodulation. Neuron 76(1), 240–256 (2012). http://dx.doi.org/10.1016/j.neuron.2012.09.027

    Article  Google Scholar 

  16. Doya, K.: Metalearning and neuromodulation. Neural Netw. 15(4–6), 495–506 (2002). http://dx.doi.org/10.1016/s0893-6080(02)00044-8

    Article  Google Scholar 

  17. Doya, K., Samejima, K., Katagiri, K., Kawato, M.: Multiple model-based reinforcement learning. Neural Comp. 14(6), 1347–1369 (2002). http://neco.mitpress.org/cgi/content/abstract/14/6/1347

    Article  MATH  Google Scholar 

  18. Friston, K.: Functional integration and inference in the brain. Prog. Neurobiol. 68(2), 113–143 (2002). http://view.ncbi.nlm.nih.gov/pubmed/12450490

    Article  Google Scholar 

  19. Grossberg, S.: Adaptive resonance theory: how a brain learns to consciously attend, learn, and recognize a changing world. Neural Netw. 37, 1–47 (2013). http://dx.doi.org/10.1016/j.neunet.2012.09.017

    Article  Google Scholar 

  20. Haber, S., Fudge, J., McFarland, N.: Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20(6), 2369–2382 (2000)

    Google Scholar 

  21. Humphries, M.D., Khamassi, M., Gurney, K.: Dopaminergic control of the exploration-exploitation trade-off via the basal ganglia. Front. Neurosci. 6(9), 1–14 (2012)

    Google Scholar 

  22. Mannella, F., Gurney, K., Baldassarre, G.: The nucleus accumbens as a nexus between values and goals in goal-directed behavior: a review and a new hypothesis. Front. Behav. Neurosci. 7 (2013)

    Google Scholar 

  23. McClelland, J.L., McNaughton, B.L., O’Reilly, R.C.: Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol. Rev. 102(3), 419–457 (1995)

    Article  Google Scholar 

  24. McClure, S., Gilzenrat, M., Cohen, J.: An exploration-exploitation model based on norepinepherine and dopamine activity. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) Advances in Neural Information Processing Systems 18, pp. 867–874. MIT Press (2006). http://www.csbmb.princeton.edu/~smcclure/pdf/MGC_NIPS.pdf

  25. Niv, Y.: Cost, benefit, tonic, phasic: what do response rates tell us about dopamine and motivation. Annals of the New York Academy of Sciences 1104(1), 357–376 (2007). http://dx.doi.org/10.1196/annals.1390.018

    Article  Google Scholar 

  26. Pauli, W.M., Hazy, T.E., O’Reilly, R.C.: Expectancy, ambiguity, and behavioral flexibility: separable and complementary roles of the orbital frontal cortex and amygdala in processing reward expectancies. J. Cogn. Neurosci. 24(2), 351–366 (2011). http://dx.doi.org/10.1162/jocn_a_00155

    Article  Google Scholar 

  27. Pauli, W.M., O’Reilly, R.C.: Attentional control of associative learning-a possible role of the central cholinergic system. Brain Res. 1202, 43–53 (2008)

    Article  Google Scholar 

  28. Sara, S.J., Bouret, S.: Orienting and reorienting: the locus coeruleus mediates cognition through arousal. Neuron 76(1), 130–141 (2012). http://dx.doi.org/10.1016/j.neuron.2012.09.011

    Article  Google Scholar 

  29. Schultz, W.: Predictive reward signal of dopamine neurons. J. Neurophysiol. 80(1), 1–27 (1998). http://jn.physiology.org/content/80/1/1

    Google Scholar 

  30. Silver, D., Yang, Q., Li, L.: Lifelong machine learning systems: beyond learning algorithms. In: AAAI Spring Symposium Series (2013)

    Google Scholar 

  31. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)

    Google Scholar 

  32. Trappenberg, T.P.: Fundamentals of Computational Neuroscience, 2nd edn. Oxford University Press, Oxford (2009)

    MATH  Google Scholar 

  33. Yu, A.J., Dayan, P.: Uncertainty, neuromodulation, and attention. Neuron 46(4), 681–692 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Inria Bordeaux Sud-Ouest, 200 Avenue de la Vieille Tour, 33405, Talence, France

    Frédéric Alexandre & Maxime Carrere

  2. LaBRI, Université de Bordeaux, Bordeaux INP, CNRS, UMR 5800, Talence, France

    Frédéric Alexandre & Maxime Carrere

  3. Institut des Maladies Neurodégénératives, Université de Bordeaux, CNRS, UMR 5293, Bordeaux, France

    Frédéric Alexandre & Maxime Carrere

Authors
  1. Frédéric Alexandre
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maxime Carrere
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Frédéric Alexandre .

Editor information

Editors and Affiliations

  1. Galleria 1, IDSIA, Manno, Switzerland

    Bas Steunebrink

  2. Temple University, Phoenixville, Pennsylvania, USA

    Pei Wang

  3. Hong Kong Polytechnic University, Hong Kong, Hong Kong

    Ben Goertzel

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Alexandre, F., Carrere, M. (2016). Modeling Neuromodulation as a Framework to Integrate Uncertainty in General Cognitive Architectures. In: Steunebrink, B., Wang, P., Goertzel, B. (eds) Artificial General Intelligence. AGI 2016. Lecture Notes in Computer Science(), vol 9782. Springer, Cham. https://doi.org/10.1007/978-3-319-41649-6_33

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-41649-6_33

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-41648-9

  • Online ISBN: 978-3-319-41649-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation