Skip to main content
SpringerLink
Log in

Distributed Adaptive Control: A Proposal on the Neuronal Organization of Adaptive Goal Oriented Behavior

  • Chapter
From Motor Learning to Interaction Learning in Robots

Part of the book series: Studies in Computational Intelligence ((SCI,volume 264))

Abstract

In behavioral motor coordination and interaction it is a fundamental challenge how an agent can learn to perceive and act in unknown and dynamic environments. At present, it is not clear how an agent can – without any explicitly predefined knowledge – acquire internal representations of the world while interacting with the environment. To meet this challenge, we propose a biologically based cognitive architecture called Distributed Adaptive Control (DAC). DAC is organized in three different, tightly coupled, layers of control: reactive, adaptive and contextual. DAC based systems are self-contained and fully grounded, meaning that they autonomously generate representations of their primary sensory inputs, hence bootstrapping their behavior form simple to advance interactions. Following this approach, we have previously identified a novel environmentally mediated feedback loop in the organization of perception and behavior, i.e. behavioral feedback. Additionally, we could demonstrated that the dynamics of the memory structure of DAC, acquired during a foraging task, are equivalent to a Bayesian description of foraging. In this chapter we present DAC in a concise form and show how it is allowing us to extend the different subsystems to more biophysical detailed models. These further developments of the DAC architecture, not only allow to better understand the biological systems, but moreover advance DACs behavioral capabilities and generality.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Anderson, J.R., Bothell, D., Byrne, M.D., Douglass, S., Lebiere, C., Qin, Y.: An integrated theory of the mind. Psychol. Rev. 111(4), 1036–1060 (2004)

    Article  Google Scholar 

  2. Bermudez i Badia, S., Pyk, P., Verschure, P.F.M.J.: A fly-locust based neuronal control system applied to an unmanned aerial vehicle: the invertebrate neuronal principles for course stabilization, altitude control and collision avoidance. Int. J. Robot Res. 26, 759–772 (2007)

    Article  Google Scholar 

  3. Bakin, J.S., Weinberger, N.M.: Classical conditioning induces CS-specific receptive field plasticity in the auditory cortex of the guinea pig. Brain Res. 536(1-2), 271–286 (1990)

    Article  Google Scholar 

  4. Bayes, M., Price, M.: An essay towards solving a problem in the doctrine of chances. Philos. Trans. R Soc. London 53, 370–418 (1763)

    Google Scholar 

  5. Becker, S., Plumbley, M.: Unsupervised neural network learning procedures for feature extraction and classification. Appl. Intell. 6(3), 185–203 (1996)

    Article  Google Scholar 

  6. Bell, A.J.: Levels and loops: the future of artificial intelligence and neuroscience. Philos. Trans. R Soc. Lond B Biol. Sci. 354(1392), 2013–2020 (1999)

    Article  Google Scholar 

  7. Berlau, K.M., Weinberger, N.M.: Learning strategy determines auditory cortical plasticity. Neurobiol. Learn. Mem. 89(2), 153–166 (2008)

    Article  Google Scholar 

  8. Bernardet, U.: The neurobiological basis of perception and behavior: the iqr large-scale neuronal system simulator and its application. Ph.D. thesis, University of Zurich (2007)

    Google Scholar 

  9. Bernardet, U., Bermúdez i Badia, S., Verschure, P.F.M.J.: A model for the neuronal substrate of dead reckoning and memory in arthropods: a comparative computational and behavioral study. Theory Biosci. 127(2) (2008)

    Google Scholar 

  10. Braitenberg, V.: Vehicles, experiments in synthetic psychology. MIT Press, Cambridge (1984)

    Google Scholar 

  11. Brooks, R.: Intelligence without representation. Artif. Intell. 47(991), 139–159 (1991)

    Article  Google Scholar 

  12. Brooks, R.: New approaches to robotics. Science 253(5025), 1227–1232 (1991)

    Article  Google Scholar 

  13. Clancey, W.: Situated Cognition: On human knowledge and computer representations. Cambridge University Press, Cambridge (1996)

    Google Scholar 

  14. Davis, H.: Underestimating the rat’s intelligence. Brain Res. Cogn. Brain Res. 3(3-4), 291–298 (1996)

    Article  Google Scholar 

  15. Duff, A., Wyss, R., Verschure, P.F.M.J.: Learning temporally stable representations from natural sounds: Temporal stability as a general objective underlying sensory processing. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D.P. (eds.) ICANN 2007. LNCS, vol. 4669, pp. 129–138. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  16. Edeline, J.M.: Learning-induced physiological plasticity in the thalamo-cortical sensory systems: a critical evaluation of receptive field plasticity, map changes and their potential mechanisms. Prog. Neurobiol. 57(2), 165–224 (1999)

    Article  Google Scholar 

  17. Edeline, J.M., Pham, P., Weinberger, N.M.: Rapid development of learning-induced receptive field plasticity in the auditory cortex. Behav. Neurosci. 107(4), 539–551 (1993)

    Article  Google Scholar 

  18. Foldiak, P.: Learning invariance from transformation sequences. Neural Comput. 3(2), 194–200 (1991)

    Article  Google Scholar 

  19. Franzius, M., Sprekeler, H., Wiskott, L.: Slowness and sparseness lead to place, head-direction, and spatial-view cells. PLoS Comput. Biol. 3(8), e166 (2007)

    Google Scholar 

  20. Gallistel, C.R.: The Organization of Learning. MIT Press, Cambridge (1990)

    Google Scholar 

  21. Galvan, V.V., Weinberger, N.M.: Long-term consolidation and retention of learning-induced tuning plasticity in the auditory cortex of the guinea pig. Neurobiol. Learn. Mem. 77(1), 78–108 (2002)

    Article  Google Scholar 

  22. Georgopoulos, A.: New concepts in generation of movement. Neuron 13, 257–268 (1994)

    Article  Google Scholar 

  23. Gibson, J.J.: The Ecological Approach to Visual Perception. Lawrence Erlbaum, New Jersey (1979)

    Google Scholar 

  24. Harnad, S.: The symbol grounding problem. Physica D 42(1-3), 335–346 (1990)

    Article  Google Scholar 

  25. Herbort, O., Butz, M.V., Pedersen, G.: The sure reach model for motor learning and control of a redundant arm: from modeling human behavior to applications in robots. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 85–106. Springer, Heidelberg (2010)

    Google Scholar 

  26. Hipp, J., Einhäuser, W., Conradt, J., König, P.: Learning of somatosensory representations for texture discrimination using a temporal coherence principle. Network 16(2-3), 223–238 (2005)

    Article  Google Scholar 

  27. Hofstötter, C., Mintz, M., Verschure, P.F.M.J.: The cerebellum in action: a simulation and robotics study. Eur. J. Neurosci. 16(7), 1361–1376 (2002)

    Article  Google Scholar 

  28. Homberg, U.: In search of the sky compass in the insect brain. Naturwissenschaften 91, 199–208 (2004)

    Article  Google Scholar 

  29. Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. U S A 79(8), 2554–2558 (1982)

    Article  MathSciNet  Google Scholar 

  30. Hoyer, P.O., Hyvärinen, A.: A multi-layer sparse coding network learns contour coding from natural images. Vision Res. 42(12), 1593–1605 (2002)

    Article  Google Scholar 

  31. Hurri, J., Hyvärinen, A.: Simple-cell-like receptive fields maximize temporal coherence in natural video. Neural Comput. 15(3), 663–691 (2003)

    Article  MATH  Google Scholar 

  32. Hyvarinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. J. Wiley, New York (2001)

    Book  Google Scholar 

  33. Klein, D.J., König, P., Körding, K.P.: Sparse spectrotemporal coding of sounds. Eurasip Jasp 3, 659–667 (2003)

    Google Scholar 

  34. Konorski, J.: Integrative activity of the brain: An interdisciplinary approach. University of Chicago Press, Chicago (1967)

    Google Scholar 

  35. Körding, K.P., Kayser, C., Einhäuser, W., König, P.: How are complex cell properties adapted to the statistics of natural stimuli? J. Neurophysiol. 91(1), 206–212 (2004)

    Article  Google Scholar 

  36. Laird, J.: Using a computer game to develop advanced AI. Computer 34(7), 70–75 (2001)

    Article  Google Scholar 

  37. Lewicki, M.S.: Efficient coding of natural sounds. Nat. Neurosci. 5(4), 356–363 (2002)

    Article  Google Scholar 

  38. MacDonall, J.S., Goodell, J., Juliano, A.: Momentary maximizing and optimal foraging theories of performance on concurrent VR schedules. Behav. Processes 72(3), 283–299 (2006)

    Article  Google Scholar 

  39. Mackintosh, N.J.: Conditioning and associative learning. Oxford psychology series. Clarendon Press, Oxford (1990) (Reprint)

    Google Scholar 

  40. Martinez-Cantin, R., de Freitas, N., Brochu, E., Castellanos, J., Doucet, A.: A Bayesian Exploration-Exploitation Approach for Optimal Online Sensing and Planning with a Visually Guided Mobile Robot. Auton Robots (in press, 2009)

    Google Scholar 

  41. McCarthy, J., Hayes, P.J.: Some philosophical problems from the standpoint of artificial intelligence. Mach. Intell. 4, 463–502 (1969)

    MATH  Google Scholar 

  42. Mitrovic, D., Klanke, S., Vijayakumar, S.: Adaptive optimal feedback control with learned internal dynamics models. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 65–83. Springer, Heidelberg (2010)

    Google Scholar 

  43. Montemerlo, M., Thrun, S.: FastSLAM: A Scalable Method for the Simultaneous Localization and Mapping Problem in Robotics (Springer Tracts in Advanced Robotics). Springer, New York (2007)

    MATH  Google Scholar 

  44. Newell, A.: Unified theories of cognition. Harvard University Press, Cambridge (1990)

    Google Scholar 

  45. Ohl, F.W., Scheich, H.: Learning-induced plasticity in animal and human auditory cortex. Curr. Opin. Neurobiol. 15(4), 470–477 (2005)

    Article  Google Scholar 

  46. Oja, E.: A simplified neuron model as a principal component analyzer. J. Math. Biol. 15(3), 267–273 (1982)

    Article  MATH  MathSciNet  Google Scholar 

  47. Oja, E., Ogawa, H., Wangviwattana, J.: Principal component analysis by homogeneous neural networks, Part I: The weighted subspace criterion. IEICE Trans. Inf. Syst. 75, 366–375 (1992)

    Google Scholar 

  48. Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381(6583), 607–609 (1996)

    Article  Google Scholar 

  49. O’Regan, J.K., Noe, A.: A sensorimotor account of vision and visual consciousness. Behav. Brain Sci. 24(5), 939–973 (2001)

    Article  Google Scholar 

  50. Pavlov, I.P.: Conditioned reflexes: an investigation of the physiological activity of the cerebral cortex. Oxford University Press, Oxford (1927)

    Google Scholar 

  51. Pfeifer, R., Scheier, C.: From perception to action: the right direction? In: Gaussier, P., Nicoud, J. (eds.) From Perception to Action Conference, 1994, Proceedings, Los Alamitos, California, pp. 1–11 (1994)

    Google Scholar 

  52. Pfeifer, R., Scheier, C.: Understanding Intelligence. MIT Press, Cambridge (1999)

    Google Scholar 

  53. Rao, R., Olshausen, B., Lewicki, M.: Probabilistic Models of the Brain: Perception and Neural Function. MIT Press, Cambridge (2002)

    Google Scholar 

  54. Rescorla, R., Wagner, A.: A theory of pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In: Black, A., Prokasy, W.F. (eds.) Classical Conditioning II: Current Research and Theory, pp. 64–99. Appleton Century Crofts, New York (1972)

    Google Scholar 

  55. Roberts, W.: Foraging by rats on a radial maze:learning, memory, and decision rules. In: Gormezano, I., Wasserman, E. (eds.) Learning and memory: The behavioral and biological substrates, pp. 7–24. Lawrence Erlbaum, New Jersey (1992)

    Google Scholar 

  56. Rosenblatt, F.: The perceptron: A probabilistic model for information storage and organization in the brain. Psychol. Rev. 65(6), 386–408 (1958)

    Article  MathSciNet  Google Scholar 

  57. Rutkowski, R.G., Weinberger, N.M.: Encoding of learned importance of sound by magnitude of representational area in primary auditory cortex. Proc. Natl. Acad. Sci. U S A 102(38), 13664–13669 (2005)

    Article  Google Scholar 

  58. Sanchez-Montanes, M.A., König, P., Verschure, P.F.M.J.: Learning sensory maps with real-world stimuli in real time using a biophysically realistic learning rule. IEEE Trans. Neural Netw. 13(3), 619–632 (2002)

    Article  Google Scholar 

  59. Schultz, W.: Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 57, 87–115 (2006)

    Article  Google Scholar 

  60. Simoncelli, E.P., Olshausen, B.A.: Natural image statistics and neural representation. Annu. Rev. Neurosci. 24, 1193–1216 (2001)

    Article  Google Scholar 

  61. Smith, E.C., Lewicki, M.S.: Efficient auditory coding. Nature 439(7079), 978–982 (2006)

    Article  Google Scholar 

  62. Sporns, O., Kötter, R.: Motifs in brain networks. PLoS Biol. 2(11), e369 (2004)

    Article  Google Scholar 

  63. Squire, L.R., Kandel, E.R.: Memory: From mind to molecules. Scientific American Library, New York (1999)

    Google Scholar 

  64. Sur, M., Leamey, C.A.: Development and plasticity of cortical areas and networks. Nat. Rev. Neurosci. 2(4), 251–262 (2001)

    Article  Google Scholar 

  65. Sutton, R., Barto, A.G.: Reinforcement learning: An Introduction. MIT Press, Cambridge (1998)

    Google Scholar 

  66. Thorndike, E.: Animal Intelligence. Macmillan, New York (1911)

    Google Scholar 

  67. Thorpe, S., Fize, D., Marlot, C.: Speed of processing in the human visual system. Nature 381(6582), 520–522 (1996)

    Article  Google Scholar 

  68. Tovee, M.J., Rolls, E.T., Treves, A., Bellis, R.P.: Information encoding and the responses of single neurons in the primate temporal visual cortex. J. Neurophysiol. 70(2), 640–654 (1993)

    Google Scholar 

  69. Tversky, A., Slovic, B., Kahneman, B.: Judgment under uncertainty: heuristics and biases. Cambridge University Press, Cambridge (2001)

    Google Scholar 

  70. Varela, F., Thompson, E., Rosch, E.: The Embodied Mind: Cognitive Science and Human Experience. MIT Press, Cambridge (1991)

    Google Scholar 

  71. Vernon, D., Metta, G., Sandini, G.: A survey of artificial cognitive systems: Implications for the autonomous development of mental capabilities in computational agents. IEEE Trans. Evol. Comput. 11(2), 151–180 (2007)

    Article  Google Scholar 

  72. Verschure, P., Mintz, M.: A real-time model of the cerebellar circuitry underlying classical conditioning: A combined simulation and robotics study. Neurocomputing 38(40), 1019–1024 (2001)

    Article  Google Scholar 

  73. Verschure, P.F.M.J.: Synthetic epistemology: The acquisition, retention, and expression of knowledge in natural and synthetic systems. In: IEEE World Conference on Computational Intelligence, Proceedings, pp. 147–152. Anchorage, Alaska (1998)

    Google Scholar 

  74. Verschure, P.F.M.J., Althaus, P.: A real-world rational agent: Unifying old and new AI. Cogn. Sci. 27, 561–590 (2003)

    Article  Google Scholar 

  75. Verschure, P.F.M.J., Coolen, A.C.C.: Adaptive fields: distributed representations of classically conditioned associations. Network 2(2), 189–206 (1991)

    Article  Google Scholar 

  76. Verschure, P.F.M.J., Krose, B., Pfeifer, R.: Distributed adaptive control: The self-organization of structured behavior. Rob. Auton. Syst. 9, 181–196 (1993)

    Article  Google Scholar 

  77. Verschure, P.F.M.J., Pfeifer, R.: Categorization, representations, and the dynamics of system-environment interaction: a case study in autonomous systems. In: Meyer, J.A., Roitblat, H., Wilson, S. (eds.) From Animals to Animats: Proceedings of the Second International Conference on Simulation of Adaptive behavior, Honolulu, Hawaii, pp. 210–217. MIT Press, Cambridge (1992)

    Google Scholar 

  78. Verschure, P.F.M.J., Voegtlin, T., Douglas, R.J.: Environmentally mediated synergy between perception and behaviour in mobile robots. Nature 425(6958), 620–624 (2003)

    Article  Google Scholar 

  79. Voegtlin, T., Verschure, P.F.M.J.: What can robots tell us about brains? A synthetic approach towards the study of learning and problem solving. Rev. Neurosci. 10(3-4), 291–310 (1999)

    Google Scholar 

  80. Wallis, G.: Using spatio-temporal correlations to learn invariant object recognition. Neural Netw. 9(9), 1513–1519 (1996)

    Article  Google Scholar 

  81. Weinberger, N.M.: Learning-induced changes of auditory receptive fields. Curr. Opin. Neurobiol. 3(4), 570–577 (1993)

    Article  Google Scholar 

  82. Weinberger, N.M.: Physiological memory in primary auditory cortex: characteristics and mechanisms. Neurobiol. Learn. Mem. 70(1-2), 226–251 (1998)

    Article  Google Scholar 

  83. Wiskott, L., Sejnowski, T.J.: Slow feature analysis: unsupervised learning of invariances. Neural Comput. 14(4), 715–770 (2002)

    Article  MATH  Google Scholar 

  84. Wyss, R.: Sensory and motor coding in the organization of behavior. Ph.D. thesis, ETHZ (2003)

    Google Scholar 

  85. Wyss, R., König, P., Verschure, P.F.M.J.: Invariant representations of visual patterns in a temporal population code. Proc. Natl. Acad. Sci. U S A 100(1), 324–329 (2003)

    Article  Google Scholar 

  86. Wyss, R., König, P., Verschure, P.F.M.J.: A model of the ventral visual system based on temporal stability and local memory. PLoS Biol. 4(5), e120 (2006)

    Google Scholar 

  87. Wyss, R., Verschure, P.F.M.J., Konig, P.: Properties of a temporal population code. Rev. Neurosci. 14(1-2), 21–33 (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. INI Institute of Neuroinformatics, UNI-ETH Zürich, Winterthurerstrasse 190, CH-8057, Zürich, Switzerland

    Armin Duff

  2. ICREA Institució Catalana de Recerca i Estudis Avançats, Passeig Lluís Companys 23, E-08010, Barcelona, Spain

    Paul F. M. J. Verschure

  3. SPECS, IUA, Technology Department, Universitat Pompeu Fabra, Carrer de Roc Boronat 138, E-08018, Barcelona, Spain

    César Rennó-Costa, Encarni Marcos, Andre L. Luvizotto, Andrea Giovannucci, Marti Sanchez-Fibla & Ulysses Bernardet

Authors
  1. Armin Duff
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. César Rennó-Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Encarni Marcos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andre L. Luvizotto
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Andrea Giovannucci
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Marti Sanchez-Fibla
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ulysses Bernardet
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Paul F. M. J. Verschure
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institut des Systèmes Intelligents et de Robotique (CNRS UMR 7222), Université Pierre et Marie Curie Pyramide, Tour 55 Boîte courrier 173, 4 Place Jussieu, 75252, PARIS cedex 05, France

    Olivier Sigaud

  2. Dept. Schölkopf, Max-Planck Institute for Biological Cybernetics, Spemannstraße 38,Rm 223, 72076, Tübingen, Germany

    Jan Peters

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Duff, A. et al. (2010). Distributed Adaptive Control: A Proposal on the Neuronal Organization of Adaptive Goal Oriented Behavior. In: Sigaud, O., Peters, J. (eds) From Motor Learning to Interaction Learning in Robots. Studies in Computational Intelligence, vol 264. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-05181-4_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-05181-4_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-05180-7

  • Online ISBN: 978-3-642-05181-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation