Skip to main content
SpringerLink
Log in

The Importance of Self-excitation in Spiking Neural Networks Evolved to Recognize Temporal Patterns

  • Conference paper
  • First Online:
Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation (ICANN 2019)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11727))

Included in the following conference series:

  • 2993 Accesses

Abstract

Biological and artificial spiking neural networks process information by changing their states in response to the temporal patterns of input and of the activity of the network itself. Here we analyse very small networks, evolved to recognize three signals in a specific pattern (ABC) in a continuous temporal stream of signals (...CABCACB...). This task can be accomplished by networks with just four neurons (three interneurons and one output). We show that evolving the networks in the presence of noise and variation of the intervals of silence between signals biases the solutions towards networks that can maintain their states (a form of memory), while the majority of networks evolved without variable intervals between signals cannot do so. We demonstrate that in most networks, the evolutionary process leads to the presence of superfluous connections that can be pruned without affecting the ability of the networks to perform the task and, if the unpruned network can maintain memory, so does the pruned network. We then analyse how these small networks can perform their tasks, using a paradigm of finite state transducers. This analysis shows that self-excitatory loops (autapses) in these networks are crucial for both the recognition of the pattern and for memory maintenance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahissar, E., Arieli, A.: Figuring space by time. Neuron 32, 185–201 (2001)

    Article  Google Scholar 

  2. Anderson, J.S., Lampl, I., Gillespie, D.C., Ferster, D.: The contribution of noise to contrast invariance of orientation tuning in cat visual cortex. Science 290, 1968–1972 (2000)

    Article  Google Scholar 

  3. Bialek, W., Rieke, F., van Steveninck, R.R.d.R., Warland, D., et al.: Reading a neural code. In: Neural Information Processing Systems, pp. 36–43 (1989)

    Google Scholar 

  4. Burnstock, G.: Autonomic neurotransmission: 60 years since sir Henry Dale. Annu. Rev. Pharmacol. Toxicol. 49, 1–30 (2009)

    Article  Google Scholar 

  5. Decharms, R.C., Zador, A.: Neural representation and the cortical code. Annu. Rev. Neurosci. 23, 613–647 (2000)

    Article  Google Scholar 

  6. Destexhe, A., Rudolph, M., Fellous, J.M., Sejnowski, T.: Fluctuating synaptic conductances recreate in vivo-like activity in neocortical neurons. Neuroscience 107, 13–24 (2001)

    Article  Google Scholar 

  7. Destexhe, A., Paré, D.: Impact of network activity on the integrative properties of neocortical pyramidal neurons in vivo. J. Neurophysiol. 81, 1531–1547 (1999)

    Article  Google Scholar 

  8. Faisal, A.A., Selen, L.P., Wolpert, D.M.: Noise in the nervous system. Nat. Rev. Neurosci. 9, 292–303 (2008)

    Article  Google Scholar 

  9. Finn, I.M., Priebe, N.J., Ferster, D.: The emergence of contrast-invariant orientation tuning in simple cells of cat visual cortex. Neuron 54, 137–152 (2007)

    Article  Google Scholar 

  10. Florian, R.V.: Biologically inspired neural networks for the control of embodied agents. Center for Cognitive and Neural Studies (Cluj-Napoca, Romania), Tech. rep. Coneural-03-03 (2003)

    Google Scholar 

  11. Gerstner, W., Kempter, R., van Hemmen, J.L., Wagner, H.: A neuronal learning rule for sub-millisecond temporal coding. Nature 383, 76–78 (1996)

    Article  Google Scholar 

  12. Huxter, J., Burgess, N., Okeefe, J.: Independent rate and temporal coding in hippocampal pyramidal cells. Nature 425, 828–832 (2003)

    Article  Google Scholar 

  13. Jacobson, G., et al.: Subthreshold voltage noise of rat neocortical pyramidal neurones. J. Physiol. 564, 145–60 (2005)

    Article  Google Scholar 

  14. Joris, P., Yin, T.: A matter of time: internal delays in binaural processing. Trends Neurosci. 30, 70–78 (2007)

    Article  Google Scholar 

  15. Laurent, G.: Dynamical representation of odors by oscillating and evolving neural assemblies. Trends Neurosci. 19, 489–496 (1996)

    Article  Google Scholar 

  16. Maex, R., Steuber, V.: The first second: models of short-term memory traces in the brain. Neural Netw. 22, 1105–1112 (2009)

    Article  Google Scholar 

  17. Major, G., Tank, D.: Persistent neural activity: prevalence and mechanisms. Curr. Opin. Neurobiol. 14, 675–684 (2004)

    Article  Google Scholar 

  18. Marder, E.: Variability, compensation, and modulation in neurons and circuits. Proc. Nat. Acad. Sci. U.S.A. 108, 15542–15548 (2011)

    Article  Google Scholar 

  19. Natschläger, T., Maass, W.: Spiking neurons and the induction of finite state machines. Theoret. Comput. Sci. 287, 251–265 (2002)

    Article  MathSciNet  Google Scholar 

  20. Naud, R., Marcille, N., Clopath, C., Gerstner, W.: Firing patterns in the adaptive exponential integrate-and-fire model. Biol. Cybern. 99, 335–347 (2008)

    Article  MathSciNet  Google Scholar 

  21. Paré, D., Shink, E., Gaudreau, H., Destexhe, A., Lang, E.J.: Impact of spontaneous synaptic activity on the resting properties of cat neocortical pyramidal neurons in vivo. J. Neurophysiol. 79, 1450–1460 (1998)

    Article  Google Scholar 

  22. Prinz, A.A., Bucher, D., Marder, E.: Similar network activity from disparate circuit parameters. Nat. Neurosci. 7, 1345–1352 (2004)

    Article  Google Scholar 

  23. Rieke, F., Warland, D., de Ruyter van Steveninck, R., Bialek, W.: Spikes: Exploring the Neural Code. MIT Press, Cambridge (1999)

    MATH  Google Scholar 

  24. Rutishauser, U., Douglas, R.J.: State-dependent computation using coupled recurrent networks. Neural Comput. 21, 478–509 (2009)

    Article  MathSciNet  Google Scholar 

  25. Isaacson, J.S.: Odor representations in mammalian cortical circuits. Curr. Opin. Neurobiol. 20, 328–31 (2010)

    Article  Google Scholar 

  26. Saada, R., Miller, N., Hurwitz, I., Susswein, A.J.: Autaptic excitation elicits persistent activity and a plateau potential in a neuron of known behavioral function. Curr. Biol. 19, 479–84 (2009)

    Article  Google Scholar 

  27. Seung, H.S., Lee, D.D., Reis, B.Y., Tank, D.W.: The autapse: a simple illustration of short-term analog memory storage by tuned synaptic feedback. J. Comput. Neurosci. 9, 171–185 (2000)

    Article  Google Scholar 

  28. Sipser, M.: Introduction to the Theory of Computation. International Thomson Publishing, Stamford (1996)

    Book  Google Scholar 

  29. Stacey, W., Durand, D.: Stochastic resonance improves signal detection in hippocampal neurons. J. Neurophysiol. 83, 1394–1402 (2000)

    Article  Google Scholar 

  30. Steuber, V., De Schutter, E.: Rank order decoding of temporal parallel fibre input patterns in a complex Purkinje cell model. Neurocomputing 44–46, 183–188 (2002)

    Article  Google Scholar 

  31. Steuber, V., Willshaw, D.J.: Adaptive leaky integrator models of cerebellar Purkinje cells can learn the clustering of temporal patterns. Neurocomputing 26–27, 271–276 (1999)

    Article  Google Scholar 

  32. Steuber, V., Willshaw, D.: A biophysical model of synaptic delay learning and temporal pattern recognition in a cerebellar Purkinje cell. J. Comput. Neurosci. 17, 149–164 (2004)

    Article  Google Scholar 

  33. Steuber, V., Willshaw, D., Ooyen, A.V.: Generation of time delays: simplified models of intracellular signalling in cerebellar Purkinje cells. Netw. Comput. Neural Syst. 17, 173–191 (2006)

    Article  Google Scholar 

  34. Thorpe, S., Fize, D., Marlot, C.: Speed of processing in the human visual system. Nature 381, 520–522 (1996)

    Article  Google Scholar 

  35. Tino, P., Mills, A.J.S.: Learning beyond finite memory in recurrent networks of spiking neurons. Neural Comput. 18, 591–613 (2005)

    Article  MathSciNet  Google Scholar 

  36. Wang, C., et al.: Formation of autapse connected to neuron and its biological function. Complexity 2017, 1–9 (2017)

    MathSciNet  MATH  Google Scholar 

  37. Wiesenfeld, K., Moss, F.: Stochastic resonance and the benefits of noise: from ice ages to crayfish and squids. Nature 373, 33–36 (1995)

    Article  Google Scholar 

  38. Yaqoob, M., Wróbel, B.: Robust very small spiking neural networks evolved with noise to recognize temporal patterns. In: ALIFE 2018: Proceedings of the 2018 Conference on Artificial Life - MIT Press, pp. 665–672 (2018)

    Google Scholar 

  39. Yaqoob, M., Wróbel, B.: Very small spiking neural networks evolved to recognize a pattern in a continuous input stream. In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI) - IEEE, pp. 3496–3503 (2017)

    Google Scholar 

  40. Yaqoob, M., Wróbel, B.: Very small spiking neural networks evolved for temporal pattern recognition and robust to perturbed neuronal parameters. In: Artificial Neural Networks and Machine Learning - ICANN, pp. 322–331 (2018)

    Chapter  Google Scholar 

Download references

Acknowledgements

This work was supported by the Polish National Science Center (project EvoSN, UMO-2013/08/M/ST6/00922). MY acknowledges the support of the KNOW RNA Research Center in Poznan (No. 01/KNOW2/2014) and POWR.03.02.00-00-I006/17.

Author information

Authors and Affiliations

  1. Evolving Systems Laboratory, Adam Mickiewicz University in Poznan, Poznan, Poland

    Muhammad Yaqoob & Borys Wróbel

  2. University of Hertfordshire, Hatfield, UK

    Volker Steuber

Authors
  1. Muhammad Yaqoob
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Volker Steuber
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Borys Wróbel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Borys Wróbel .

Editor information

Editors and Affiliations

  1. Helmholtz Zentrum München - Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH), Neuherberg, Germany

    Igor V. Tetko

  2. Institute of Computer Science, Czech Academy of Sciences, Prague 8, Czech Republic

    Věra Kůrková

  3. Helmholtz Zentrum München - Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH), Neuherberg, Germany

    Pavel Karpov

  4. Helmholtz Zentrum München - Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH), Neuherberg, Germany

    Fabian Theis

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yaqoob, M., Steuber, V., Wróbel, B. (2019). The Importance of Self-excitation in Spiking Neural Networks Evolved to Recognize Temporal Patterns. In: Tetko, I., Kůrková, V., Karpov, P., Theis, F. (eds) Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation. ICANN 2019. Lecture Notes in Computer Science(), vol 11727. Springer, Cham. https://doi.org/10.1007/978-3-030-30487-4_59

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-30487-4_59

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-30486-7

  • Online ISBN: 978-3-030-30487-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation