Abstract
Reward-modulated learning rules for spiking neural networks have emerged, that have been demonstrated to solve a wide range of reinforcement learning tasks. Despite this, little work has aimed to classify spike patterns by the timing of output spikes. Here, we apply a rewardmaximising learning rule to teach a spiking neural network to classify input patterns by the latency of output spikes. Furthermore, we compare the performance of two escape rate functions that drive output spiking activity: the Arrhenius & Current (A&C) model and Exponential (EXP) model. We find A&C consistently outperforms EXP, and especially in terms of the time taken to converge in learning. We also show that jittering input patterns with a low noise amplitude leads to an improvement in learning, by reducing the variation in the performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barto, A., Sutton, R.: Reinforcement learning: An introduction. MIT Press, Cambridge (1998)
Farries, M., Fairhall, A.: Reinforcement learning with modulated spike timing dependent synaptic plasticity. Journal of Neurophysiology 98(6), 3648–3665 (2007)
Florian, R.V.: The chronotron: A neuron that learns to fire temporally precise spike patterns. PloS One 7(8), e40233 (2012)
Florian, R.: Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Computation 19(6), 1468–1502 (2007)
Frémaux, N., Sprekeler, H., Gerstner, W.: Functional requirements for reward-modulated spike-timing-dependent plasticity. The Journal of Neuroscience 30(40), 13326–13337 (2010)
Frémaux, N., Sprekeler, H., Gerstner, W.: Reinforcement learning using a continuous time actor-critic framework with spiking neurons. PLoS Computational Biology 9(4), e1003024 (2013)
Friedrich, J., Urbanczik, R., Senn, W.: Spatio-temporal credit assignment in neuronal population learning. PLoS Computational Biology 7(6), e1002092 (2011)
Gardner, B., Grüning, A.: Learning temporally precise spiking patterns through reward modulated spike-timing-dependent plasticity. In: Mladenov, V., Koprinkova-Hristova, P., Palm, G., Villa, A.E.P., Appollini, B., Kasabov, N. (eds.) ICANN 2013. LNCS, vol. 8131, pp. 256–263. Springer, Heidelberg (2013)
Gardner, B., Grüning, A.: Classifying patterns in a spiking neural network. In: Proceedings of the 22nd European Symposium on Artificial Neural Networks (ESANN 2014) (2014)
Gerstner, W., Kistler, W.: Spiking neuron models: Single neurons, populations, plasticity. Cambridge University Press, Cambridge (2002)
Herrmann, A., Gerstner, W.: Noise and the psth response to current transients: I. general theory and application to the integrate-and-fire neuron. Journal of Computational Neuroscience 11(2), 135–151 (2001)
Izhikevich, E.: Solving the distal reward problem through linkage of stdp and dopamine signaling. Cerebral Cortex 17(10), 2443–2452 (2007)
Legenstein, R., Pecevski, D., Maass, W.: A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback. PLoS Computational Biology 4(10), e1000180 (2008)
Pfister, J., Toyoizumi, T., Barber, D., Gerstner, W.: Optimal spike-timing-dependent plasticity for precise action potential firing in supervised learning. Neural Computation 18(6), 1318–1348 (2006)
Plesser, H., Gerstner, W.: Noise in integrate-and-fire neurons: from stochastic input to escape rates. Neural Computation 12(2), 367–384 (2000)
Rossum, M.: A novel spike distance. Neural Computation 13(4), 751–763 (2001)
Sporea, I., Grüning, A.: Supervised learning in multilayer spiking neural networks. Neural Computation 25(2), 473–509 (2013)
Van Rossum, M., Bi, G., Turrigiano, G.: Stable hebbian learning from spike timing-dependent plasticity. The Journal of Neuroscience 20(23), 8812–8821 (2000)
Vasilaki, E., Frémaux, N., Urbanczik, R., Senn, W., Gerstner, W.: Spike-based reinforcement learning in continuous state and action space: when policy gradient methods fail. PLoS Computational Biology 5(12), e1000586 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Gardner, B., Sporea, I., Grüning, A. (2014). Classifying Spike Patterns by Reward-Modulated STDP. In: Wermter, S., et al. Artificial Neural Networks and Machine Learning – ICANN 2014. ICANN 2014. Lecture Notes in Computer Science, vol 8681. Springer, Cham. https://doi.org/10.1007/978-3-319-11179-7_94
Download citation
DOI: https://doi.org/10.1007/978-3-319-11179-7_94
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11178-0
Online ISBN: 978-3-319-11179-7
eBook Packages: Computer ScienceComputer Science (R0)