SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks

SuperSpike: Обучение с учителем в многослойных спайковых нейронных сетях
Surya Ganguli, Friedemann Zenke
2018-04-13

SuperSpikefeedback alignmentspiking neural networkssurrogate gradientthree-factor learning rule
A vast majority of computation in the brain is performed by spiking neural networks. Despite the ubiquity of such spiking, we currently lack an understanding of how biological spiking neural circuits learn and compute in vivo, as well as how we can instantiate such capabilities in artificial spiking circuits in silico. Here we revisit the problem of supervised learning in temporally coding multilayer spiking neural networks. First, by using a surrogate gradient approach, we derive SuperSpike, a nonlinear voltage-based three-factor learning rule capable of training multilayer networks of deterministic integrate-and-fire neurons to perform nonlinear computations on spatiotemporal spike patterns. Second, inspired by recent results on feedback alignment, we compare the performance of our learning rule under different credit assignment strategies for propagating output errors to hidden units. Specifically, we test uniform, symmetric, and random feedback, finding that simpler tasks can be solved with any type of feedback, while more complex tasks require symmetric feedback. In summary, our results open the door to obtaining a better scientific understanding of learning and computation in spiking neural networks by advancing our ability to train them to solve nonlinear problems involving transformations between different spatiotemporal spike time patterns.
1
More complex tasks require symmetric feedback, indicating that credit-assignment strategy affects learning difficulty.
2
Simpler tasks can be learned with uniform, symmetric, or random feedback for propagating output errors to hidden units.
3
SuperSpike is a surrogate-gradient-derived, nonlinear voltage-based three-factor learning rule for supervised training of multilayer deterministic integrate-and-fire networks.
4
The method enables spiking neural networks to perform nonlinear computations on spatiotemporal spike patterns.
5
The work advances training of spiking networks for transformations between different spatiotemporal spike-time patterns.

temporally coding multilayer spiking neural networks of deterministic integrate-and-fire neurons

supervised learning of nonlinear transformations between spatiotemporal spike patterns, including voltage-based three-factor learning and credit-assignment strategies

Publication Details
Publication Date
2018-04-13
Journal
Publisher
ISSN
Cited by
596
Access Type
Author Information
Authors
Surya Ganguli
Friedemann Zenke
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%