SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks
SuperSpike: Обучение с учителем в многослойных спайковых нейронных сетях
2018-04-13
SCID: 54.1/vzhsa2yp
Discuss with AI
SuperSpikefeedback alignmentspiking neural networkssurrogate gradientthree-factor learning rule
Figures from the paper
Abstract (AI)
A vast majority of computation in the brain is performed by spiking neural networks. Despite the ubiquity of such spiking, we currently lack an understanding of how biological spiking neural circuits learn and compute in vivo, as well as how we can instantiate such capabilities in artificial spiking circuits in silico. Here we revisit the problem of supervised learning in temporally coding multilayer spiking neural networks. First, by using a surrogate gradient approach, we derive SuperSpike, a nonlinear voltage-based three-factor learning rule capable of training multilayer networks of deterministic integrate-and-fire neurons to perform nonlinear computations on spatiotemporal spike patterns. Second, inspired by recent results on feedback alignment, we compare the performance of our learning rule under different credit assignment strategies for propagating output errors to hidden units. Specifically, we test uniform, symmetric, and random feedback, finding that simpler tasks can be solved with any type of feedback, while more complex tasks require symmetric feedback. In summary, our results open the door to obtaining a better scientific understanding of learning and computation in spiking neural networks by advancing our ability to train them to solve nonlinear problems involving transformations between different spatiotemporal spike time patterns.
Key Findings
1
More complex tasks require symmetric feedback, indicating that credit-assignment strategy affects learning difficulty.
2
Simpler tasks can be learned with uniform, symmetric, or random feedback for propagating output errors to hidden units.
3
SuperSpike is a surrogate-gradient-derived, nonlinear voltage-based three-factor learning rule for supervised training of multilayer deterministic integrate-and-fire networks.
4
The method enables spiking neural networks to perform nonlinear computations on spatiotemporal spike patterns.
5
The work advances training of spiking networks for transformations between different spatiotemporal spike-time patterns.
Research Object
temporally coding multilayer spiking neural networks of deterministic integrate-and-fire neurons
Research Subject
supervised learning of nonlinear transformations between spatiotemporal spike patterns, including voltage-based three-factor learning and credit-assignment strategies
Publication Details
Publication Date
2018-04-13
Journal
Publisher
ISSN
Cited by
596
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai1
Cited by5
Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks2019
The Heidelberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks2020
Spiking Neural Networks Hardware Implementations and Challenges: A Survey2019
Spiking Neural Networks: A Survey2022
Federated Learning With Spiking Neural Networks2021