Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks

Обучение с использованием суррогатного градиента в спайковых нейронных сетях: перенос возможностей градиентной оптимизации в спайковые нейронные сети
Friedemann Zenke, Emre Neftci, Hesham Mostafa
2019-11-01

data-driven learningneuromorphic processorsspiking neural networkssurrogate gradient learningsynaptic plasticity
Spiking neural networks (SNNs) are nature's versatile solution to fault-tolerant, energy-efficient signal processing. To translate these benefits into hardware, a growing number of neuromorphic spiking NN processors have attempted to emulate biological NNs. These developments have created an imminent need for methods and tools that enable such systems to solve real-world signal processing problems. Like conventional NNs, SNNs can be trained on real, domain-specific data; however, their training requires the overcoming of a number of challenges linked to their binary and dynamical nature. This article elucidates step-by-step the problems typically encountered when training SNNs and guides the reader through the key concepts of synaptic plasticity and data-driven learning in the spiking setting. Accordingly, it gives an overview of existing approaches and provides an introduction to surrogate gradient (SG) methods, specifically, as a particularly flexible and efficient method to overcome the aforementioned challenges.
1
Spiking neural networks offer fault-tolerant and energy-efficient signal processing, motivating their implementation on neuromorphic hardware.
2
Surrogate gradient methods are presented as a flexible and efficient approach for overcoming the nondifferentiability challenges of spiking neural networks.
3
The article surveys existing SNN training approaches and provides a step-by-step introduction to surrogate gradient learning.
4
The article systematically explains the main obstacles in SNN training, including concepts related to synaptic plasticity and data-driven learning.
5
Training SNNs on real, domain-specific data is challenging because their binary spike outputs and dynamical behavior complicate gradient-based optimization.

spiking neural networks (SNNs)

training SNNs using surrogate-gradient methods, including challenges arising from their binary and dynamical nature

Publication Details
Publication Date
2019-11-01
Journal
Publisher
ISSN
Cited by
1864
Access Type
Author Information
Authors
Friedemann Zenke
Emre Neftci
Hesham Mostafa
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%