Spiking Neural Networks: A Survey
Импульсные нейронные сети: обзор
2022-01-01
SCID: 54.1/jhthsy78
Discuss with AI
Efficient deep learningEvent-driven computationNeuromorphic computingSpiking neural networksSurrogate gradient learning
Figures from the paper
Abstract (AI)
The field of Deep Learning (DL) has seen a remarkable series of developments with increasingly accurate and robust algorithms. However, the increase in performance has been accompanied by an increase in the parameters, complexity, and training and inference time of the models, which means that we are rapidly reaching a point where DL may no longer be feasible. On the other hand, some specific applications need to be carefully considered when developing DL models due to hardware limitations or power requirements. In this context, there is a growing interest in efficient DL algorithms, with Spiking Neural Networks (SNNs) being one of the most promising paradigms. Due to the inherent asynchrony and sparseness of spike trains, these types of networks have the potential to reduce power consumption while maintaining relatively good performance. This is attractive for efficient DL and if successful, could replace traditional Artificial Neural Networks (ANNs) in many applications. However, despite significant progress, the performance of SNNs on benchmark datasets is often lower than that of traditional ANNs. Moreover, due to the non-differentiable nature of their activation functions, it is difficult to train SNNs with direct backpropagation, so appropriate training strategies must be found. Nevertheless, significant efforts have been made to develop competitive models. This survey covers the main ideas behind SNNs and reviews recent trends in learning rules and network architectures, with a particular focus on biologically inspired strategies. It also provides some practical considerations of state-of-the-art SNNs and discusses relevant research opportunities.
Key Findings
1
Recent research has developed increasingly competitive SNN models through new learning rules, network architectures, and biologically inspired methods.
2
SNNs are promising for hardware- and power-constrained applications, but their benchmark performance often remains below that of traditional artificial neural networks.
3
Spiking neural networks exploit asynchronous and sparse spike trains, offering potential reductions in power consumption for deep learning applications.
4
The non-differentiable activation functions of SNNs make direct backpropagation difficult, requiring specialized training strategies.
5
The survey reviews SNN fundamentals, state-of-the-art practical considerations, recent methodological trends, and open research opportunities.
Research Object
Spiking Neural Networks (SNNs)
Research Subject
Their energy-efficient deep-learning performance, trainability, learning rules, and network architectures, particularly biologically inspired strategies
Publication Details
Publication Date
2022-01-01
Journal
Publisher
ISSN
Cited by
251
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai12
Deep Residual Learning for Image Recognition2016
ImageNet classification with deep convolutional neural networks2017
Very Deep Convolutional Networks for Large-Scale Image Recognition2014
ImageNet: A large-scale hierarchical image database2009
Gradient-based learning applied to document recognition1998
PyTorch: An Imperative Style, High-Performance Deep Learning Library2019
Learning Multiple Layers of Features from Tiny Images2024
The Organization of Behavior: A Neuropsychological Theory1950
Neuronal Dynamics2014
Training Spiking Neural Networks Using Lessons From Deep Learning2023
Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better2023
SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks2018