Spiking Neural Networks: A Survey

Импульсные нейронные сети: обзор
Jaime S. Cardoso, João D. Nunes, Marcelo Carvalho, Diogo Carneiro
2022-01-01

Efficient deep learningEvent-driven computationNeuromorphic computingSpiking neural networksSurrogate gradient learning
The field of Deep Learning (DL) has seen a remarkable series of developments with increasingly accurate and robust algorithms. However, the increase in performance has been accompanied by an increase in the parameters, complexity, and training and inference time of the models, which means that we are rapidly reaching a point where DL may no longer be feasible. On the other hand, some specific applications need to be carefully considered when developing DL models due to hardware limitations or power requirements. In this context, there is a growing interest in efficient DL algorithms, with Spiking Neural Networks (SNNs) being one of the most promising paradigms. Due to the inherent asynchrony and sparseness of spike trains, these types of networks have the potential to reduce power consumption while maintaining relatively good performance. This is attractive for efficient DL and if successful, could replace traditional Artificial Neural Networks (ANNs) in many applications. However, despite significant progress, the performance of SNNs on benchmark datasets is often lower than that of traditional ANNs. Moreover, due to the non-differentiable nature of their activation functions, it is difficult to train SNNs with direct backpropagation, so appropriate training strategies must be found. Nevertheless, significant efforts have been made to develop competitive models. This survey covers the main ideas behind SNNs and reviews recent trends in learning rules and network architectures, with a particular focus on biologically inspired strategies. It also provides some practical considerations of state-of-the-art SNNs and discusses relevant research opportunities.
1
Recent research has developed increasingly competitive SNN models through new learning rules, network architectures, and biologically inspired methods.
2
SNNs are promising for hardware- and power-constrained applications, but their benchmark performance often remains below that of traditional artificial neural networks.
3
Spiking neural networks exploit asynchronous and sparse spike trains, offering potential reductions in power consumption for deep learning applications.
4
The non-differentiable activation functions of SNNs make direct backpropagation difficult, requiring specialized training strategies.
5
The survey reviews SNN fundamentals, state-of-the-art practical considerations, recent methodological trends, and open research opportunities.

Spiking Neural Networks (SNNs)

Their energy-efficient deep-learning performance, trainability, learning rules, and network architectures, particularly biologically inspired strategies

Publication Details
Publication Date
2022-01-01
Journal
Publisher
ISSN
Cited by
251
Access Type
Author Information
Authors
Jaime S. Cardoso
João D. Nunes
Marcelo Carvalho
Diogo Carneiro
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%