Federated Learning With Spiking Neural Networks
Федеративное обучение с использованием импульсных нейронных сетей
2021-01-01
SCID: 54.1/yvrztfce
Discuss with AI
CIFAR-10 and CIFAR-100energy efficiencyfederated learningprivacy-preserving trainingspiking neural networks
Figures from the paper
Abstract (AI)
As neural networks get widespread adoption in resource-constrained embedded devices, there is a growing need for low-power neural systems. Spiking Neural Networks (SNNs) are emerging to be an energy-efficient alternative to the traditional Artificial Neural Networks (ANNs) which are known to be computationally intensive. From an application perspective, as federated learning involves multiple energy-constrained devices, there is a huge scope to leverage energy efficiency provided by SNNs. Despite its importance, there has been little attention on training SNNs on a large-scale distributed system like federated learning. In this paper, we bring SNNs to a more realistic federated learning scenario. Specifically, we design a federated learning method for training decentralized and privacy preserving SNNs. To validate the proposed method, we experimentally evaluate the advantages of SNNs on various aspects of federated learning with CIFAR10 and CIFAR100 benchmarks. We observe that SNNs outperform ANNs in terms of overall accuracy by over 15% when the data is distributed across a large number of clients in the federation while providing up to$4.3\times$energy efficiency. In addition to efficiency, we also analyze the sensitivity of the proposed federated SNN framework to data distribution among the clients, stragglers, and gradient noise and perform a comprehensive comparison with ANNs. The source code is available athttps://github.com/Intelligent-Computing-Lab-Yale/FedSNN.
Key Findings
1
Federated SNNs provide up to 4.3× energy efficiency compared with artificial neural networks in the evaluated settings.
2
On CIFAR10 and CIFAR100, SNNs achieve over 15% higher overall accuracy than ANNs when data is distributed across many federation clients.
3
The paper introduces a federated learning method for decentralized and privacy-preserving training of spiking neural networks (SNNs).
4
The study analyzes federated SNN sensitivity to client data distributions, stragglers, and gradient noise, with comprehensive comparisons against ANNs.
5
The work evaluates SNNs in a more realistic large-scale federated learning scenario and releases the source code publicly.
Research Object
decentralized, privacy-preserving Spiking Neural Networks (SNNs) trained in federated learning across energy-constrained devices
Research Subject
the accuracy, energy efficiency, and robustness of federated SNN training to client data distribution, stragglers, and gradient noise, compared with ANNs
Publication Details
Publication Date
2021-01-01
Journal
Publisher
ISSN
Cited by
91
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai4
Very Deep Convolutional Networks for Large-Scale Image Recognition2014
Learning Multiple Layers of Features from Tiny Images2024
Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks2019
SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks2018