SplitFed: When Federated Learning Meets Split Learning
SplitFed: когда федеративное обучение встречается с разделённым обучением
2022-06-28
SCID: 54.1/7nauaxps
Discuss with AI
PixelDPSplitFed learningdifferential privacyfederated learningsplit learning
Figures from the paper
Abstract (AI)
Federated learning (FL) and split learning (SL) are two popular distributed machine learning approaches. Both follow a model-to-data scenario; clients train and test machine learning models without sharing raw data. SL provides better model privacy than FL due to the machine learning model architecture split between clients and the server. Moreover, the split model makes SL a better option for resource-constrained environments. However, SL performs slower than FL due to the relay-based training across multiple clients. In this regard, this paper presents a novel approach, named splitfed learning (SFL), that amalgamates the two approaches eliminating their inherent drawbacks, along with a refined architectural configuration incorporating differential privacy and PixelDP to enhance data privacy and model robustness. Our analysis and empirical results demonstrate that (pure) SFL provides similar test accuracy and communication efficiency as SL while significantly decreasing its computation time per global epoch than in SL for multiple clients. Furthermore, as in SL, its communication efficiency over FL improves with the number of clients. Besides, the performance of SFL with privacy and robustness measures is further evaluated under extended experimental settings.
Key Findings
1
Like split learning, SFL’s communication efficiency relative to federated learning improves as the number of clients increases.
2
Pure SFL achieves test accuracy and communication efficiency similar to split learning while significantly reducing computation time per global epoch across multiple clients.
3
SplitFed learning (SFL) amalgamates federated learning and split learning to address SL’s slower training and FL’s weaker model privacy.
4
The proposed architecture incorporates differential privacy and PixelDP to strengthen data privacy and model robustness, with performance evaluated under extended experimental settings.
Research Object
splitfed learning (SFL), a distributed machine learning approach combining federated learning and split learning
Research Subject
SFL’s test accuracy, communication efficiency, computation time per global epoch, privacy, and robustness compared with federated and split learning
Publication Details
Publication Date
2022-06-28
Journal
Publisher
ISSN
Cited by
1141
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai9
Deep Residual Learning for Image Recognition2016
ImageNet classification with deep convolutional neural networks2017
Very Deep Convolutional Networks for Large-Scale Image Recognition2014
Gradient-based learning applied to document recognition1998
PyTorch: An Imperative Style, High-Performance Deep Learning Library2019
Advances and Open Problems in Federated Learning2020
Deep Learning with Differential Privacy2016
End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things2020
Detailed comparison of communication efficiency of split learning and federated learning2019
Cited by5
Split Learning Over Wireless Networks: Parallel Design and Resource Management2023
End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things2020
Split Learning in 6G Edge Networks2024
Efficient Parallel Split Learning Over Resource-Constrained Wireless Edge Networks2024
UDSem: A Unified Distributed Learning Framework for Semantic Communications Over Wireless Networks2023