Detailed comparison of communication efficiency of split learning and federated learning
Подробное сравнение эффективности коммуникации при раздельном и федеративном обучении
2019-09-18
SCID: 54.1/mtapuman
Discuss with AI
communication efficiencydistributed machine learningfederated learningmodel size and client countsplit learning
Figures from the paper
Abstract (AI)
We compare communication efficiencies of two compelling distributed machine learning approaches of split learning and federated learning. We show useful settings under which each method outperforms the other in terms of communication efficiency. We consider various practical scenarios of distributed learning setup and juxtapose the two methods under various real-life scenarios. We consider settings of small and large number of clients as well as small models (1M - 6M parameters), large models (10M - 200M parameters) and very large models (1 Billion-100 Billion parameters). We show that increasing number of clients or increasing model size favors split learning setup over the federated while increasing the number of data samples while keeping the number of clients or model size low makes federated learning more communication efficient.
Key Findings
1
Federated learning is more communication-efficient when the number of data samples increases while client count and model size remain low.
2
Increasing model size favors split learning, across models ranging from 1M–6M to 1B–100B parameters.
3
Neither method universally dominates; communication efficiency depends on client population, model size, and data-sample volume.
4
Split learning becomes more communication-efficient than federated learning as the number of clients increases.
5
The paper systematically compares the communication efficiency of split learning and federated learning across practical distributed-learning scenarios.
Research Object
Split learning and federated learning in distributed machine-learning setups
Research Subject
Communication efficiency under varying numbers of clients, model sizes, and data-sample counts
Publication Details
Publication Date
2019-09-18
Journal
Publisher
ISSN
Cited by
217
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
Cited by7
Advances and Open Problems in Federated Learning2020
Federated Learning in Mobile Edge Networks: A Comprehensive Survey2020
SplitFed: When Federated Learning Meets Split Learning2022
Edge Learning for B5G Networks With Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing2023
Split Learning Over Wireless Networks: Parallel Design and Resource Management2023
End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things2020
Split Learning for collaborative deep learning in healthcare2019