End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things
Сквозная оценка федеративного и разделённого обучения для Интернета вещей
2020-09-01
SCID: 54.1/zj86khx2
Discuss with AI
Communication overheadFederated learningInternet of ThingsNon-IID dataSplit neural networks
Figures from the paper
Abstract (AI)
Federated learning (FL) and split neural networks (SplitNN) are state-of-art distributed machine learning techniques to enable machine learning without directly accessing raw data on clients or end devices. In theory, such distributed machine learning techniques have great potential in distributed applications, in which data are typically generated and collected at the client-side while the collected data should be processed by the application deployed at the server-side. However, there is still a significant gap in evaluating the performance of those techniques concerning their practicality in the Internet of Things (IoT)-enabled distributed systems constituted by resource-constrained devices. This work is the first attempt to provide empirical comparisons of FL and SplitNN in real-world IoT settings in terms of learning performance and device implementation overhead. We consider a variety of datasets, different model architectures, multiple clients, and various performance metrics. For the learning performance (i.e., model accuracy and convergence time), we empirically evaluate both FL and SplitNN under different types of data distributions such as imbalanced and non-independent and identically distributed (non-IID) data. We show that the learning performance of SplitNN is better than FL under an imbalanced data distribution but worse than FL under an extreme non-IID data distribution. For implementation overhead, we mount both FL and SplitNN on Raspberry Pi devices and comprehensively evaluate their overhead, including training time, communication overhead, power consumption, and memory usage. Our key observations are that under the IoT scenario where the communication traffic is the primary concern, FL appears to perform better over SplitNN because FL has a significantly lower communication overhead compared with SplitNN. However, our experimental results also demonstrate that neither FL or SplitNN can be applied to a heavy model, e.g., with several million parameters, on resource-constrained IoT devices because its training cost would be too expensive for such devices. Source code is released and available: https://github.com/Minki-Kim95/Federated-Learning-and-Split-Learning-with-raspberry-pi.
Key Findings
1
Neither Federated Learning nor SplitNN can be applied effectively to heavy models with several million parameters in resource-constrained IoT environments.
2
On Raspberry Pi devices, Federated Learning has significantly lower communication overhead than SplitNN, making it preferable when communication traffic is the primary constraint.
3
SplitNN achieves better learning performance than Federated Learning with imbalanced data, but performs worse under extremely non-IID data distributions.
4
The evaluation measures training time, communication overhead, power consumption, and memory usage across multiple datasets, model architectures, and clients.
5
The study provides an empirical end-to-end comparison of Federated Learning and SplitNN in real-world IoT settings, including learning performance and device overhead.
Research Object
Federated learning and split neural networks deployed on resource-constrained Internet of Things devices
Research Subject
Their learning performance and device implementation overhead under different data distributions, model architectures, and IoT resource constraints
Publication Details
Publication Date
2020-09-01
Journal
Publisher
ISSN
Cited by
305
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai3
Cited by4
SplitFed: When Federated Learning Meets Split Learning2022
Edge Learning for B5G Networks With Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing2023
Split Learning Over Wireless Networks: Parallel Design and Resource Management2023
Edge-Cloud Collaborative Computing on Distributed Intelligence and Model Optimization: A Survey2026