End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things

Сквозная оценка федеративного и разделённого обучения для Интернета вещей
‪Surya Nepal‬, Seyit Ahmet Camtepe, Chandra Thapa, Yansong Gao, Minki Kim, Hyoungshick Kim, Alsharif Abuadbba, Yeonjae Kim, Kyuyeon Kim
2020-09-01

Communication overheadFederated learningInternet of ThingsNon-IID dataSplit neural networks
Federated learning (FL) and split neural networks (SplitNN) are state-of-art distributed machine learning techniques to enable machine learning without directly accessing raw data on clients or end devices. In theory, such distributed machine learning techniques have great potential in distributed applications, in which data are typically generated and collected at the client-side while the collected data should be processed by the application deployed at the server-side. However, there is still a significant gap in evaluating the performance of those techniques concerning their practicality in the Internet of Things (IoT)-enabled distributed systems constituted by resource-constrained devices. This work is the first attempt to provide empirical comparisons of FL and SplitNN in real-world IoT settings in terms of learning performance and device implementation overhead. We consider a variety of datasets, different model architectures, multiple clients, and various performance metrics. For the learning performance (i.e., model accuracy and convergence time), we empirically evaluate both FL and SplitNN under different types of data distributions such as imbalanced and non-independent and identically distributed (non-IID) data. We show that the learning performance of SplitNN is better than FL under an imbalanced data distribution but worse than FL under an extreme non-IID data distribution. For implementation overhead, we mount both FL and SplitNN on Raspberry Pi devices and comprehensively evaluate their overhead, including training time, communication overhead, power consumption, and memory usage. Our key observations are that under the IoT scenario where the communication traffic is the primary concern, FL appears to perform better over SplitNN because FL has a significantly lower communication overhead compared with SplitNN. However, our experimental results also demonstrate that neither FL or SplitNN can be applied to a heavy model, e.g., with several million parameters, on resource-constrained IoT devices because its training cost would be too expensive for such devices. Source code is released and available: https://github.com/Minki-Kim95/Federated-Learning-and-Split-Learning-with-raspberry-pi.
1
Neither Federated Learning nor SplitNN can be applied effectively to heavy models with several million parameters in resource-constrained IoT environments.
2
On Raspberry Pi devices, Federated Learning has significantly lower communication overhead than SplitNN, making it preferable when communication traffic is the primary constraint.
3
SplitNN achieves better learning performance than Federated Learning with imbalanced data, but performs worse under extremely non-IID data distributions.
4
The evaluation measures training time, communication overhead, power consumption, and memory usage across multiple datasets, model architectures, and clients.
5
The study provides an empirical end-to-end comparison of Federated Learning and SplitNN in real-world IoT settings, including learning performance and device overhead.

Federated learning and split neural networks deployed on resource-constrained Internet of Things devices

Their learning performance and device implementation overhead under different data distributions, model architectures, and IoT resource constraints

Publication Details
Publication Date
2020-09-01
Journal
Publisher
ISSN
Cited by
305
Access Type
Author Information
Authors
‪Surya Nepal‬
Seyit Ahmet Camtepe
Chandra Thapa
Yansong Gao
Minki Kim
Hyoungshick Kim
Alsharif Abuadbba
Yeonjae Kim
Kyuyeon Kim
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%