Split Learning Over Wireless Networks: Parallel Design and Resource Management
Раздельное обучение в беспроводных сетях: параллельное проектирование и управление ресурсами
2023-02-08
SCID: 54.1/rzfk7949
Discuss with AI
device clusteringparallel model trainingresource managementsplit learningwireless networks
Figures from the paper
Abstract (AI)
Split learning (SL) is a collaborative learning framework, which can train an artificial intelligence (AI) model between a device and an edge server by splitting the AI model into a device-side model and a server-side model at a cut layer. The existing SL approach conducts the training process sequentially across devices, which incurs significant training latency especially when the number of devices is large. In this paper, we design a novel SL scheme to reduce the training latency, namedCluster-basedParallelSL(CPSL) which conducts model training in a “first-parallel-then-sequential” manner. Specifically, the CPSL is to partition devices into several clusters, parallelly train device-side models in each cluster and aggregate them, and then sequentially train the whole AI model across clusters, thereby parallelizing the training process and reducing training latency. Furthermore, we propose a resource management algorithm to minimize the training latency of CPSL considering device heterogeneity and network dynamics in wireless networks. This is achieved by stochastically optimizing the cut layer selection, device clustering, and radio spectrum allocation. The proposed two-timescale algorithm can jointly make the cut layer selection decision in a large timescale and device clustering and radio spectrum allocation decisions in a small timescale. Extensive simulation results on non-independent and identically distributed data demonstrate that the proposed solution can greatly reduce the training latency as compared with the existing SL benchmarks, while adapting to network dynamics.
Key Findings
1
A resource-management algorithm jointly optimizes cut-layer selection, device clustering, and radio-spectrum allocation under device heterogeneity and wireless-network dynamics.
2
CPSL partitions devices into clusters, aggregates device-side models within each cluster, and then trains the complete model across clusters sequentially.
3
Simulations with non-independent and identically distributed data show that CPSL substantially outperforms existing split-learning benchmarks in training latency while adapting to changing network conditions.
4
The proposed Cluster-based Parallel Split Learning (CPSL) reduces training latency by parallelizing device-side training within clusters before sequentially processing clusters.
5
The proposed two-timescale algorithm makes cut-layer decisions on a large timescale and clustering and spectrum-allocation decisions on a small timescale.
Research Object
Split learning training of AI models across heterogeneous wireless devices and edge servers
Research Subject
Parallelization and resource management for minimizing training latency under device heterogeneity and dynamic wireless network conditions
Publication Details
Publication Date
2023-02-08
Journal
Publisher
ISSN
Cited by
345
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai8
Gradient-based learning applied to document recognition1998
Learning Multiple Layers of Features from Tiny Images2024
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices2018
SplitFed: When Federated Learning Meets Split Learning2022
AI-Assisted Network-Slicing Based Next-Generation Wireless Networks2020
End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things2020
Detailed comparison of communication efficiency of split learning and federated learning2019
Split Learning for collaborative deep learning in healthcare2019