Efficient Parallel Split Learning Over Resource-Constrained Wireless Edge Networks
Эффективное параллельное разделённое обучение в беспроводных периферийных сетях с ограниченными ресурсами
2024-01-26
SCID: 54.1/gpt2btps
Discuss with AI
cut layer selectionlast-layer gradient aggregationparallel split learningsubchannel allocationwireless edge networks
Figures from the paper
Abstract (AI)
The increasingly deeper neural networks hinder the democratization of privacy-enhancing distributed learning, such as federated learning (FL), to resource-constrained devices. To overcome this challenge, in this paper, we advocate the integration of edge computing paradigm and parallel split learning (PSL), allowing multiple edge devices to offload substantial training workloads to an edge server via layer-wise model split. By observing that existing PSL schemes incur excessive training latency and a large volume of data transmissions, we propose an innovative PSL framework, namely, efficient parallel split learning (EPSL), to accelerate model training. To be specific, EPSL parallelizes client-side model training andreduces the dimension of activations' gradientsfor backpropagation (BP) vialast-layer gradient aggregation, leading to a significant reduction in server-side training and communication latency. Moreover, by considering the heterogeneous channel conditions and computing capabilities at edge devices, we jointly optimize subchannel allocation, power control, and cut layer selection to minimize the per-round latency. Simulation results show that the proposed EPSL framework significantly decreases the training latency needed to achieve a target accuracy compared with the state-of-the-art benchmarks, and the tailored resource management and layer split strategy can considerably reduce latency than the counterpart without optimization.
Key Findings
1
EPSL jointly optimizes subchannel allocation, transmit-power control, and model cut-layer selection under heterogeneous device capabilities and wireless channels.
2
Last-layer gradient aggregation substantially lowers server-side training and communication latency compared with existing parallel split learning schemes.
3
Optimized resource management and layer-split selection considerably reduce latency compared with an otherwise equivalent scheme without optimization.
4
Simulations show EPSL requires significantly less training latency to reach target accuracy than state-of-the-art benchmarks.
5
The proposed efficient parallel split learning (EPSL) framework accelerates training by parallelizing client-side computation and aggregating last-layer gradients to reduce backpropagation data dimensions.
Research Object
parallel split learning over resource-constrained wireless edge networks, involving multiple edge devices and an edge server
Research Subject
training latency and communication efficiency of parallel split learning, including the effects of layer-wise model splitting, last-layer gradient aggregation, and joint wireless resource and cut-layer optimization
Publication Details
Publication Date
2024-01-26
Journal
Publisher
ISSN
Cited by
170
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai7
Deep Residual Learning for Image Recognition2016
ImageNet classification with deep convolutional neural networks2017
Gradient-based learning applied to document recognition1998
SplitFed: When Federated Learning Meets Split Learning2022
Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications2021
Split Learning Over Wireless Networks: Parallel Design and Resource Management2023
Split Learning in 6G Edge Networks2024