Split Learning Over Wireless Networks: Parallel Design and Resource Management

Раздельное обучение в беспроводных сетях: параллельное проектирование и управление ресурсами
Weihua Zhuang, Mushu Li, Xuemin Shen, Wen Rong Wu, Conghao Zhou, Kaige Qu, Xu Li, Weisen Shi
2023-02-08

device clusteringparallel model trainingresource managementsplit learningwireless networks
Split learning (SL) is a collaborative learning framework, which can train an artificial intelligence (AI) model between a device and an edge server by splitting the AI model into a device-side model and a server-side model at a cut layer. The existing SL approach conducts the training process sequentially across devices, which incurs significant training latency especially when the number of devices is large. In this paper, we design a novel SL scheme to reduce the training latency, namedCluster-basedParallelSL(CPSL) which conducts model training in a “first-parallel-then-sequential” manner. Specifically, the CPSL is to partition devices into several clusters, parallelly train device-side models in each cluster and aggregate them, and then sequentially train the whole AI model across clusters, thereby parallelizing the training process and reducing training latency. Furthermore, we propose a resource management algorithm to minimize the training latency of CPSL considering device heterogeneity and network dynamics in wireless networks. This is achieved by stochastically optimizing the cut layer selection, device clustering, and radio spectrum allocation. The proposed two-timescale algorithm can jointly make the cut layer selection decision in a large timescale and device clustering and radio spectrum allocation decisions in a small timescale. Extensive simulation results on non-independent and identically distributed data demonstrate that the proposed solution can greatly reduce the training latency as compared with the existing SL benchmarks, while adapting to network dynamics.
1
A resource-management algorithm jointly optimizes cut-layer selection, device clustering, and radio-spectrum allocation under device heterogeneity and wireless-network dynamics.
2
CPSL partitions devices into clusters, aggregates device-side models within each cluster, and then trains the complete model across clusters sequentially.
3
Simulations with non-independent and identically distributed data show that CPSL substantially outperforms existing split-learning benchmarks in training latency while adapting to changing network conditions.
4
The proposed Cluster-based Parallel Split Learning (CPSL) reduces training latency by parallelizing device-side training within clusters before sequentially processing clusters.
5
The proposed two-timescale algorithm makes cut-layer decisions on a large timescale and clustering and spectrum-allocation decisions on a small timescale.

Split learning training of AI models across heterogeneous wireless devices and edge servers

Parallelization and resource management for minimizing training latency under device heterogeneity and dynamic wireless network conditions

Publication Details
Publication Date
2023-02-08
Journal
Publisher
ISSN
Cited by
345
Access Type
Author Information
Authors
Weihua Zhuang
Mushu Li
Xuemin Shen
Wen Rong Wu
Conghao Zhou
Kaige Qu
Xu Li
Weisen Shi
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%