Channel Pruning for Accelerating Very Deep Neural Networks
Обрезка каналов для ускорения очень глубоких нейронных сетей
2017-07-19
SCID: 54.1/qug4nzeh
Discuss with AI
LASSO regression based channel selectionResNet and Xception accelerationVGG-16 accelerationchannel pruningleast squares reconstruction
Figures from the paper
Abstract (AI)
In this paper, we introduce a new channel pruning method to accelerate very deep convolutional neural networks.Given a trained CNN model, we propose an iterative two-step algorithm to effectively prune each layer, by a LASSO regression based channel selection and least square reconstruction. We further generalize this algorithm to multi-layer and multi-branch cases. Our method reduces the accumulated error and enhance the compatibility with various architectures. Our pruned VGG-16 achieves the state-of-the-art results by 5x speed-up along with only 0.3% increase of error. More importantly, our method is able to accelerate modern networks like ResNet, Xception and suffers only 1.4%, 1.0% accuracy loss under 2x speed-up respectively, which is significant. Code has been made publicly available.
Key Findings
1
Algorithm generalized to multi-layer and multi-branch network architectures, reducing accumulated error and improving compatibility.
2
Channel selection is performed by LASSO regression and channel reconstruction via least squares.
3
Code for the method has been made publicly available.
4
Introduced a channel pruning method for accelerating very deep convolutional neural networks using an iterative two-step algorithm.
5
Method accelerates modern networks: ResNet and Xception achieve ~2x speed-up with 1.4% and 1.0% accuracy loss respectively.
6
Pruned VGG-16 achieves 5x speed-up with only 0.3% increase in error, representing state-of-the-art results.
Research Object
Very deep convolutional neural networks (trained CNN models) subject to channel pruning
Research Subject
Channel pruning method and its effect on inference acceleration and accuracy (layer-wise LASSO-based channel selection and least-squares reconstruction, including multi-layer/multi-branch generalization) resulting in speed-up and accuracy trade-offs
Publication Details
Publication Date
2017-07-19
Journal
Publisher
ISSN
Cited by
353
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai7
Deep Residual Learning for Image Recognition2016
Very Deep Convolutional Networks for Large-Scale Image Recognition2014
Scikit-learn: Machine Learning in Python2012
ImageNet: A large-scale hierarchical image database2009
Gradient-based learning applied to document recognition1998
Learning Multiple Layers of Features from Tiny Images2024
Xception: Deep Learning with Depthwise Separable Convolutions2016