Practical Block-Wise Neural Network Architecture Generation

Практическая генерация архитектур нейронных сетей по блокам
Zhao Zhong, Junjie Yan, Wei Wu, Jing Shao, Cheng‐Lin Liu
2018-06-01

BlockQNNCIFAR-10Q-learningblock-wise network generationneural architecture search
Convolutional neural networks have gained a remarkable success in computer vision. However, most usable network architectures are hand-crafted and usually require expertise and elaborate design. In this paper, we provide a block-wise network generation pipeline called BlockQNN which automatically builds high-performance networks using the Q-Learning paradigm with epsilon-greedy exploration strategy. The optimal network block is constructed by the learning agent which is trained sequentially to choose component layers. We stack the block to construct the whole auto-generated network. To accelerate the generation process, we also propose a distributed asynchronous framework and an early stop strategy. The block-wise generation brings unique advantages: (1) it performs competitive results in comparison to the hand-crafted state-of-the-art networks on image classification, additionally, the best network generated by BlockQNN achieves 3.54% top-1 error rate on CIFAR-10 which beats all existing auto-generate networks. (2) in the meanwhile, it offers tremendous reduction of the search space in designing networks which only spends 3 days with 32 GPUs, and (3) moreover, it has strong generalizability that the network built on CIFAR also performs well on a larger-scale ImageNet dataset.
1
A distributed asynchronous framework and early-stopping strategy accelerate the neural architecture generation process.
2
BlockQNN achieves competitive image-classification performance compared with hand-crafted state-of-the-art networks.
3
BlockQNN automatically generates convolutional network blocks using Q-learning with epsilon-greedy exploration, then stacks the learned block into a complete architecture.
4
The best BlockQNN-generated network reaches a 3.54% top-1 error rate on CIFAR-10, outperforming existing automatically generated networks.
5
The block-wise approach substantially reduces search complexity, requiring only 3 days on 32 GPUs, while networks learned on CIFAR generalize effectively to ImageNet.

automatically generated convolutional neural network architectures for image classification

the performance, search-space reduction, generation efficiency, and generalizability of block-wise neural network architecture generation

Publication Details
Publication Date
2018-06-01
Journal
Publisher
ISSN
Access Type
Author Information
Authors
Zhao Zhong
Junjie Yan
Wei Wu
Jing Shao
Cheng‐Lin Liu
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%