ImageNet classification with deep convolutional neural networks
Классификация ImageNet с использованием глубоких сверточных нейронных сетей
2017-05-24
SCID: 54.1/xbbm6eum
Discuss with AI
GPU-accelerated convolutionImageNet classificationdeep convolutional neural networksdropout regularizationtop-5 error rate
Figures from the paper
Abstract (AI)
We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully connected layers we employed a recently developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
Key Findings
1
A deep convolutional neural network classified 1.2 million ImageNet images across 1,000 classes, achieving 37.5% top-1 and 17.0% top-5 error rates.
2
A model variant won ILSVRC-2012 with a 15.3% top-5 error rate, outperforming the second-best entry’s 26.2%.
3
Dropout regularization effectively reduced overfitting in the fully connected layers.
4
Non-saturating neurons and an efficient GPU convolution implementation substantially accelerated network training.
5
The 60-million-parameter architecture used five convolutional layers, max-pooling, three fully connected layers, and a final 1,000-way softmax.
Research Object
A large, deep convolutional neural network trained on the ImageNet (ILSVRC) dataset
Research Subject
Deep convolutional neural network classification performance, including top-1 and top-5 error rates and the effects of network architecture and training techniques
Publication Details
Publication Date
2017-05-24
Journal
Publisher
ISSN
Cited by
112937
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai5
Deep Residual Learning for Image Recognition2016
ImageNet: A large-scale hierarchical image database2009
Learning Multiple Layers of Features from Tiny Images2024
Improving neural networks by preventing co-adaptation of feature detectors2012
Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations2009
Cited by20
Deep Residual Learning for Image Recognition2016
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows2021
Squeeze-and-Excitation Networks2018
Rethinking the Inception Architecture for Computer Vision2016
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
Distilling the Knowledge in a Neural Network2015
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale2020
Sequence to Sequence Learning with Neural Networks2014
Representation Learning: A Review and New Perspectives2013
A survey on Image Data Augmentation for Deep Learning2019
Convolutional Neural Networks for Sentence Classification2014
Representation Learning with Contrastive Predictive Coding2018
Aggregated Residual Transformations for Deep Neural Networks2017
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network2017
Learning Deep Features for Discriminative Localization2016
ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks2020
Machine learning: Trends, perspectives, and prospects2015
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices2018
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising2017
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions2021