Deep Residual Learning for Image Recognition
Глубокое остаточное обучение для распознавания изображений
2016-06-01
SCID: 54.1/pfjygd2z
Discuss with AI
COCO object detectionILSVRC 2015ImageNetdeep residual learningresidual networks
Figures from the paper
Abstract (AI)
Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers - 8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.
Key Findings
1
An ensemble of these residual networks achieved 3.57% error on the ImageNet test set, winning 1st place in ILSVRC 2015 classification.
2
Deep residual networks formed the foundation for winning 1st places in ILSVRC & COCO 2015 tasks: ImageNet detection, localization, COCO detection, and segmentation.
3
Evaluated residual nets up to 152 layers on ImageNet (8× deeper than VGG) while having lower complexity.
4
Introduced a residual learning framework that reformulates layers to learn residual functions with reference to layer inputs, easing training of very deep networks.
5
Residual networks are empirically easier to optimize and can gain accuracy from considerably increased depth.
6
Using extremely deep residual representations produced a 28% relative improvement on the COCO object detection dataset.
Research Object
Deep residual neural networks (residual nets) for image recognition
Research Subject
Effect of residual learning formulation on training optimization, accuracy gains from substantially increased network depth, and performance on ImageNet, CIFAR-10 and COCO recognition/detection tasks
Publication Details
Publication Date
2016-06-01
Journal
Publisher
ISSN
Cited by
235342
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai7
ImageNet classification with deep convolutional neural networks2017
Very Deep Convolutional Networks for Large-Scale Image Recognition2014
Learning Multiple Layers of Features from Tiny Images2024
Neural networks for pattern recognition1994
Neural Networks for Pattern Recognition1995
Improving neural networks by preventing co-adaptation of feature detectors2012
Pattern Recognition and Neural Networks1996
Cited by20
ImageNet classification with deep convolutional neural networks2017
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows2021
Highly accurate protein structure prediction with AlphaFold2021
Squeeze-and-Excitation Networks2018
Semi-Supervised Classification with Graph Convolutional Networks2016
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer2019
Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks2017
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale2020
A survey on Image Data Augmentation for Deep Learning2019
Aggregated Residual Transformations for Deep Neural Networks2017
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network2017
A Comprehensive Survey on Graph Neural Networks2020
Non-local Neural Networks2018
ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks2020
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices2018
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising2017
A Survey of Large Language Models2026
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions2021
Graph neural networks: A review of methods and applications2020
TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation2021