Gradient-based learning applied to document recognition
Градиентное обучение для распознавания документов
1998-01-01
SCID: 54.1/sse9wjec
Discuss with AI
convolutional neural networksdocument recognitiongradient-based learninggraph transformer networkshandwritten character recognition
Figures from the paper
Abstract (AI)
Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.
Key Findings
1
A graph transformer network combining convolutional character recognizers achieved record accuracy on business and personal cheque reading and was deployed commercially at several million cheques per day.
2
Convolutional neural networks outperform other reviewed techniques on a standard handwritten digit recognition task because they address variability in two-dimensional shapes.
3
Experiments in online handwriting recognition demonstrate advantages of global training and the flexibility of graph transformer networks.
4
Gradient-based learning with multilayer neural networks can synthesize complex decision surfaces for high-dimensional handwritten character classification with minimal preprocessing.
5
Graph transformer networks enable multimodule document-recognition systems to be trained globally with gradient-based optimization against an overall performance measure.
Research Object
Handwritten character recognition systems (including convolutional neural network recognizers and graph transformer network–based multimodule systems) applied to document recognition
Research Subject
recognition accuracy and global end-to-end training performance for variable 2D handwritten patterns and multimodule document-reading systems
Publication Details
Publication Date
1998-01-01
Journal
Publisher
ISSN
Cited by
61757
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai1
Cited by20
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows2021
Sequence to Sequence Learning with Neural Networks2014
Representation Learning: A Review and New Perspectives2013
A survey on Image Data Augmentation for Deep Learning2019
Convolutional Neural Networks for Sentence Classification2014
A Comprehensive Survey on Graph Neural Networks2020
Improving neural networks by preventing co-adaptation of feature detectors2012
Graph neural networks: A review of methods and applications2020
Deep Learning with Differential Privacy2016
A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures2019
Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning2016
Deep Learning in Medical Image Analysis2017
A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions2021
Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting2018
Hierarchical Attention Networks for Document Classification2016
Artificial intelligence in healthcare: past, present and future2017
Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?2016
Deep Learning for Computer Vision: A Brief Review2018
Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks2017