Reducing the Dimensionality of Data with Neural Networks
Снижение размерности данных с помощью нейронных сетей
2006-07-27
SCID: 54.1/3q9byxxu
Discuss with AI
autoencoderdimensionality reductionmultilayer neural networkprincipal components analysis (PCA)weight initialization
Figures from the paper
Abstract (AI)
High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such "autoencoder" networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data.
Key Findings
1
An effective weight-initialization method is described that enables deep autoencoders to learn useful low-dimensional codes.
2
Deep autoencoder–learned codes outperform principal components analysis for dimensionality reduction according to the abstract.
3
Gradient descent fine-tuning of autoencoder weights is effective only when initial weights are near a good solution.
4
Multilayer neural networks with a small central layer can convert high-dimensional data into low-dimensional codes by reconstructing inputs.
Research Object
Deep autoencoder neural network trained to reconstruct high-dimensional input vectors with a small central (bottleneck) layer
Research Subject
Learning low-dimensional codes (dimensionality reduction) via effective weight initialization and gradient-descent fine-tuning to produce representations superior to principal component analysis
Publication Details
Publication Date
2006-07-27
Journal
Publisher
ISSN
Cited by
20937
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai1
Cited by20
Adam: A Method for Stochastic Optimization2014
Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks2017
Image-to-Image Translation with Conditional Adversarial Networks2017
Representation Learning: A Review and New Perspectives2013
Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups2012
Machine learning: Trends, perspectives, and prospects2015
Improving neural networks by preventing co-adaptation of feature detectors2012
Whatever next? Predictive brains, situated agents, and the future of cognitive science2013
Object Detection With Deep Learning: A Review2019
Least Squares Generative Adversarial Networks2017
Deep Learning in Medical Image Analysis2017
Deep Learning for Computer Vision: A Brief Review2018
Deep Learning for Anomaly Detection: A Review2021
Deep learning for healthcare: review, opportunities and challenges2017
Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks2016
Deep learning applications and challenges in big data analytics2015
Deep Learning for Generic Object Detection: A Survey2019
Recent advances and applications of machine learning in solid-state materials science2019
Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations2009
Deep Convolutional Neural Networks for Hyperspectral Image Classification2015