Intriguing properties of neural networks
Интригующие свойства нейронных сетей
2013-12-21
SCID: 54.1/uu3639ub
Discuss with AI
adversarial perturbationsdeep neural networkssemantic informationtransferabilityunit analysis
Figures from the paper
Abstract (AI)
Deep neural networks are highly expressive models that have recently achieved state of the art performance on speech and visual recognition tasks. While their expressiveness is the reason they succeed, it also causes them to learn uninterpretable solutions that could have counter-intuitive properties. In this paper we report two such properties. First, we find that there is no distinction between individual high level units and random linear combinations of high level units, according to various methods of unit analysis. It suggests that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks. Second, we find that deep neural networks learn input-output mappings that are fairly discontinuous to a significant extend. We can cause the network to misclassify an image by applying a certain imperceptible perturbation, which is found by maximizing the network's prediction error. In addition, the specific nature of these perturbations is not a random artifact of learning: the same perturbation can cause a different network, that was trained on a different subset of the dataset, to misclassify the same input.
Key Findings
1
Adversarial perturbations that maximize prediction error transfer across networks trained on different dataset subsets, indicating they reflect shared model properties rather than random artifacts.
2
Deep neural networks learn significantly discontinuous input-output mappings, enabling imperceptible perturbations to induce misclassification.
3
High-level semantic information is distributed across activation spaces rather than localized in individual neural-network units.
4
Random linear combinations of high-level units are indistinguishable from individual units under multiple unit-analysis methods.
Research Object
Deep neural networks
Research Subject
the semantic organization of high-level representations and the discontinuity and transferability of their input-output mappings
Publication Details
Publication Date
2013-12-21
Journal
Publisher
ISSN
Cited by
14020
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai2
Cited by20
Advances and Open Problems in Federated Learning2020
DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks2016
Randaugment: Practical automated data augmentation with a reduced search space2020
Convolutional neural networks: an overview and application in radiology2018
Deep learning with convolutional neural networks for EEG decoding and visualization2017
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence2023
A review of uncertainty quantification in deep learning: Techniques, applications and challenges2021
A survey on semi-supervised learning2019
Leakage and the reproducibility crisis in machine-learning-based science2023
Text Data Augmentation for Deep Learning2021
Artificial intelligence: A powerful paradigm for scientific research2021
Deep learning in bioinformatics2016
End-to-end Optimized Image Compression2016
Toward Causal Representation Learning2021
A Unifying Review of Deep and Shallow Anomaly Detection2021
Deep learning for AI2021
On instabilities of deep learning in image reconstruction and the potential costs of AI2020
Quo vadis artificial intelligence?2022
A Survey of Methods for Explaining Black Box Models2018
Explainable, trustworthy, and ethical machine learning for healthcare: A survey2022