Toward Causal Representation Learning
К стремлению к обучению причинных представлений
2021-02-26
SCID: 54.1/apmzzk37
Discuss with AI
causal inferencecausal representation learningdiscovery of high-level causal variablesgraphical causalitytransfer and generalization
Figures from the paper
Abstract (AI)
The two fields of machine learning and graphical causality arose and are developed separately. However, there is, now, cross-pollination and increasing interest in both fields to benefit from the advances of the other. In this article, we review fundamental concepts of causal inference and relate them to crucial open problems of machine learning, including transfer and generalization, thereby assaying how causality can contribute to modern machine learning research. This also applies in the opposite direction: we note that most work in causality starts from the premise that the causal variables are given. A central problem for AI and causality is, thus, causal representation learning, that is, the discovery of high-level causal variables from low-level observations. Finally, we delineate some implications of causality for machine learning and propose key research areas at the intersection of both communities.
Key Findings
1
Causal representation learning—the discovery of high-level causal variables from low-level observations—is identified as a central open problem.
2
Causality concepts can address core ML problems like transfer and generalization by relating causal inference to these challenges.
3
Most causal inference work assumes causal variables are given, highlighting a central gap in both AI and causality.
4
The paper proposes key interdisciplinary research directions at the intersection of causality and machine learning and outlines implications of causality for ML research.
5
There is increasing cross-pollination between machine learning and graphical causality, with mutual benefits for both fields.
Research Object
Causal representation learning (discovery of high-level causal variables from low-level observations)
Research Subject
Methods and principles for discovering and validating high-level causal variables and their causal relationships from observational data to support transfer, generalization, and causal inference in machine learning
Publication Details
Publication Date
2021-02-26
Journal
Publisher
ISSN
Cited by
1161
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai17
ImageNet classification with deep convolutional neural networks2017
ImageNet: A large-scale hierarchical image database2009
Statistical Learning Theory1999
Artificial intelligence: a modern approach1995
Representation Learning: A Review and New Perspectives2013
Representation Learning with Contrastive Predictive Coding2018
Intriguing properties of neural networks2013
Causality: Models, Reasoning, and Inference2009
Reinforcement Learning: A Survey1996
Speech recognition with deep recurrent neural networks2013
Semi-Supervised Learning2006
An overview of deep learning in medical imaging focusing on MRI2018
Support Vector Machines2020
A Theory of Causal Learning in Children: Causal Maps and Bayes Nets.2004
Interaction Networks for Learning about Objects, Relations and Physics2016
Evaluating the Effectiveness of Large Language Models in Representing Textual Descriptions of Geometry and Spatial Relations (Short Paper)2023
Qualitative Spatial Reasoning over Questions (Short Paper)2022
Cited by4
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next2022
Text Data Augmentation for Deep Learning2021
A comprehensive survey of deep learning for time series forecasting: architectural diversity and open challenges2025
AI, agentic models and lab automation for scientific discovery — the beginning of scAInce2025