On the Opportunities and Risks of Foundation Models
О возможностях и рисках фундаментальных моделей
2021-08-16
SCID: 54.1/utusbyn3
Discuss with AI
emergent capabilitiesfoundation modelsmodel homogenizationscale in deep learningsociotechnical impacts
Figures from the paper
Abstract (AI)
AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities,and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.
Key Findings
1
Addressing opportunities and risks of foundation models requires deep interdisciplinary research because their impacts are fundamentally sociotechnical, spanning legal, ethical, economic, and environmental domains.
2
Current understanding of how foundation models work, when they fail, and their full capabilities is limited, necessitating further research.
3
Foundation models are large-scale, broadly trained models (e.g., BERT, DALL-E, GPT-3) that are adaptable to many downstream tasks across modalities such as language, vision, robotics, reasoning, and human interaction.
4
Homogenization around foundation models provides strong leverage for many applications but risks propagating the foundation model's defects to all adapted downstream models.
5
Scale yields new emergent capabilities not present in smaller models, increasing effectiveness across diverse tasks but also creating unknown behaviors and failure modes.
Research Object
Foundation models (large-scale pretrained models such as BERT, DALL-E, GPT-3)
Research Subject
Opportunities, capabilities, technical principles, risks, applications, societal impacts, emergent behaviors, and failure modes of foundation models when adapted to downstream tasks
Publication Details
Publication Date
2021-08-16
Journal
Publisher
ISSN
Cited by
2672
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai20
ImageNet classification with deep convolutional neural networks2017
Very Deep Convolutional Networks for Large-Scale Image Recognition2014
ImageNet: A large-scale hierarchical image database2009
PyTorch: An Imperative Style, High-Performance Deep Learning Library2019
Distilling the Knowledge in a Neural Network2015
Learning Multiple Layers of Features from Tiny Images2024
Book Review: Pattern Recognition and Machine Learning2007
A survey on Image Data Augmentation for Deep Learning2019
Representation Learning with Contrastive Predictive Coding2018
Causality: Models, Reasoning, and Inference2009
The Graph Neural Network Model2008
Transformers: State-of-the-Art Natural Language Processing2020
Towards A Rigorous Science of Interpretable Machine Learning2017
Overcoming catastrophic forgetting in neural networks2017
AI4People—An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations2018
Recurrent neural network based language model2010
The potential for artificial intelligence in healthcare2019
Dense Passage Retrieval for Open-Domain Question Answering2020
Why Are There Still So Many Jobs? The History and Future of Workplace Automation2015
Transformer-XL: Attentive Language Models beyond a Fixed-Length Context2019
Cited by20
A Survey on Evaluation of Large Language Models2024
Large language models encode clinical knowledge2023
Generative Agents: Interactive Simulacra of Human Behavior2023
Generative AI2023
Can Large Language Models Transform Computational Social Science?2023
Academic Integrity considerations of AI Large Language Models in the post-pandemic era: ChatGPT and beyond2023
Augmenting large language models with chemistry tools2024
Self-Supervised Speech Representation Learning: A Review2022
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction2023
Towards a standard for identifying and managing bias in artificial intelligence2022
GPT (Generative Pre-Trained Transformer)— A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directions2024
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT2023
Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation2024
The ethics of ChatGPT in medicine and healthcare: a systematic review on Large Language Models (LLMs)2024
Can Generative AI improve social science?2024
Computing Education in the Era of Generative AI2024
Vision-language models for medical report generation and visual question answering: a review2024
Human and AI Ensembles: When Can They Work?2021
ChatMOF: an artificial intelligence system for predicting and generating metal-organic frameworks using large language models2024
Review of large vision models and visual prompt engineering2023