BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BLOOM: многоязычная открытая языковая модель с 176 миллиардами параметров
2022-11-09
SCID: 54.1/spjs6kj8
Discuss with AI
176B-parameterBLOOMROOTS corpusdecoder-only Transformermultilingual language model
Figures from the paper
Abstract (AI)
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.
Key Findings
1
BLOOM achieves competitive performance on a wide variety of benchmarks compared to existing models.
2
BLOOM is a 176-billion-parameter open-access decoder-only Transformer language model developed by a large collaborative effort.
3
BLOOM was trained on the ROOTS corpus containing hundreds of sources across 59 languages (46 natural languages and 13 programming languages).
4
Multitask prompted finetuning further improves BLOOM's performance on evaluated tasks.
5
The authors publicly release BLOOM models and code under the Responsible AI License to support democratized research and applications.
Research Object
BLOOM, a 176-billion-parameter open-access decoder-only multilingual language model trained on the ROOTS corpus
Research Subject
Model performance and accessibility: evaluation of BLOOM's competitive performance across diverse benchmarks and the effects of multitask prompted finetuning, alongside its open-access release to democratize LLM technology
Publication Details
Publication Date
2022-11-09
Journal
Publisher
ISSN
Cited by
286
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
Cited by14
LLaMA: Open and Efficient Foundation Language Models2023
A Survey of Large Language Models2026
Large language models encode clinical knowledge2023
A Comprehensive Overview of Large Language Models2025
🧜Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models2025
Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models2023
A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly2024
A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges2024
Regulating ChatGPT and other Large Generative AI Models2023
Generative AI and ChatGPT in School Children’s Education: Evidence from a School Lesson2023
The path forward for large language models in medicine is open2024
RenAIssance: A Survey Into AI Text-to-Image Generation in the Era of Large Model2024
LLM-Commentator: Novel fine-tuning strategies of large language models for automatic commentary generation using football event data2024
Mapping the neuronal building blocks of human language with language models2026