LLaMA: Open and Efficient Foundation Language Models
LLaMA: открытые и эффективные базовые языковые модели
2023-02-27
SCID: 54.1/vtr2pbdz
Discuss with AI
GPT-3LLaMAfoundation language modelslanguage model benchmarkspublicly available datasets
Figures from the paper
Abstract (AI)
We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. We release all our models to the research community.
Key Findings
1
LLaMA introduces foundation language models spanning 7B to 65B parameters.
2
LLaMA-13B outperforms the much larger GPT-3 175B model on most benchmarks.
3
LLaMA-65B is competitive with Chinchilla-70B and PaLM-540B, despite having fewer parameters than PaLM-540B.
4
The authors release all LLaMA models to the research community.
5
The models are trained on trillions of tokens using only publicly available datasets, without proprietary or inaccessible data.
Research Object
LLaMA collection of foundation language models (7B–65B parameter models)
Research Subject
Their training efficiency and benchmark performance when trained exclusively on publicly available datasets
Publication Details
Publication Date
2023-02-27
Journal
Publisher
ISSN
Cited by
23690
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai2
Cited by20
A Survey of Large Language Models2026
A Survey on Evaluation of Large Language Models2024
Lost in the Middle: How Language Models Use Long Contexts2024
A survey on large language model based autonomous agents2024
A Comprehensive Overview of Large Language Models2025
Unifying Large Language Models and Knowledge Graphs: A Roadmap2024
BloombergGPT: A Large Language Model for Finance2023
🧜Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models2025
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints2023
Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey2023
Autonomous chemical research with large language models2023
The future landscape of large language models in medicine2023
Active Retrieval Augmented Generation2023
Large Language Models for Software Engineering: A Systematic Literature Review2024
From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy2023
A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges2024
Retrieval-Augmented Generation for AI-Generated Content: A Survey2026
Structured information extraction from scientific text with large language models2024
Sentiment Analysis in the Era of Large Language Models: A Reality Check2024
Benchmarking Retrieval-Augmented Generation for Medicine2024