GPT-NeoX-20B: An Open-Source Autoregressive Language Model
GPT-NeoX-20B: Открытая авторегрессивная языковая модель
2022-01-01
SCID: 54.1/k9xvvsje
Discuss with AI
BigScience workshopGPT-NeoX-20Blarge language modelsmodel release 2022open-source autoregressive language model
Figures from the paper
Abstract (AI)
Sidney Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, Usvsn Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, Samuel Weinbach. Proceedings of BigScience Episode #5 -- Workshop on Challenges & Perspectives in Creating Large Language Models. 2022.
Key Findings
1
GPT-NeoX-20B is presented as a freely available alternative to proprietary large language models, supporting reproducible research and wider access.
2
The paper introduces GPT-NeoX-20B, an open-source 20-billion-parameter autoregressive language model.
3
The work documents the design and implementation of a large-scale autoregressive transformer trained and released by the authors’ collaboration.
Research Object
GPT-NeoX-20B autoregressive language model
Research Subject
Design, implementation, and evaluation of an open-source 20-billion-parameter autoregressive language model
Publication Details
Publication Date
2022-01-01
Journal
Publisher
ISSN
Cited by
406
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai2
Cited by8
A Comprehensive Overview of Large Language Models2025
BloombergGPT: A Large Language Model for Finance2023
Large Language Models for Software Engineering: A Systematic Literature Review2024
A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges2024
Structured information extraction from scientific text with large language models2024
A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs)2024
A Survey on Large Language Models for Code Generation2025
Benchmark Evaluations, Applications, and Challenges of Large Vision Language Models: A Survey2025