GPT-NeoX-20B: An Open-Source Autoregressive Language Model
GPT-NeoX-20B: Открытая авторегрессивная языковая модель
2022-01-01
SCID: 54.1/k9xvvsje
Discuss with AI
BigScience workshopGPT-NeoX-20Blarge language modelsmodel release 2022open-source autoregressive language model
Figures from the paper
Abstract (AI)
Sidney Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, Usvsn Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, Samuel Weinbach. Proceedings of BigScience Episode #5 -- Workshop on Challenges & Perspectives in Creating Large Language Models. 2022.
Key Findings
1
GPT-NeoX-20B is presented as a freely available alternative to proprietary large language models, supporting reproducible research and wider access.
2
The paper introduces GPT-NeoX-20B, an open-source 20-billion-parameter autoregressive language model.
3
The work documents the design and implementation of a large-scale autoregressive transformer trained and released by the authors’ collaboration.
Research Object
GPT-NeoX-20B autoregressive language model
Research Subject
Design, implementation, and evaluation of an open-source 20-billion-parameter autoregressive language model
Publication Details
Publication Date
2022-01-01
Journal
Publisher
ISSN
Cited by
410
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest