GLM: General Language Model Pretraining with Autoregressive Blank Infilling

GLM: предварительное обучение общей языковой модели с авторегрессионным заполнением пропусков
Xiao Liu, Jiezhong Qiu, Ming Ding, Zhilin Yang, Yujie Qian, Zhengxiao Du, Jie Tang
2022-01-01
SCID:  54.1/kc4nwarc Discuss with AI
GLMautoregressive blank infillinggeneral language model pretraininglanguage model pretraining
Abstract (AI)
Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, Jie Tang. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.
Key Findings
1
GLM combines blank infilling with autoregressive generation, enabling bidirectional context understanding while supporting sequence generation.
2
The method is presented as a unified pretraining approach for natural language understanding and generation tasks.
3
The paper introduces GLM, a general language model pretraining framework based on autoregressive blank infilling.
Research Object

general language models pretrained with autoregressive blank infilling

Research Subject

the effectiveness of autoregressive blank-infilling pretraining for language modeling

Publication Details
Publication Date
2022-01-01
Journal
Publisher
ISSN
Cited by
954
Access Type
Author Information
Authors
Xiao Liu
Jiezhong Qiu
Ming Ding
Zhilin Yang
Yujie Qian
Zhengxiao Du
Jie Tang
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%