Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors
Не считайте, а предсказывайте! Систематическое сравнение семантических векторов, основанных на подсчёте контекстов и на предсказании контекстов
2014-01-01
SCID: 54.1/kfjtjqx3
Discuss with AI
context-predicting modelscount-based semantic vectorsdistributional semanticslexical semantics tasksword embeddings
Figures from the paper
Abstract (AI)
Context-predicting models (more commonly known as embeddings or neural language models) are the new kids on the distributional semantics block.Despite the buzz surrounding these models, the literature is still lacking a systematic comparison of the predictive models with classic, count-vector-based distributional semantic approaches.In this paper, we perform such an extensive evaluation, on a wide range of lexical semantics tasks and across many parameter settings.The results, to our own surprise, show that the buzz is fully justified, as the context-predicting models obtain a thorough and resounding victory against their count-based counterparts.
Key Findings
1
Context-predicting models substantially outperform count-based models across the evaluated lexical semantics benchmarks.
2
Evaluation covers a wide range of lexical semantic tasks and numerous parameter settings.
3
The results provide strong empirical support for the effectiveness of predictive distributional semantic representations.
4
The study systematically compares context-predicting embeddings and neural language models with classic context-counting distributional semantic vectors.
Research Object
Context-counting and context-predicting distributional semantic models
Research Subject
Their comparative performance across lexical semantics tasks and parameter settings
Publication Details
Publication Date
2014-01-01
Journal
Publisher
ISSN
Cited by
1364
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest