Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors

Не считайте, а предсказывайте! Систематическое сравнение семантических векторов, основанных на подсчёте контекстов и на предсказании контекстов
Germán Kruszewski, Marco Baroni, Georgiana Dinu
2014-01-01

context-predicting modelscount-based semantic vectorsdistributional semanticslexical semantics tasksword embeddings
Context-predicting models (more commonly known as embeddings or neural language models) are the new kids on the distributional semantics block.Despite the buzz surrounding these models, the literature is still lacking a systematic comparison of the predictive models with classic, count-vector-based distributional semantic approaches.In this paper, we perform such an extensive evaluation, on a wide range of lexical semantics tasks and across many parameter settings.The results, to our own surprise, show that the buzz is fully justified, as the context-predicting models obtain a thorough and resounding victory against their count-based counterparts.
1
Context-predicting models substantially outperform count-based models across the evaluated lexical semantics benchmarks.
2
Evaluation covers a wide range of lexical semantic tasks and numerous parameter settings.
3
The results provide strong empirical support for the effectiveness of predictive distributional semantic representations.
4
The study systematically compares context-predicting embeddings and neural language models with classic context-counting distributional semantic vectors.

Context-counting and context-predicting distributional semantic models

Their comparative performance across lexical semantics tasks and parameter settings

Publication Details
Publication Date
2014-01-01
Journal
Publisher
ISSN
Cited by
1364
Access Type
Author Information
Authors
Germán Kruszewski
Marco Baroni
Georgiana Dinu
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%