Enriching Word Vectors with Subword Information
Обогащение векторов слов информацией о субсловах
2017-12-01
SCID: 54.1/8zkhyr6t
Discuss with AI
character n-gramsmorphological word representationsskipgram modelsubword informationword similarity and analogy tasks
Figures from the paper
Abstract (AI)
Continuous word representations, trained on large unlabeled corpora are useful for many natural language processing tasks. Popular models that learn such representations ignore the morphology of words, by assigning a distinct vector to each word. This is a limitation, especially for languages with large vocabularies and many rare words. In this paper, we propose a new approach based on the skipgram model, where each word is represented as a bag of character n-grams. A vector representation is associated to each character n-gram; words being represented as the sum of these representations. Our method is fast, allowing to train models on large corpora quickly and allows us to compute word representations for words that did not appear in the training data. We evaluate our word representations on nine different languages, both on word similarity and analogy tasks. By comparing to recently proposed morphological word representations, we show that our vectors achieve state-of-the-art performance on these tasks.
Key Findings
1
Across nine languages, the representations are evaluated on word similarity and analogy tasks.
2
Character n-gram representations incorporate word morphology and better address large vocabularies and rare words than whole-word vectors.
3
Compared with recently proposed morphological representations, the method achieves state-of-the-art performance on these tasks.
4
The method trains quickly on large corpora and can generate representations for words absent from the training data.
5
The paper enriches skipgram word vectors by representing each word as the sum of vectors for its character n-grams.
Research Object
Continuous word vector representations augmented with character n-gram (subword) components
Research Subject
the effectiveness of subword-based word representations for morphological modeling, including performance on word similarity and analogy tasks and representation of unseen words
Publication Details
Publication Date
2017-12-01
Journal
Publisher
ISSN
Cited by
11057
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai2
Cited by17
A Unifying Review of Deep and Shallow Anomaly Detection2021
Modeling aspects of the language of life through transfer-learning protein sequences2019
A review on sentiment analysis from social media platforms2023
Social media data for conservation science: A methodological overview2019
Large language models (LLMs): survey, technical frameworks, and future challenges2024
The language of proteins: NLP, machine learning & protein sequences2021
Distributional Semantics and Linguistic Theory2019
Data-driven materials research enabled by natural language processing and information extraction2020
A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs)2024
Vision-language models for medical report generation and visual question answering: a review2024
Open Research Knowledge Graph2019
Financial Sentiment Analysis: Techniques and Applications2024
Open Research Knowledge Graph: Next Generation Infrastructure for Semantic Scholarly Knowledge2019
Generate FAIR Literature Surveys with Scholarly Knowledge Graphs2020
Machine Learning Approaches to TCR Repertoire Analysis2022
Triple Classification for Scholarly Knowledge Graph Completion2021
Comparing research contributions in a scholarly knowledge graph2019