Convolutional Neural Networks for Sentence Classification
Сверточные нейронные сети для классификации предложений
2014-01-01
SCID: 54.1/z5d4a25r
Discuss with AI
Convolutional Neural Networksfine-tuningpre-trained word vectorssentence classificationsentiment analysis
Figures from the paper
Abstract (AI)
We report on a series of experiments with convolutional neural networks (CNN) trained on top of pre-trained word vectors for sentence-level classification tasks.We show that a simple CNN with little hyperparameter tuning and static vectors achieves excellent results on multiple benchmarks.Learning task-specific vectors through fine-tuning offers further gains in performance.We additionally propose a simple modification to the architecture to allow for the use of both task-specific and static vectors.The CNN models discussed herein improve upon the state of the art on 4 out of 7 tasks, which include sentiment analysis and question classification.
Key Findings
1
A proposed simple architecture modification enables simultaneous use of both task-specific (fine-tuned) and static vectors.
2
A simple CNN trained on pre-trained word vectors with little hyperparameter tuning achieves excellent sentence-level classification results across multiple benchmarks.
3
Fine-tuning word vectors for specific tasks provides additional performance gains over static vectors.
4
The described CNN models improve state-of-the-art results on 4 out of 7 evaluated tasks, including sentiment analysis and question classification.
5
Using static (non-fine-tuned) pre-trained word vectors already yields strong performance for sentence classification tasks.
Research Object
Convolutional neural network models for sentence-level classification trained on top of pre-trained word vectors
Research Subject
Effect of CNN architectures and word-vector strategies (static vs. fine-tuned task-specific, and combined) on sentence-level classification performance across benchmarks
Publication Details
Publication Date
2014-01-01
Journal
Publisher
ISSN
Cited by
14831
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai6
ImageNet classification with deep convolutional neural networks2017
Gradient-based learning applied to document recognition1998
Distributed Representations of Words and Phrases and their Compositionality2013
Speech recognition with deep recurrent neural networks2013
Improving neural networks by preventing co-adaptation of feature detectors2012
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank2013
Cited by19
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing2022
A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects2021
Hierarchical Attention Networks for Document Classification2016
CodeBERT: A Pre-Trained Model for Programming and Natural Languages2020
Convolutional 2D Knowledge Graph Embeddings2018
EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks2019
A Survey on Knowledge Graph-Based Recommender Systems2020
End-to-end encrypted traffic classification with one-dimensional convolution neural networks2017
DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning2017
FinBERT : A Large Language Model for Extracting Information from Financial Text*2022
Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN2016
A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges2022
When physics meets machine learning: a survey of physics-informed machine learning2025
Knowledge Graph Completion: A Review2020
"What is relevant in a text document?": An interpretable machine learning approach2017
Financial Sentiment Analysis: Techniques and Applications2024
Semantic Text Matching for Long-Form Documents2019
‘Bingo’—a large language model- and graph neural network-based workflow for the prediction of essential genes from protein data2023
PAI-NET: Retrieval-Augmented Generation Patent Network Using Prior Art Information2025