Sentiment Analysis in the Era of Large Language Models: A Reality Check
Анализ тональности в эпоху больших языковых моделей: проверка реальностью
2024-01-01
SCID: 54.1/wbxv2ux7
Discuss with AI
SENTIEVAL benchmarkaspect-based sentiment analysisfew-shot learninglarge language modelssentiment analysis
Figures from the paper
Abstract (AI)
Sentiment analysis (SA) has been a longstanding research area in natural language processing.With the recent advent of large language models (LLMs), there is great potential for their employment on SA problems.However, the extent to which current LLMs can be leveraged for different sentiment analysis tasks remains unclear.This paper aims to provide a comprehensive investigation into the capabilities of LLMs in performing various sentiment analysis tasks, from conventional sentiment classification to aspect-based sentiment analysis and multifaceted analysis of subjective texts.We evaluate performance across 13 tasks on 26 datasets and compare the results against small language models (SLMs) trained on domain-specific datasets.Our study reveals that while LLMs demonstrate satisfactory performance in simpler tasks, they lag behind in more complex tasks requiring a deeper understanding of specific sentiment phenomena or structured sentiment information.However, LLMs significantly outperform SLMs in fewshot learning settings, suggesting their potential when annotation resources are limited.We also highlight the limitations of current evaluation practices in assessing LLMs' SA abilities and propose a novel benchmark, SENTIEVAL, for a more comprehensive and realistic evaluation.Data and code are available at https:// github.com/DAMO-NLP-SG/LLM-Sentiment.
Key Findings
1
LLMs achieve satisfactory performance on simpler sentiment analysis tasks but underperform on complex tasks requiring nuanced sentiment understanding or structured sentiment information.
2
LLMs significantly outperform small language models in few-shot settings, indicating strong potential when labeled data and annotation resources are limited.
3
The authors identify shortcomings in current evaluation practices for LLM-based sentiment analysis and introduce SENTIEVAL for more comprehensive and realistic evaluation.
4
The study evaluates large language models across 13 sentiment analysis tasks and 26 datasets, comparing them with domain-specific small language models.
Research Object
large language models applied to sentiment analysis tasks
Research Subject
the capabilities and performance of large language models across conventional, aspect-based, and multifaceted sentiment analysis, including few-shot effectiveness and evaluation limitations
Publication Details
Publication Date
2024-01-01
Journal
Publisher
ISSN
Cited by
897
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai5
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer2019
LLaMA: Open and Efficient Foundation Language Models2023
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank2013
GPT-4 Technical Report2023
Sparks of Artificial General Intelligence: Early experiments with GPT-42023
Cited by5
A Survey on Evaluation of Large Language Models2024
Unleashing the potential of prompt engineering for large language models2025
Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects2023
Causal Prompting for Implicit Sentiment Analysis With Large Language Models2026
Leveraging fine-tuning of large language models for aspect-based sentiment analysis in resource-scarce environments2026