Sentiment Analysis in the Era of Large Language Models: A Reality Check

Анализ тональности в эпоху больших языковых моделей: проверка реальностью
Lidong Bing, Wenxuan Zhang, Sinno Jialin Pan, Yue Deng, Bing Liu
2024-01-01

SENTIEVAL benchmarkaspect-based sentiment analysisfew-shot learninglarge language modelssentiment analysis
Sentiment analysis (SA) has been a longstanding research area in natural language processing.With the recent advent of large language models (LLMs), there is great potential for their employment on SA problems.However, the extent to which current LLMs can be leveraged for different sentiment analysis tasks remains unclear.This paper aims to provide a comprehensive investigation into the capabilities of LLMs in performing various sentiment analysis tasks, from conventional sentiment classification to aspect-based sentiment analysis and multifaceted analysis of subjective texts.We evaluate performance across 13 tasks on 26 datasets and compare the results against small language models (SLMs) trained on domain-specific datasets.Our study reveals that while LLMs demonstrate satisfactory performance in simpler tasks, they lag behind in more complex tasks requiring a deeper understanding of specific sentiment phenomena or structured sentiment information.However, LLMs significantly outperform SLMs in fewshot learning settings, suggesting their potential when annotation resources are limited.We also highlight the limitations of current evaluation practices in assessing LLMs' SA abilities and propose a novel benchmark, SENTIEVAL, for a more comprehensive and realistic evaluation.Data and code are available at https:// github.com/DAMO-NLP-SG/LLM-Sentiment.
1
LLMs achieve satisfactory performance on simpler sentiment analysis tasks but underperform on complex tasks requiring nuanced sentiment understanding or structured sentiment information.
2
LLMs significantly outperform small language models in few-shot settings, indicating strong potential when labeled data and annotation resources are limited.
3
The authors identify shortcomings in current evaluation practices for LLM-based sentiment analysis and introduce SENTIEVAL for more comprehensive and realistic evaluation.
4
The study evaluates large language models across 13 sentiment analysis tasks and 26 datasets, comparing them with domain-specific small language models.

large language models applied to sentiment analysis tasks

the capabilities and performance of large language models across conventional, aspect-based, and multifaceted sentiment analysis, including few-shot effectiveness and evaluation limitations

Publication Details
Publication Date
2024-01-01
Journal
Publisher
ISSN
Cited by
897
Access Type
Author Information
Authors
Lidong Bing
Wenxuan Zhang
Sinno Jialin Pan
Yue Deng
Bing Liu
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%