Speaker Normalization for Self-Supervised Speech Emotion Recognition
Нормализация говорящего для самоконтролируемого распознавания эмоций в речи
2022-04-27
SCID: 54.1/5ey2tjzr
Discuss with AI
IEMOCAP datasetgradient-based adversarial learningself-supervised speech emotion recognitionspeaker normalizationspeaker-independent recognition
Figures from the paper
Abstract (AI)
Large speech emotion recognition datasets are hard to obtain, and small datasets may contain biases. Deep-net-based classifiers, in turn, are prone to exploit those biases and find shortcuts such as speaker characteristics. These shortcuts usually harm a model’s ability to generalize. To address this challenge, we propose a gradient-based adversary learning framework that learns a speech emotion recognition task while normalizing speaker characteristics from the feature representation. We demonstrate the efficacy of our method on both speaker-independent and speaker-dependent settings and obtain new state-of-the-art results on the challenging IEMOCAP dataset.
Key Findings
1
It proposes a gradient-based adversarial learning framework that jointly learns emotion recognition while normalizing speaker characteristics in feature representations.
2
The approach achieves new state-of-the-art results on the challenging IEMOCAP speech emotion recognition dataset.
3
The method is evaluated in both speaker-independent and speaker-dependent settings, demonstrating effectiveness across both evaluation scenarios.
4
The paper identifies speaker characteristics as dataset biases that self-supervised speech emotion recognition models may exploit as harmful shortcuts.
Research Object
speech emotion recognition models and their feature representations
Research Subject
the effect of gradient-based adversarial speaker normalization on reducing speaker-characteristic shortcuts and improving generalization in speaker-independent and speaker-dependent emotion recognition
Publication Details
Publication Date
2022-04-27
Journal
Publisher
ISSN
Cited by
69
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai5
Representation Learning with Contrastive Predictive Coding2018
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units2021
VoxCeleb: A Large-Scale Speaker Identification Dataset2017
The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English2018
TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech2021