Speaker Normalization for Self-Supervised Speech Emotion Recognition

Нормализация говорящего для самоконтролируемого распознавания эмоций в речи
Itai Gat, Hagai Aronowitz, Weizhong Zhu, Edmilson Da Silva Morais, Ron Hoory
2022-04-27

IEMOCAP datasetgradient-based adversarial learningself-supervised speech emotion recognitionspeaker normalizationspeaker-independent recognition
Large speech emotion recognition datasets are hard to obtain, and small datasets may contain biases. Deep-net-based classifiers, in turn, are prone to exploit those biases and find shortcuts such as speaker characteristics. These shortcuts usually harm a model’s ability to generalize. To address this challenge, we propose a gradient-based adversary learning framework that learns a speech emotion recognition task while normalizing speaker characteristics from the feature representation. We demonstrate the efficacy of our method on both speaker-independent and speaker-dependent settings and obtain new state-of-the-art results on the challenging IEMOCAP dataset.
1
It proposes a gradient-based adversarial learning framework that jointly learns emotion recognition while normalizing speaker characteristics in feature representations.
2
The approach achieves new state-of-the-art results on the challenging IEMOCAP speech emotion recognition dataset.
3
The method is evaluated in both speaker-independent and speaker-dependent settings, demonstrating effectiveness across both evaluation scenarios.
4
The paper identifies speaker characteristics as dataset biases that self-supervised speech emotion recognition models may exploit as harmful shortcuts.

speech emotion recognition models and their feature representations

the effect of gradient-based adversarial speaker normalization on reducing speaker-characteristic shortcuts and improving generalization in speaker-independent and speaker-dependent emotion recognition

Publication Details
Publication Date
2022-04-27
Journal
Publisher
ISSN
Cited by
69
Access Type
Author Information
Authors
Itai Gat
Hagai Aronowitz
Weizhong Zhu
Edmilson Da Silva Morais
Ron Hoory
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%