Online Neural Speaker Diarization With Target Speaker Tracking
Онлайн-нейронная диаризация речи с отслеживанием целевых дикторов
2024-01-01
SCID: 54.1/grkgnkvy
Discuss with AI
DIHARD IIIonline speaker diarizationspeaker embeddingstarget speaker trackingtarget speaker voice activity detection
Figures from the paper
Abstract (AI)
This paper proposes an online target speaker voice activity detection (TS-VAD) system for speaker diarization tasks that does not rely on prior knowledge from clustering-based diarization systems to obtain target speaker embeddings. By adapting conventional TS-VAD for real-time operation, our framework identifies speaker activities using self-generated embeddings, ensuring consistent performance and avoiding permutation inconsistencies during inference. In the inference phase, we employ a front-end model to extract frame-level speaker embeddings for each incoming signal block. Subsequently, we predict each speaker's detection state based on these frame-level embeddings and the previously estimated target speaker embeddings. The target speaker embeddings are then updated by aggregating the frame-level embeddings according to the current block's predictions. Our model predicts results block-by-block and iteratively updates target speaker embeddings until reaching the end of the signal. Experimental results demonstrate that the proposed method outperforms offline clustering-based diarization systems on the DIHARD III and AliMeeting datasets. Additionally, this approach is extended to multi-channel data, achieving comparable performance to state-of-the-art offline diarization systems.
Key Findings
1
An online target-speaker voice activity detection framework performs diarization without requiring speaker embeddings from prior clustering-based systems.
2
Its extension to multi-channel audio achieves performance comparable to state-of-the-art offline diarization systems.
3
Self-generated target-speaker embeddings maintain consistent speaker assignments and avoid permutation inconsistencies during inference.
4
The method enables real-time block-by-block inference by extracting frame-level embeddings, predicting speaker activity, and iteratively updating target-speaker embeddings.
5
The proposed approach outperforms offline clustering-based diarization systems on the DIHARD III and AliMeeting datasets.
Research Object
online target-speaker voice activity detection system for speaker diarization
Research Subject
real-time speaker activity identification and iterative target-speaker embedding tracking without clustering-based initialization, including permutation-consistent inference
Publication Details
Publication Date
2024-01-01
Journal
IEEE/ACM Transactions on Audio Speech and Language Processing
Publisher
Institute of Electrical and Electronics Engineers
ISSN
2329-9290
Cited by
12
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai5
Front-End Factor Analysis for Speaker Verification2010
VoxCeleb: A Large-Scale Speaker Identification Dataset2017
A study on data augmentation of reverberant speech for robust speech recognition2017
Joint Speaker Counting, Speech Recognition, and Speaker Identification for Overlapped Speech of any Number of Speakers2020
Microsoft Speaker Diarization System for the Voxceleb Speaker Recognition Challenge 20202021