Online Neural Speaker Diarization With Target Speaker Tracking

Онлайн-нейронная диаризация речи с отслеживанием целевых дикторов
Weiqing Wang, Ming Li
2024-01-01

DIHARD IIIonline speaker diarizationspeaker embeddingstarget speaker trackingtarget speaker voice activity detection
This paper proposes an online target speaker voice activity detection (TS-VAD) system for speaker diarization tasks that does not rely on prior knowledge from clustering-based diarization systems to obtain target speaker embeddings. By adapting conventional TS-VAD for real-time operation, our framework identifies speaker activities using self-generated embeddings, ensuring consistent performance and avoiding permutation inconsistencies during inference. In the inference phase, we employ a front-end model to extract frame-level speaker embeddings for each incoming signal block. Subsequently, we predict each speaker's detection state based on these frame-level embeddings and the previously estimated target speaker embeddings. The target speaker embeddings are then updated by aggregating the frame-level embeddings according to the current block's predictions. Our model predicts results block-by-block and iteratively updates target speaker embeddings until reaching the end of the signal. Experimental results demonstrate that the proposed method outperforms offline clustering-based diarization systems on the DIHARD III and AliMeeting datasets. Additionally, this approach is extended to multi-channel data, achieving comparable performance to state-of-the-art offline diarization systems.
1
An online target-speaker voice activity detection framework performs diarization without requiring speaker embeddings from prior clustering-based systems.
2
Its extension to multi-channel audio achieves performance comparable to state-of-the-art offline diarization systems.
3
Self-generated target-speaker embeddings maintain consistent speaker assignments and avoid permutation inconsistencies during inference.
4
The method enables real-time block-by-block inference by extracting frame-level embeddings, predicting speaker activity, and iteratively updating target-speaker embeddings.
5
The proposed approach outperforms offline clustering-based diarization systems on the DIHARD III and AliMeeting datasets.

online target-speaker voice activity detection system for speaker diarization

real-time speaker activity identification and iterative target-speaker embedding tracking without clustering-based initialization, including permutation-consistent inference

Publication Details
Publication Date
2024-01-01
Journal
IEEE/ACM Transactions on Audio Speech and Language Processing
Publisher
Institute of Electrical and Electronics Engineers
ISSN
2329-9290
Cited by
12
Access Type
Author Information
Authors
Weiqing Wang
Ming Li
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%