FSTF-AN: Fused Sparse Temporal-Frequency Attentive Network for Multi-Channel Speech Enhancement

FSTF-AN: объединённая разреженная временно-частотная сеть с механизмом внимания для многоканального улучшения речи
Wenjie Zhang, Mingjiang Wang, Shiyun Xu, Yinghan Cao, Zehua Zhang
2025-01-01

fused sparse temporal-frequency attentionmulti-channel speech enhancementmulti-scale feed-forward networkspeech quality and intelligibilitytop-k fused sparse self-attention
The Transformer has achieved impressive performance in the multi-channel speech enhancement field; however, it struggles to capture local features, which leads to the loss of speech details. To enhance the extraction of local features in the network, we propose a fused sparse temporal-frequency attentive network (FSTF-AN), which aims to fully capture features across the temporal-frequency, frequency, and temporal dimensions. We propose top-$k$fused sparse self-attention, which employs a fusion strategy to adaptively retain the most crucial attention scores when computing self-attention maps, thereby eliminating irrelevant information interference and better aggregating features. Furthermore, we propose a multi-scale fused feed-forward network, which effectively captures multi-scale features, further enhancing the network's ability to capture local features. The experimental results demonstrate that FSTF-AN exhibits significant advantages over other SOTA models, effectively enhancing speech quality and intelligibility.
1
A multi-scale fused feed-forward network captures features at multiple scales, strengthening the model’s ability to represent local speech details.
2
Experiments show that FSTF-AN significantly outperforms other state-of-the-art models in enhancing speech quality and intelligibility.
3
FSTF-AN is proposed to improve multi-channel speech enhancement by better capturing local speech features across temporal-frequency, frequency, and temporal dimensions.
4
The top-k fused sparse self-attention mechanism adaptively retains the most important attention scores, reducing irrelevant-information interference during feature aggregation.

multi-channel speech enhancement

local and multi-scale temporal-frequency feature extraction and speech-quality/intelligibility enhancement

Publication Details
Publication Date
2025-01-01
Journal
Publisher
ISSN
Cited by
2
Access Type
Author Information
Authors
Wenjie Zhang
Mingjiang Wang
Shiyun Xu
Yinghan Cao
Zehua Zhang
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%