Self-Attention Temporal Convolutional Network for Long-Term Daily Living Activity Detection

Сеть временных свёрток с самовниманием для обнаружения долгосрочной активности в повседневной жизни
Rui Dai, François Brémond, Luca Minciullo, Lorenzo Garattoni, Gianpiero Francesca, Francois Bremond
2019-09-01

DAHLIA datasetSA-TCNSelf-Attention Temporal Convolutional NetworkTemporal Convolutional Networkslong-term daily living activity detection
In this paper, we address the detection of daily living activities in long-term untrimmed videos. The detection of daily living activities is challenging due to their long temporal components, low inter-class variation and high intra-class variation. To tackle these challenges, recent approaches based on Temporal Convolutional Networks (TCNs) have been proposed. Such methods can capture long-term temporal patterns using a hierarchy of temporal convolutional filters, pooling and up sampling steps. However, as one of the important features of convolutional networks, TCNs process a local neighborhood across time which leads to inefficiency in modeling the long-range dependencies between these temporal patterns of the video. In this paper, we propose Self-Attention - Temporal Convolutional Network (SA-TCN), which is able to capture both complex activity patterns and their dependencies within long-term untrimmed videos. We evaluate our proposed model on DAily Home LIfe Activity Dataset (DAHLIA) and Breakfast datasets. Our proposed method achieves state-of-the-art performance on both DAHLIA and Breakfast dataset.
1
Daily living activity detection in long-term untrimmed videos is challenging due to long temporal components, low inter-class variation, and high intra-class variation.
2
SA-TCN was evaluated on DAHLIA (DAily Home LIfe Activity) and Breakfast datasets and achieves state-of-the-art performance on both datasets.
3
Temporal Convolutional Networks (TCNs) capture long-term temporal patterns via hierarchical temporal convolutions, pooling, and upsampling but are inefficient at modeling long-range dependencies because they process local temporal neighborhoods.
4
The paper proposes Self-Attention - Temporal Convolutional Network (SA-TCN) that combines self-attention with TCNs to capture complex activity patterns and their long-range dependencies in long-term videos.

Long-term untrimmed videos of daily living activities (DAHLIA and Breakfast datasets)

Detection of daily living activities including modeling long-range temporal dependencies and complex activity patterns using a Self-Attention Temporal Convolutional Network (SA-TCN)

Publication Details
Publication Date
2019-09-01
Journal
Publisher
ISSN
Cited by
28
Access Type
Author Information
Authors
Rui Dai
François Brémond
Luca Minciullo
Lorenzo Garattoni
Gianpiero Francesca
Francois Bremond
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%