Knowing What, Where and When to Look: Efficient Video Action Modeling with Attention

Attentive video modeling is essential for action recognition in unconstrained videos due to their rich yet redundant information over space and time. However, introducing attention in a deep neural network for action recognition is challenging for two reasons... (read more)

PDF Abstract
No code implementations yet. Submit your code now

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK USES EXTRA
TRAINING DATA
RESULT BENCHMARK
Action Recognition EgoGesture TSM+W3 Top-1 Accuracy 94.3 # 1
Top-5 Accuracy 99.2 # 1
Action Recognition EPIC-Kitchens TSM+W3 - full res Top-1 Accuracy 34.2 # 1
Action Recognition Something-Something V1 TSM+W3 (16 frames, ResNet50) Top 1 Accuracy 52.6 # 10
Top 5 Accuracy 81.3 # 7
Action Recognition Something-Something V2 TSM+W3 (16 frames, RGB ResNet-50) Top-1 Accuracy 66.5 # 4
Top-5 Accuracy 90.4 # 5

Methods used in the Paper