Search Results for author:
Found 6 papers, 0 papers with code
Audio-Enhanced Text-to-Video Retrieval using Text-Conditioned Feature Alignment
Nonetheless, the objective of the text-to-video retrieval task is to capture the complementary audio and video information that is pertinent to the text query rather than simply achieving better audio and video alignment.
Ranked #10 on
Video Retrieval
on MSR-VTT
Selective Structured State-Spaces for Long-Form Video Understanding
To address this limitation, we present a novel Selective S4 (i. e., S5) model that employs a lightweight mask generator to adaptively select informative image tokens resulting in more efficient and accurate modeling of long-term spatiotemporal dependencies in videos.
Ranked #2 on
Video Classification
on Breakfast
Multiscale Audio Spectrogram Transformer for Efficient Audio Classification
Audio event has a hierarchical architecture in both time and frequency and can be grouped together to construct more abstract semantic audio classes.
Ranked #12 on
Audio Classification
on VGGSound
Dynamic Inference With Grounding Based Vision and Language Models
For example, recent image and language models with more than 200M parameters have been proposed to learn visual grounding in the pre-training step and show impressive results on downstream vision and language tasks.
AVT: Audio-Video Transformer for Multimodal Action Recognition
AVT uses a combination of video and audio signals to improve action recognition accuracy, leveraging the effective spatio-temporal representation by the video Transformer.
Ranked #4 on
Multi-modal Classification
on VGG-Sound
Multiscale Multimodal Transformer for Multimodal Action Recognition
In this work, we develop a multiscale multimodal Transformer (MMT) that employs hierarchical representation learning.
Ranked #1 on
Multi-modal Classification
on VGG-Sound
