Search Results for author:
Found 7 papers, 6 papers with code
Synchformer: Efficient Synchronization from Sparse Cues
Our objective is audio-visual synchronization with a focus on 'in-the-wild' videos, such as those on YouTube, where synchronization cues can be sparse.
Sparse in Space and Time: Audio-visual Synchronisation with Trainable Selectors
This contrasts with the case of synchronising videos of talking heads, where audio-visual correspondence is dense in both time and space.
Taming Visually Guided Sound Generation
In this work, we propose a single model capable of generating visually relevant, high-fidelity sounds prompted with a set of frames from open-domain videos in less time than it takes to play it on a single GPU.
The CORSMAL benchmark for the prediction of the properties of containers
In this paper, we present a range of methods and an open framework to benchmark acoustic and visual perception for the estimation of the capacity of a container, and the type, mass, and amount of its content.
Top-1 CORSMAL Challenge 2020 Submission: Filling Mass Estimation Using Multi-modal Observations of Human-robot Handovers
CORSMAL 2020 Challenge focuses on the perception part of this problem: the robot needs to estimate the filling mass of a container held by a human.
A Better Use of Audio-Visual Cues: Dense Video Captioning with Bi-modal Transformer
We show the effectiveness of the proposed model with audio and visual modalities on the dense video captioning task, yet the module is capable of digesting any two modalities in a sequence-to-sequence task.
Multi-modal Dense Video Captioning
We apply automatic speech recognition (ASR) system to obtain a temporally aligned textual description of the speech (similar to subtitles) and treat it as a separate input alongside video frames and the corresponding audio track.
Ranked #11 on
Dense Video Captioning
on ActivityNet Captions
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
