Search Results for author:
Found 31 papers, 19 papers with code
AutoAD III: The Prequel -- Back to the Pixels
Generating Audio Description (AD) for movies is a challenging task that requires fine-grained visual understanding and an awareness of the characters and their names.
Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation
To generate composite animations from a multi-track timeline, we propose a new test-time denoising method.
AutoAD II: The Sequel -- Who, When, and What in Movie Audio Description
Audio Description (AD) is the task of generating descriptions of visual content, at suitable time intervals, for the benefit of visually impaired audiences.
CoVR: Learning Composed Video Retrieval from Web Video Captions
Most CoIR approaches require manually annotated datasets, comprising image-text-image triplets, where the text describes a modification from the query image to the target image.
Ranked #1 on
Composed Video Retrieval (CoVR)
on WebVid-CoVR
TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis
We show that maintaining the motion generation loss, along with the contrastive training, is crucial to obtain good performance.
SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generation
Motivated by the observation that the correspondence between actions and body parts is encoded in powerful language models, we extract this knowledge by prompting GPT-3 with text such as "what are the body parts involved in the action <action name>?
Going Beyond Nouns With Vision & Language Models Using Synthetic Data
We contribute Synthetic Visual Concepts (SyViC) - a million-scale synthetic dataset and data generation codebase allowing to generate additional suitable data to improve VLC understanding and compositional reasoning of VL models.
Ranked #68 on
Visual Reasoning
on Winoground
AutoAD: Movie Description in Context
The objective of this paper is an automatic Audio Description (AD) model that ingests movies and outputs AD in text form.
Weakly-supervised Fingerspelling Recognition in British Sign Language Videos
Through extensive evaluations, we verify our method for automatic annotation and our model architecture.
TEACH: Temporal Action Composition for 3D Humans
In particular, our goal is to enable the synthesis of a series of actions, which we refer to as temporal action composition.
Automatic dense annotation of large-vocabulary sign language videos
Recently, sign language researchers have turned to sign language interpreted TV broadcasts, comprising (i) a video of continuous signing and (ii) subtitles corresponding to the audio content, as a readily available and large-scale source of training data.
A CLIP-Hitchhiker's Guide to Long Video Retrieval
Our goal in this paper is the adaptation of image-text models for long video retrieval.
Ranked #4 on
Zero-Shot Action Recognition
on Charades
Scaling up sign spotting through sign language dictionaries
The focus of this work is $\textit{sign spotting}$ - given a video of an isolated sign, our task is to identify $\textit{whether}$ and $\textit{where}$ it has been signed in a continuous, co-articulated sign language video.
TEMOS: Generating diverse human motions from textual descriptions
In contrast to most previous work which focuses on generating a single, deterministic, motion from a textual description, we design a variational approach that can produce multiple diverse human motions.
Ranked #7 on
Motion Synthesis
on InterHuman
BBC-Oxford British Sign Language Dataset
In this work, we introduce the BBC-Oxford British Sign Language (BOBSL) dataset, a large-scale video collection of British Sign Language (BSL).
Towards unconstrained joint hand-object reconstruction from RGB videos
Our work aims to obtain 3D reconstruction of hands and manipulated objects from monocular videos.
Ranked #5 on
hand-object pose
on HO-3D
Aligning Subtitles in Sign Language Videos
The goal of this work is to temporally align asynchronous subtitles in sign language videos.
Sign Segmentation with Changepoint-Modulated Pseudo-Labelling
The objective of this work is to find temporal boundaries between signs in continuous sign language.
Action-Conditioned 3D Human Motion Synthesis with Transformer VAE
By sampling from this latent space and querying a certain duration through a series of positional encodings, we synthesize variable-length motion sequences conditioned on a categorical action.
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval
Our objective in this work is video-text retrieval - in particular a joint embedding that enables efficient text-to-video retrieval.
Ranked #4 on
Video Retrieval
on QuerYD
(using extra training data)
Read and Attend: Temporal Localisation in Sign Language Videos
Our contributions are as follows: (1) we demonstrate the ability to leverage large quantities of continuous signing videos with weakly-aligned subtitles to localise signs in continuous sign language; (2) we employ the learned attention to automatically generate hundreds of thousands of annotations for a large sign vocabulary; (3) we collect a set of 37K manually verified sign instances across a vocabulary of 950 sign classes to support our study of sign language recognition; (4) by training on the newly annotated data from our method, we outperform the prior state of the art on the BSL-1K sign language recognition benchmark.
Sign language segmentation with temporal convolutional networks
The objective of this work is to determine the location of temporal boundaries between signs in continuous sign language videos.
Watch, read and lookup: learning to spot signs from multiple supervisors
The focus of this work is sign spotting - given a video of an isolated sign, our task is to identify whether and where it has been signed in a continuous, co-articulated sign language video.
BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues
Recent progress in fine-grained gesture and action classification, and machine translation, point to the possibility of automated sign language recognition becoming a reality.
Ranked #4 on
Sign Language Recognition
on WLASL-2000
Synthetic Humans for Action Recognition from Unseen Viewpoints
Although synthetic training data has been shown to be beneficial for tasks such as human pose estimation, its use for RGB human action recognition is relatively unexplored.
Learning joint reconstruction of hands and manipulated objects
Previous work has made significant progress towards reconstruction of hand poses and object shapes in isolation.
Ranked #7 on
hand-object pose
on DexYCB
BodyNet: Volumetric Inference of 3D Human Body Shapes
Human shape estimation is an important task for video editing, animation and fashion industry.
Ranked #3 on
3D Human Pose Estimation
on Surreal
(using extra training data)
Learning from Synthetic Humans
In this work we present SURREAL (Synthetic hUmans foR REAL tasks): a new large-scale dataset with synthetically-generated but realistic images of people rendered from 3D sequences of human motion capture data.
Long-term Temporal Convolutions for Action Recognition
Typical human actions last several seconds and exhibit characteristic spatio-temporal structure.
Ranked #63 on
Action Recognition
on HMDB-51
Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
Each video is annotated by multiple free-text descriptions, action labels, action intervals and classes of interacted objects.
