Search Results for author:
Found 18 papers, 8 papers with code
ViC-MAE: Self-Supervised Representation Learning from Images and Video with Contrastive Masked Autoencoders
We show that visual representations learned under ViC-MAE generalize well to both video and image classification tasks.
Ranked #4 on
Image Classification
on Places365
(using extra training data)
Phenaki: Variable Length Video Generation From Open Domain Textual Description
To the best of our knowledge, this is the first time a paper studies generating videos from time variable prompts.
Ranked #4 on
Video Prediction
on BAIR Robot Pushing
RiCS: A 2D Self-Occlusion Map for Harmonizing Volumetric Objects
We test the effectiveness of our representation on the human image harmonization task by predicting shading that is coherent with a given background image.
Contact-Aware Retargeting of Skinned Motion
Self-contacts, such as when hands touch each other or the torso or the head, are important attributes of human body language and dynamics, yet existing methods do not model or preserve these contacts.
Stochastic Scene-Aware Motion Prediction
A long-standing goal in computer vision is to capture, model, and realistically synthesize human behavior.
Single-image Full-body Human Relighting
We present a single-image data-driven method to automatically relight images with full-body humans in them.
Task-Generic Hierarchical Human Motion Prior using VAEs
We demonstrate the effectiveness of our hierarchical motion variational autoencoder in a variety of tasks including video-based human pose estimation, motion completion from partial observations, and motion synthesis from sparse key-frames.
Ranked #4 on
Motion Synthesis
on LaFAN1
Contact and Human Dynamics from Monocular Video
Existing deep models predict 2D and 3D kinematic poses from video that are approximately accurate, but contain visible errors that violate physical constraints, such as feet penetrating the ground and bodies leaning at extreme angles.
High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networks
Predicting future video frames is extremely challenging, as there are many factors of variation that make up the dynamics of how frames change through time.
Unsupervised Learning of Object Structure and Dynamics from Videos
Extracting and predicting object structure and dynamics from videos without supervision is a major challenge in machine learning.
Ranked #11 on
Video Prediction
on KTH
Learning Latent Dynamics for Planning from Pixels
Planning has been very successful for control tasks with known environment dynamics.
Ranked #2 on
Continuous Control
on DeepMind Walker Walk (Images)
MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics
Our model jointly learns a feature embedding for motion modes (that the motion sequence can be reconstructed from) and a feature transformation that represents the transition of one motion mode to the next motion mode.
Ranked #7 on
Human Pose Forecasting
on Human3.6M
(ADE metric)
Hierarchical Long-term Video Prediction without Supervision
Much of recent research has been devoted to video prediction and generation, yet most of the previous works have demonstrated only limited success in generating videos on short-term horizons.
Neural Kinematic Networks for Unsupervised Motion Retargetting
We propose a recurrent neural network architecture with a Forward Kinematics layer and cycle consistency based adversarial training objective for unsupervised motion retargetting.
Decomposing Motion and Content for Natural Video Sequence Prediction
To the best of our knowledge, this is the first end-to-end trainable network architecture with motion and content separation to model the spatiotemporal dynamics for pixel-level future prediction in natural videos.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
Learning to Generate Long-term Future via Hierarchical Prediction
To avoid inherent compounding errors in recursive pixel-level prediction, we propose to first estimate high-level structure in the input frames, then predict how that structure evolves in the future, and finally by observing a single frame from the past and the predicted high-level structure, we construct the future frames without having to observe any of the pixel-level predictions.
Improving Object Detection with Deep Convolutional Networks via Bayesian Optimization and Structured Prediction
Object detection systems based on the deep convolutional neural network (CNN) have recently made ground- breaking advances on several object detection benchmarks.
Who Do I Look Like? Determining Parent-Offspring Resemblance via Gated Autoencoders
Recent years have seen a major push for face recognition technology due to the large expansion of image sharing on social networks.
Ranked #4 on
Kinship Verification
on KinFaceW-II
