Browse SoTA > Computer Vision > Video > Video Prediction

Video Prediction

63 papers with code · Computer Vision
Subtask of Video

Video Prediction is the task of predicting future frames given past video frames.

Source: Photo-Realistic Video Prediction on Natural Videos of Largely Changing Frames

Benchmarks

Greatest papers with code

Unsupervised Learning for Physical Interaction through Video Prediction

NeurIPS 2016 tensorflow/models

A core challenge for an agent learning to interact with the world is to predict how its actions affect objects in its environment.

VIDEO PREDICTION

Unsupervised learning of object structure and dynamics from videos

NeurIPS 2019 google-research/google-research

Extracting and predicting object structure and dynamics from videos without supervision is a major challenge in machine learning.

ACTION RECOGNITION CONTINUOUS CONTROL OBJECT TRACKING VIDEO PREDICTION

Video-to-Video Synthesis

NeurIPS 2018 NVIDIA/vid2vid

We study the problem of video-to-video synthesis, whose goal is to learn a mapping function from an input source video (e. g., a sequence of semantic segmentation masks) to an output photorealistic video that precisely depicts the content of the source video.

SEMANTIC SEGMENTATION VIDEO PREDICTION VIDEO-TO-VIDEO SYNTHESIS

Learning a Driving Simulator

3 Aug 2016commaai/research

Comma. ai's approach to Artificial Intelligence for self-driving cars is based on an agent that learns to clone driver behaviors and plans maneuvers by simulating future events in the road.

SELF-DRIVING CARS VIDEO PREDICTION

Deep multi-scale video prediction beyond mean square error

17 Nov 2015dyelax/Adversarial_Video_Generation

Learning to predict future images from a video sequence involves the construction of an internal representation that models the image evolution accurately, and therefore, to some degree, its content and dynamics.

ADVERSARIAL TRAINING OPTICAL FLOW ESTIMATION VIDEO PREDICTION

Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning

25 May 2016coxlab/prednet

Here, we explore prediction of future frames in a video sequence as an unsupervised learning rule for learning about the structure of the visual world.

OBJECT RECOGNITION VIDEO PREDICTION

Improving Semantic Segmentation via Video Propagation and Label Relaxation

CVPR 2019 NVIDIA/semantic-segmentation

In this paper, we present a video prediction-based methodology to scale up training sets by synthesizing new training samples in order to improve the accuracy of semantic segmentation networks.

 Ranked #1 on Semantic Segmentation on CamVid (using extra training data)

SEMANTIC SEGMENTATION VIDEO PREDICTION

Scalable Gradients for Stochastic Differential Equations

5 Jan 2020google-research/torchsde

The adjoint sensitivity method scalably computes gradients of solutions to ordinary differential equations.

MOTION CAPTURE VIDEO PREDICTION

Model-Based Reinforcement Learning for Atari

1 Mar 2019werner-duvaud/muzero-general

We describe Simulated Policy Learning (SimPLe), a complete model-based deep RL algorithm based on video prediction models and present a comparison of several model architectures, including a novel architecture that yields the best results in our setting.

ATARI GAMES VIDEO PREDICTION

Future Frame Prediction for Anomaly Detection -- A New Baseline

28 Dec 2017stevenliuwen/ano_pred_cvpr2018

To predict a future frame with higher quality for normal events, other than the commonly used appearance (spatial) constraints on intensity and gradient, we also introduce a motion (temporal) constraint in video prediction by enforcing the optical flow between predicted frames and ground truth frames to be consistent, and this is the first work that introduces a temporal constraint into the video prediction task.

ANOMALY DETECTION OPTICAL FLOW ESTIMATION VIDEO PREDICTION