Optical Flow Estimation
650 papers with code • 10 benchmarks • 33 datasets
Optical Flow Estimation is a computer vision task that involves computing the motion of objects in an image or a video sequence. The goal of optical flow estimation is to determine the movement of pixels or features in the image, which can be used for various applications such as object tracking, motion analysis, and video compression.
Approaches for optical flow estimation include correlation-based, block-matching, feature tracking, energy-based, and more recently gradient-based.
Further readings:
Definition source: Devon: Deformable Volume Network for Learning Optical Flow
Image credit: Optical Flow Estimation
Benchmarks
These leaderboards are used to track progress in Optical Flow Estimation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Optical Flow Estimation models and implementationsDatasets
KITTI
FlyingThings3D
Vimeo90K
MPI Sintel
Virtual KITTI
MegaDepth
VisDrone
Event-Camera Dataset
N-CARS
MVSEC
Most implemented papers
Deep multi-scale video prediction beyond mean square error
dyelax/Adversarial_Video_Generation
•
•
Learning to predict future images from a video sequence involves the construction of an internal representation that models the image evolution accurately, and therefore, to some degree, its content and dynamics.
Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation
avinashpaliwal/Super-SloMo
•
•
CVPR 2018
Finally, the two input images are warped and linearly fused to form each intermediate frame.
Representation Flow for Action Recognition
piergiaj/representation-flow-cvpr19
•
•
CVPR 2019
Our representation flow layer is a fully-differentiable layer designed to capture the `flow' of any representation channel within a convolutional neural network for action recognition.
Depth-Aware Video Frame Interpolation
The proposed model then warps the input frames, depth maps, and contextual features based on the optical flow and local interpolation kernels for synthesizing the output frame.
Attention-guided Network for Ghost-free High Dynamic Range Imaging
Ghosting artifacts caused by moving objects or misalignments is a key challenge in high dynamic range (HDR) imaging for dynamic scenes.
What Matters in Unsupervised Optical Flow
google-research/google-research
•
•
ECCV 2020
We systematically compare and analyze a set of key components in unsupervised optical flow to identify which photometric loss, occlusion handling, and smoothness regularization is most effective.
Large-scale, Fast and Accurate Shot Boundary Detection through Spatio-temporal Convolutional Neural Networks
Since current datasets are not large enough to train an accurate SBD CNN, we present a new dataset containing more than 3. 5 million frames of sharp and gradual transitions.
Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model
Hzzone/Precipitation-Nowcasting
•
•
NeurIPS 2017
To address these problems, we propose both a new model and a benchmark for precipitation nowcasting.
Video Enhancement with Task-Oriented Flow
Many video enhancement algorithms rely on optical flow to register frames in a video sequence.
Im2Flow: Motion Hallucination from Static Images for Action Recognition
Second, we show the power of hallucinated flow for recognition, successfully transferring the learned motion into a standard two-stream network for activity recognition.
