Optical Flow Estimation
652 papers with code • 10 benchmarks • 33 datasets
Optical Flow Estimation is a computer vision task that involves computing the motion of objects in an image or a video sequence. The goal of optical flow estimation is to determine the movement of pixels or features in the image, which can be used for various applications such as object tracking, motion analysis, and video compression.
Approaches for optical flow estimation include correlation-based, block-matching, feature tracking, energy-based, and more recently gradient-based.
Further readings:
Definition source: Devon: Deformable Volume Network for Learning Optical Flow
Image credit: Optical Flow Estimation
Benchmarks
These leaderboards are used to track progress in Optical Flow Estimation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Optical Flow Estimation models and implementationsDatasets
KITTI
FlyingThings3D
Vimeo90K
MPI Sintel
Virtual KITTI
MegaDepth
VisDrone
Event-Camera Dataset
N-CARS
MVSEC
Latest papers
Motion-Aware Video Frame Interpolation
zdyshine/Video-Frame-Interpolation-Summary
•
•
Subsequently, a cross-scale motion structure is presented to estimate and refine intermediate flow maps by the extracted features.
Taylor Videos for Action Recognition
Addressing these challenges, we propose the Taylor video, a new video format that highlights the dominate motions (e. g., a waving hand) in each of its frames named the Taylor frame.
Recurrent Partial Kernel Network for Efficient Optical Flow Estimation
hmorimitsu/ptlflow
•
•
The 38th Annual AAAI Conference on Artificial Intelligence (AAAI) 2024
However, this impacts the widespread adoption of optical flow methods and makes it harder to train more general models since the optical flow data is hard to obtain.
Multimodal Action Quality Assessment
qinghuannn/pamfn
•
•
To leverage multimodal information for AQA, i. e., RGB, optical flow and audio information, we propose a Progressive Adaptive Multimodal Fusion Network (PAMFN) that separately models modality-specific information and mixed-modality information.
VONet: Unsupervised Video Object Learning With Parallel U-Net Attention and Object-wise Sequential VAE
hnyu/vonet
•
•
Unsupervised video object learning seeks to decompose video scenes into structural object representations without any supervision from depth, optical flow, or segmentation.
Deep Linear Array Pushbroom Image Restoration: A Degradation Pipeline and Jitter-Aware Restoration Network
jhw2000/jarnet
•
•
Both the proposed JARNet and LAP image synthesis pipeline establish a foundation for addressing this intricate challenge.
RomniStereo: Recurrent Omnidirectional Stereo Matching
To bridge the gap between OSM and RAFT, we mainly propose an opposite adaptive weighting scheme to seamlessly transform the outputs of spherical sweeping of OSM into the required inputs for the recurrent update, thus creating a recurrent omnidirectional stereo matching (RomniStereo) algorithm.
Rethinking RAFT for Efficient Optical Flow
n3slami/Ef-RAFT
•
•
To address these problems, this paper proposes a novel approach based on the RAFT framework.
Hierarchical Graph Pattern Understanding for Zero-Shot VOS
nust-machine-intelligence-laboratory/hgpu
•
•
However, existing optical flow-based methods have a significant dependency on optical flow, which results in poor performance when the optical flow estimation fails for a particular scene.
Open-DDVM: A Reproduction and Extension of Diffusion Model for Optical Flow Estimation
dqiaole/flowdiffusion_pytorch
•
•
Recently, Google proposes DDVM which for the first time demonstrates that a general diffusion model for image-to-image translation task works impressively well on optical flow estimation task without any specific designs like RAFT.
