LiteFlowNet3: Resolving Correspondence Ambiguity for More Accurate Optical Flow Estimation

ECCV 2020  ·  Tak-Wai Hui, Chen Change Loy ·

Deep learning approaches have achieved great success in addressing the problem of optical flow estimation. The keys to success lie in the use of cost volume and coarse-to-fine flow inference. However, the matching problem becomes ill-posed when partially occluded or homogeneous regions exist in images. This causes a cost volume to contain outliers and affects the flow decoding from it. Besides, the coarse-to-fine flow inference demands an accurate flow initialization. Ambiguous correspondence yields erroneous flow fields and affects the flow inferences in subsequent levels. In this paper, we introduce LiteFlowNet3, a deep network consisting of two specialized modules, to address the above challenges. (1) We ameliorate the issue of outliers in the cost volume by amending each cost vector through an adaptive modulation prior to the flow decoding. (2) We further improve the flow accuracy by exploring local flow consistency. To this end, each inaccurate optical flow is replaced with an accurate one from a nearby position through a novel warping of the flow field. LiteFlowNet3 not only achieves promising results on public benchmarks but also has a small model size and a fast runtime.

PDF Abstract ECCV 2020 PDF ECCV 2020 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Optical Flow Estimation KITTI 2012 LiteFlowNet3 Average End-Point Error 1.3 # 4
Noc 0.7 # 1
Optical Flow Estimation KITTI 2012 LiteFlowNet3-S Average End-Point Error 1.3 # 4
Noc 0.7 # 1
Optical Flow Estimation KITTI 2015 LiteFlowNet3 Fl-all 7.34 # 9
Fl-fg 7.75 # 1
Optical Flow Estimation KITTI 2015 LiteFlowNet3-S Fl-all 7.22 # 8
Fl-fg 6.96 # 2
Optical Flow Estimation Sintel-clean LiteFlowNet3 Average End-Point Error 2.99 # 16
Optical Flow Estimation Sintel-clean LiteFlowNet3-S Average End-Point Error 3.03 # 17
Optical Flow Estimation Sintel-final LiteFlowNet3 Average End-Point Error 4.45 # 17
Optical Flow Estimation Sintel-final LiteFlowNet3-S Average End-Point Error 4.53 # 19

Methods


No methods listed for this paper. Add relevant methods here