Towards Fast, Accurate and Stable 3D Dense Face Alignment

Existing methods of 3D dense face alignment mainly concentrate on accuracy, thus limiting the scope of their practical applications. In this paper, we propose a novel regression framework named 3DDFA-V2 which makes a balance among speed, accuracy and stability. Firstly, on the basis of a lightweight backbone, we propose a meta-joint optimization strategy to dynamically regress a small set of 3DMM parameters, which greatly enhances speed and accuracy simultaneously. To further improve the stability on videos, we present a virtual synthesis method to transform one still image to a short-video which incorporates in-plane and out-of-plane face moving. On the premise of high accuracy and stability, 3DDFA-V2 runs at over 50fps on a single CPU core and outperforms other state-of-the-art heavy models simultaneously. Experiments on several challenging datasets validate the efficiency of our method. Pre-trained models and code are available at https://github.com/cleardusk/3DDFA_V2.

PDF Abstract ECCV 2020 PDF ECCV 2020 Abstract

Results from the Paper


 Ranked #1 on 3D Face Reconstruction on Florence (Mean NME metric)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Face Alignment AFLW 3DDFA_V2 Mean NME 4.43 # 2
Face Alignment AFLW2000-3D 3DDFA_V2 Mean NME 3.51% # 6
3D Face Reconstruction AFLW2000-3D 3DDFA-V2 Mean NME 3.56% # 4
3D Face Reconstruction Florence 3DDFA_V2 Mean NME 3.56 # 1
3D Face Reconstruction NoW Benchmark 3DDFA_V2 Mean Reconstruction Error (mm) 1.57 # 11
Stdev Reconstruction Error (mm) 1.39 # 13
Median Reconstruction Error 1.23 # 10
3D Face Reconstruction REALY 3DDFA-v2 @nose 1.903 (±0.517) # 14
@mouth 1.597 (±0.478) # 10
@forehead 2.447 (±0.647) # 17
@cheek 1.757 (±0.642) # 19
all 1.926 # 12
3D Face Reconstruction REALY (side-view) 3DDFA-v2 @nose 1.883 (±0.499) # 12
all 1.943 # 8
@mouth 1.642 (±0.501) # 7
@forehead 2.465 (±0.622) # 12
@cheek 1.781 (±0.636) # 14
3D Face Reconstruction Stirling-HQ (FG2018 3D face reconstruction challenge) 3DDFA_V2 Mean Reconstruction Error (mm) 1.91 # 2
3D Face Reconstruction Stirling-LQ (FG2018 3D face reconstruction challenge) 3DDFA_V2 Mean Reconstruction Error (mm) 2.10 # 3

Methods