Towards Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs

4 Dec 2017  ·  Jun Yu, Xingxin Xu, Fei Gao, Shengjie Shi, Meng Wang, DaCheng Tao, Qingming Huang ·

Face photo-sketch synthesis aims at generating a facial sketch/photo conditioned on a given photo/sketch. It is of wide applications including digital entertainment and law enforcement... Precisely depicting face photos/sketches remains challenging due to the restrictions on structural realism and textural consistency. While existing methods achieve compelling results, they mostly yield blurred effects and great deformation over various facial components, leading to the unrealistic feeling of synthesized images. To tackle this challenge, in this work, we propose to use the facial composition information to help the synthesis of face sketch/photo. Specially, we propose a novel composition-aided generative adversarial network (CA-GAN) for face photo-sketch synthesis. In CA-GAN, we utilize paired inputs including a face photo/sketch and the corresponding pixel-wise face labels for generating a sketch/photo. In addition, to focus training on hard-generated components and delicate facial structures, we propose a compositional reconstruction loss. Finally, we use stacked CA-GANs (SCA-GAN) to further rectify defects and add compelling details. Experimental results show that our method is capable of generating both visually comfortable and identity-preserving face sketches/photos over a wide range of challenging data. Our method achieves the state-of-the-art quality, reducing best previous Frechet Inception distance (FID) by a large margin. Besides, we demonstrate that the proposed method is of considerable generalization ability. We have made our code and results publicly available: https://fei-hdu.github.io/ca-gan/. read more

PDF Abstract

Datasets


Results from the Paper


 Ranked #1 on Face Sketch Synthesis on CUFS (FID metric)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Face Sketch Synthesis CUFS SCA-GAN FSIM 71.6% # 2
FID 34.2 # 2
NLDA 95.7 # 1
Face Sketch Synthesis CUFS SCA-GAN4 FID 32.8 # 1
Face Sketch Synthesis CUFS CA-GAN FSIM 71.2% # 3
FID 39.7 # 3
NLDA 95.6 # 2
Face Sketch Synthesis CUFSF CA-GAN FSIM 72.7% # 2
FID 19.6 # 2
NLDA 78.1 # 1
Face Sketch Synthesis CUFSF SCA-GAN FSIM 72.9% # 1
FID 18.2 # 1
NLDA 78 # 2

Methods


No methods listed for this paper. Add relevant methods here