Talking Face Generation

10 papers with code • 1 benchmarks • 2 datasets

Talking face generation aims to synthesize a sequence of face images that correspond to given speech semantics.

( Image credit: Talking Face Generation by Adversarially Disentangled Audio-Visual Representation )

Latest papers with code

Text2Video: Text-driven Talking-head Video Synthesis with Phonetic Dictionary

sibozhang/Text2Video 29 Apr 2021

With the advance of deep learning technology, automatic video generation from audio or text has become an emerging and promising research topic.

Talking Face Generation Video Generation

1
29 Apr 2021

Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation

Hangz-nju-cuhk/Talking-Face_PC-AVS 22 Apr 2021

While speech content information can be defined by learning the intrinsic synchronization between audio-visual modalities, we identify that a pose code will be complementarily learned in a modulated convolution-based reconstruction framework.

Talking Face Generation

242
22 Apr 2021

Stochastic Talking Face Generation Using Latent Distribution Matching

ry85/Stochastic-Talking-Face-Generation-Using-Latent-Distribution-Matching 21 Nov 2020

Indeed, just having the ability to generate a single talking face would make a system almost robotic in nature.

Talking Face Generation Video Generation

0
21 Nov 2020

A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild

Rudrabha/Wav2Lip 23 Aug 2020

However, they fail to accurately morph the lip movements of arbitrary identities in dynamic, unconstrained talking face videos, resulting in significant parts of the video being out-of-sync with the new audio.

Unconstrained Lip-synchronization

2,303
23 Aug 2020

Hierarchical Cross-Modal Talking Face Generation With Dynamic Pixel-Wise Loss

lelechen63/ATVGnet CVPR 2019

We devise a cascade GAN approach to generate talking face video, which is robust to different face shapes, view angles, facial characteristics, and noisy audio conditions.

Talking Face Generation

168
01 Jun 2019

Capture, Learning, and Synthesis of 3D Speaking Styles

TimoBolkart/voca CVPR 2019

To address this, we introduce a unique 4D face dataset with about 29 minutes of 4D scans captured at 60 fps and synchronized audio from 12 speakers.

3D Face Animation Talking Face Generation +1

553
08 May 2019

ReenactGAN: Learning to Reenact Faces via Boundary Transfer

wywu/ReenactGAN ECCV 2018

A transformer is subsequently used to adapt the boundary of source face to the boundary of target face.

Face Reenactment Talking Face Generation +1

153
29 Jul 2018

Talking Face Generation by Adversarially Disentangled Audio-Visual Representation

Hangz-nju-cuhk/Talking-Face-Generation-DAVS 20 Jul 2018

Talking face generation aims to synthesize a sequence of face images that correspond to a clip of speech.

Lip Reading Talking Face Generation +1

646
20 Jul 2018

Talking Face Generation by Conditional Recurrent Adversarial Network

susanqq/Talking_Face_Generation 13 Apr 2018

Given an arbitrary face image and an arbitrary speech clip, the proposed work attempts to generating the talking face video with accurate lip synchronization while maintaining smooth transition of both lip and facial movement over the entire video clip.

Constrained Lip-synchronization Video Generation

41
13 Apr 2018