Breathing Life into Faces: Speech-driven 3D Facial Animation with Natural Head Pose and Detailed Shape

no code implementations • 31 Oct 2023 • Wei Zhao, Yijun Wang, Tianyu He, Lianying Yin, Jianxin Lin, Xin Jin

To augment the richness of 3D facial animation, we construct a new 3D dataset with detailed shapes and learn to synthesize facial details in line with speech content.

EMoG: Synthesizing Emotive Co-speech 3D Gesture with Diffusion Model

no code implementations • 20 Jun 2023 • Lianying Yin, Yijun Wang, Tianyu He, Jinming Liu, Wei Zhao, Bohan Li, Xin Jin, Jianxin Lin

In this paper, we present a novel framework (EMoG) to tackle the above challenges with denoising diffusion models: 1) To alleviate the one-to-many problem, we incorporate emotion clues to guide the generation process, making the generation much easier; 2) To model joint correlation, we propose to decompose the difficult gesture generation into two sub-problems: joint correlation modeling and temporal dynamics modeling.

Denoising Gesture Generation