Gesture Generation
34 papers with code • 4 benchmarks • 6 datasets
Generation of gestures, as a sequence of 3d poses
Libraries
Use these libraries to find Gesture Generation models and implementations
DVS128 Gesture
BEAT
BEAT2
Most implemented papers
robosuite: A Modular Simulation Framework and Benchmark for Robot Learning
robosuite is a simulation framework for robot learning powered by the MuJoCo physics engine.
The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation
genea-workshop/genea_numerical_evaluations
•
•
On the other hand, all synthetic motion is found to be vastly less appropriate for the speech than the original motion-capture recordings.
Learning Individual Styles of Conversational Gesture
Specifically, we perform cross-modal translation from "in-the-wild'' monologue speech of a single speaker to their hand and arm motion.
Speech Gesture Generation from the Trimodal Context of Text, Audio, and Speaker Identity
ai4r/Gesture-Generation-from-Trimodal-Context
•
•
In this paper, we present an automatic gesture generation model that uses the multimodal context of speech text, audio, and speaker identity to reliably generate gestures.
BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesis
Achieving realistic, vivid, and human-like synthesized conversational gestures conditioned on multi-modal data is still an unsolved problem due to the lack of available datasets, models and standard evaluation metrics.
Generating Holistic 3D Human Motion from Speech
yhw-yhw/talkshow
•
•
CVPR 2023
This work addresses the problem of generating 3D holistic body motions from human speech.
The GENEA Challenge 2023: A large scale evaluation of gesture generation models in monadic and dyadic settings
The effect of the interlocutor is even more subtle, with submitted systems at best performing barely above chance.
EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling
We propose EMAGE, a framework to generate full-body human gestures from audio and masked gestures, encompassing facial, local body, hands, and global movements.
Analyzing Input and Output Representations for Speech-Driven Gesture Generation
GestureGeneration/Speech_driven_gesture_generation_with_autoencoder
•
•
arXiv 2019
We evaluate different representation sizes in order to find the most effective dimensionality for the representation.
Gesticulator: A framework for semantically-aware speech-driven gesture generation
During speech, people spontaneously gesticulate, which plays a key role in conveying information.
