Search Results for author:
Found 38 papers, 20 papers with code
Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers
Next, we finetune a retrieval model on a small subset where the best caption of each video is manually selected and then employ the model in the whole dataset to select the best caption as the annotation.
Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesis
Since video content is highly redundant, we argue that naively bringing advances of image models to the video generation domain reduces motion fidelity, visual quality and impairs scalability.
Ranked #1 on
Text-to-Video Generation
on MSR-VTT
SPAD : Spatially Aware Multiview Diffusers
We present SPAD, a novel approach for creating consistent multi-view images from text prompts or single images.
AToM: Amortized Text-to-Mesh using 2D Diffusion
We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh framework optimized across multiple text prompts simultaneously.
Diffusion Priors for Dynamic View Synthesis from Monocular Videos
Dynamic novel view synthesis aims to capture the temporal evolution of visual content within videos.
SceneWiz3D: Towards Text-guided 3D Scene Composition
We are witnessing significant breakthroughs in the technology for generating 3D objects from text.
iNVS: Repurposing Diffusion Inpainters for Novel View Synthesis
Our approach focuses on maximizing the reuse of visible pixels from the source image.
HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusion
Our model enforces the joint learning of image appearance, spatial relationship, and geometry in a unified network, where each branch in the model complements to each other with both structural awareness and textural richness.
AutoDecoding Latent 3D Diffusion Models
We present a novel approach to the generation of static and articulated 3D assets that has a 3D autodecoder at its core.
Text-Guided Synthesis of Eulerian Cinemagraphs
We introduce Text2Cinemagraph, a fully automated method for creating cinemagraphs from text descriptions - an especially challenging task when prompts feature imaginary elements and artistic styles, given the complexity of interpreting the semantics and motions of these images.
Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors
We present Magic123, a two-stage coarse-to-fine approach for high-quality, textured 3D meshes generation from a single unposed image in the wild using both2D and 3D priors.
Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models
Most captivatingly, our PGM unlocks the director's mode, where the game is played by specifying goals for the agents in the form of a prompt.
3D generation on ImageNet
Existing 3D-from-2D generators are typically designed for well-curated single-category datasets, where all the objects have (approximately) the same scale, 3D location, and orientation, and the camera always points to the center of the scene.
Invertible Neural Skinning
Next, we combine PIN with a differentiable LBS module to build an expressive and end-to-end Invertible Neural Skinning (INS) pipeline.
Unsupervised Volumetric Animation
We propose a novel approach for unsupervised 3D animation of non-rigid deformable objects.
InfiniCity: Infinite-Scale City Synthesis
Toward infinite-scale 3D city synthesis, we propose a novel framework, InfiniCity, which constructs and renders an unconstrainedly large and 3D-grounded environment from random noises.
3DAvatarGAN: Bridging Domains for Personalized Editable Avatars
Finally, we propose a novel inversion method for 3D-GANs linking the latent spaces of the source and the target domains.
DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesis
Existing 3D-aware image synthesis approaches mainly focus on generating a single canonical object and show limited capacity in composing a complex scene containing a variety of objects.
Training and Tuning Generative Neural Radiance Fields for Attribute-Conditional 3D-Aware Face Generation
Generative Neural Radiance Fields (GNeRF) based 3D-aware GANs have demonstrated remarkable capabilities in generating high-quality images while maintaining strong 3D consistency.
Playable Environments: Video Manipulation in Space and Time
We present Playable Environments - a new representation for interactive video generation and manipulation in space and time.
3D-Aware Semantic-Guided Generative Model for Human Synthesis
However, they usually struggle to generate high-quality images representing non-rigid objects, such as the human body, which is of a great interest for many computer graphics applications.
Controllable Person Image Synthesis with Spatially-Adaptive Warped Normalization
Moreover, we propose a novel Self-Training Part Replacement (STPR) strategy to refine the model for the texture-transfer task, which improves the quality of the generated clothes and the preservation ability of non-target regions.
Motion Representations for Articulated Animation
To facilitate animation and prevent the leakage of the shape of the driving object, we disentangle shape and pose of objects in the region space.
Ranked #1 on
Video Reconstruction
on Tai-Chi-HD (512)
Playable Video Generation
This paper introduces the unsupervised learning problem of playable video generation (PVG).
Whitening for Self-Supervised Representation Learning
Most of the current self-supervised representation learning (SSL) methods are based on the contrastive loss and the instance-discrimination task, where augmented versions of the same image instance ("positives") are contrasted with instances extracted from other images ("negatives").
TriGAN: Image-to-Image Translation for Multi-Source Domain Adaptation
In this paper we propose the first approach for Multi-Source Domain Adaptation (MSDA) based on Generative Adversarial Networks.
Motion-supervised Co-Part Segmentation
To overcome this limitation, we propose a self-supervised deep learning method for co-part segmentation.
Ranked #3 on
Unsupervised Human Pose Estimation
on Tai-Chi-HD
First Order Motion Model for Image Animation
To achieve this, we decouple appearance and motion information using a self-supervised formulation.
Ranked #1 on
Video Reconstruction
on Tai-Chi-HD
DwNet: Dense warp-based network for pose-guided human video generation
In this paper, we focus on human motion transfer - generation of a video depicting a particular subject, observed in a single image, performing a series of motions exemplified by an auxiliary (driving) video.
Attention-based Fusion for Multi-source Human Image Generation
We present a generalization of the person-image generation task, in which a human image is generated conditioned on a target pose and a set X of source appearance images.
Whitening and Coloring transform for GANs
In this paper we propose to generalize both BN and cBN using a Whitening and Coloring based batch normalization.
Appearance and Pose-Conditioned Human Image Generation using Deformable GANs
Specifically, given an image xa of a person and a target pose P(xb), extracted from a different image xb, we synthesize a new image of that person in pose P(xb), while preserving the visual details in xa.
Unsupervised Domain Adaptation using Feature-Whitening and Consensus Loss
A classifier trained on a dataset seldom works on other datasets obtained under different conditions due to domain shift.
Animating Arbitrary Objects via Deep Motion Transfer
This is achieved through a deep architecture that decouples appearance and motion information.
Enhancing Perceptual Attributes with Bayesian Style Generation
Our approach takes as input a natural image and exploits recent models for deep style transfer and generative adversarial networks to change its style in order to modify a specific high-level attribute.
Whitening and Coloring batch transform for GANs
In this paper we propose to generalize both BN and cBN using a Whitening and Coloring based batch normalization.
Deformable GANs for Pose-based Human Image Generation
Specifically, given an image of a person and a target pose, we synthesize a new image of that person in the novel pose.
Ranked #5 on
Gesture-to-Gesture Translation
on NTU Hand Digit
Generative Adversarial Network
Gesture-to-Gesture Translation
+2
How to Make an Image More Memorable? A Deep Style Transfer Approach
In this work, we show that it is possible to automatically retrieve the best style seeds for a given image, thus remarkably reducing the number of human attempts needed to find a good match.
