Search Results for author:
Found 51 papers, 37 papers with code
Aligning and Projecting Images to Class-conditional Generative Networks
We present a method for projecting an input image into the space of a class-conditional generative neural network.
Customizing Text-to-Image Diffusion with Camera Viewpoint Control
Model customization introduces new concepts to existing text-to-image models, enabling the generation of the new concept in novel contexts.
Lazy Diffusion Transformer for Interactive Image Editing
We demonstrate that our approach is competitive with state-of-the-art inpainting methods in terms of quality and fidelity while providing a 10x speedup for typical user interactions, where the editing mask represents 10% of the image.
VideoGigaGAN: Towards Detail-rich Video Super-Resolution
We introduce VideoGigaGAN, a new generative VSR model that can produce videos with high-frequency details and temporal consistency.
Jump Cut Smoothing for Talking Heads
We leverage the appearance of the subject from the other source frames in the video, fusing it with a mid-level representation driven by DensePose keypoints and face landmarks.
Customizing Motion in Text-to-Video Diffusion Models
To avoid overfitting to the new custom motion, we introduce an approach for regularization over videos.
One-step Diffusion with Distribution Matching Distillation
We introduce Distribution Matching Distillation (DMD), a procedure to transform a diffusion model into a one-step image generator with minimal impact on image quality.
Online Detection of AI-Generated Images
With advancements in AI-generated images coming on a continuous basis, it is increasingly difficult to distinguish traditionally-sourced images (e. g., photos, artwork) from AI-generated ones.
DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data
Furthermore, our metric outperforms both prior learned metrics and recent large vision models on these tasks.
Evaluating Data Attribution for Text-to-Image Models
The problem of data attribution in such models -- which of the images in the training set are most responsible for the appearance of a given generated image -- is a difficult yet important one.
Ablating Concepts in Text-to-Image Diffusion Models
To achieve this goal, we propose an efficient method of ablating concepts in the pretrained model, i. e., preventing the generation of a target concept.
Scaling up GANs for Text-to-Image Synthesis
From a technical standpoint, it also marked a drastic change in the favored architecture to design generative image models.
Ranked #18 on
Image Generation
on ImageNet 256x256
Zero-shot Image-to-Image Translation
However, it is still challenging to directly apply these models for editing real images for two reasons.
Ranked #13 on
Text-based Image Editing
on PIE-Bench
Domain Expansion of Image Generators
First, we note the generator contains a meaningful, pretrained latent space.
Multi-Concept Customization of Text-to-Image Diffusion
Can we teach a model to quickly acquire a new concept, given a few examples?
3D-FM GAN: Towards 3D-Controllable Face Manipulation
While concatenating GAN inversion and a 3D-aware, noise-to-image GAN is a straight-forward solution, it is inefficient and may lead to noticeable drop in editing quality.
Spatially-Adaptive Multilayer Selection for GAN Inversion and Editing
We propose a new method to invert and edit such complex images in the latent space of GANs, such as StyleGAN2.
ASSET: Autoregressive Semantic Scene Editing with Transformers at High Resolutions
We present ASSET, a neural architecture for automatically modifying an input high-resolution image according to a user's edits on its semantic segmentation map.
BlobGAN: Spatially Disentangled Scene Representations
Blobs are differentiably placed onto a feature grid that is decoded into an image by a generative adversarial network.
Any-resolution Training for High-resolution Image Synthesis
To take advantage of varied-size data, we introduce continuous-scale training, a process that samples patches at random scales to train a new generator with variable output resolutions.
Ensembling Off-the-shelf Models for GAN Training
Can the collective "knowledge" from a large bank of pretrained vision models be leveraged to improve GAN training?
Ranked #1 on
Image Generation
on AFHQ Cat
GAN-Supervised Dense Visual Alignment
We propose GAN-Supervised Learning, a framework for learning discriminative models and their GAN-generated training data jointly end-to-end.
Preconditioned Gradient Descent for Over-Parameterized Nonconvex Matrix Factorization
This over-parameterized regime of matrix factorization significantly slows down the convergence of local search algorithms, from a linear rate with $r=r^{\star}$ to a sublinear rate when $r>r^{\star}$.
Contrastive Feature Loss for Image Prediction
Training supervised image synthesis models requires a critic to compare two images: the ground truth to the result.
Editing Conditional Radiance Fields
In this paper, we explore enabling user editing of a category-level NeRF - also known as a conditional radiance field - trained on a shape category.
Ranked #1 on
Novel View Synthesis
on PhotoShape
Ensembling with Deep Generative Views
Here, we investigate whether such views can be applied to real images to benefit downstream analysis tasks such as image classification.
On Aliased Resizing and Surprising Subtleties in GAN Evaluation
Furthermore, we show that if compression is used on real training images, FID can actually improve if the generated images are also subsequently compressed.
Few-shot Image Generation via Cross-domain Correspondence
Training generative models, such as GANs, on a target domain containing limited examples (e. g., 10) can easily result in overfitting.
Ranked #3 on
10-shot image generation
on Babies
The Low-Rank Simplicity Bias in Deep Networks
We show empirically that our claim holds true on finite width linear and non-linear models on practical learning paradigms and show that on natural data, these are often the solutions that generalize well.
Anycost GANs for Interactive Image Synthesis and Editing
Generative adversarial networks (GANs) have enabled photorealistic image synthesis and editing.
CDPAM: Contrastive learning for perceptual audio similarity
The DPAM approach of Manocha et al. learns a full-reference metric trained directly on human judgments, and thus correlates well with human perception.
Spatially-Adaptive Pixelwise Networks for Fast Image Translation
We introduce a new generator architecture, aimed at fast and efficient high-resolution image-to-image translation.
Few-shot Image Generation with Elastic Weight Consolidation
Few-shot image generation seeks to generate more data of a given domain, with only few available training examples.
Ranked #4 on
10-shot image generation
on Babies
How many samples is a good initial point worth in Low-rank Matrix Recovery?
Optimizing the threshold over regions of the landscape, we see that, for initial points not too close to the ground truth, a linear improvement in the quality of the initial guess amounts to a constant factor improvement in the sample complexity.
Contrastive Learning for Unpaired Image-to-Image Translation
Furthermore, we draw negatives from within the input image itself, rather than from the rest of the dataset.
Swapping Autoencoder for Deep Image Manipulation
Deep generative models have become increasingly effective at producing realistic images from randomly sampled seeds, but using such models for controllable manipulation of existing images remains challenging.
Transforming and Projecting Images into Class-conditional Generative Networks
We present a method for projecting an input image into the space of a class-conditional generative neural network.
Image Morphing with Perceptual Constraints and STN Alignment
In image morphing, a sequence of plausible frames are synthesized and composited together to form a smooth transformation between given instances.
A Differentiable Perceptual Audio Metric Learned from Just Noticeable Differences
Assessment of many audio processing tasks relies on subjective evaluation which is time-consuming and expensive.
CNN-generated images are surprisingly easy to spot... for now
In this work we ask whether it is possible to create a "universal" detector for telling apart real images from these generated by a CNN, regardless of architecture or dataset used.
Interactive Sketch & Fill: Multiclass Sketch-to-Image Translation
We propose an interactive GAN-based sketch-to-image translation method that helps novice users create images of simple objects.
Detecting Photoshopped Faces by Scripting Photoshop
Most malicious photo manipulations are created using standard image editing tools, such as Adobe Photoshop.
Making Convolutional Networks Shift-Invariant Again
The well-known signal processing fix is anti-aliasing by low-pass filtering before downsampling.
Ranked #26 on
Domain Generalization
on VizWiz-Classification
Deep Parametric Shape Predictions using Distance Fields
Many tasks in graphics and vision demand machinery for converting shapes into consistent representations with sparse sets of parameters; these representations facilitate rendering, editing, and storage.
Stochastic Adversarial Video Prediction
However, learning to predict raw future observations, such as frames in a video, is exceedingly challenging -- the ambiguous nature of the problem can cause a naively designed model to average together possible futures into a single, blurry prediction.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
We systematically evaluate deep features across different architectures and tasks and compare them with classic metrics.
Ranked #19 on
Video Quality Assessment
on MSU FR VQA Database
Self-Supervised Learning of Object Motion Through Adversarial Video Prediction
In this paper, we study the problem of multi-step video prediction, where the goal is to predict a sequence of future frames conditioned on a short context.
Toward Multimodal Image-to-Image Translation
Our proposed method encourages bijective consistency between the latent encoding and output modes.
Real-Time User-Guided Image Colorization with Learned Deep Priors
The system directly maps a grayscale image, along with sparse, local user "hints" to an output colorization with a Convolutional Neural Network (CNN).
Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction
We propose split-brain autoencoders, a straightforward modification of the traditional autoencoder architecture, for unsupervised representation learning.
Ranked #127 on
Self-Supervised Image Classification
on ImageNet
Representation Learning
Self-Supervised Image Classification
+1
Colorful Image Colorization
We embrace the underlying uncertainty of the problem by posing it as a classification task and use class-rebalancing at training time to increase the diversity of colors in the result.
Ranked #128 on
Self-Supervised Image Classification
on ImageNet
