Search Results for author:
Found 61 papers, 26 papers with code
DreamFlow: High-Quality Text-to-3D Generation by Approximating Probability Flow
Recent progress in text-to-3D generation has been achieved through the utilization of score distillation methods: they make use of the pre-trained text-to-image (T2I) diffusion models by distilling via the diffusion model training objective.
Direct Consistency Optimization for Compositional Text-to-Image Personalization
In particular, our method results in a superior Pareto frontier to the baselines.
Unsupervised LLM Adaptation for Question Answering
In this task, we leverage a pre-trained LLM, a publicly available QA dataset (source data), and unlabeled documents from the target domain.
Instruct-Imagen: Image Generation with Multi-modal Instruction
We introduce *multi-modal instruction* for image generation, a task representation articulating a range of generation intents with precision.
VideoPoet: A Large Language Model for Zero-Shot Video Generation
We present VideoPoet, a language model capable of synthesizing high-quality video, with matching audio, from a large variety of conditioning signals.
Ranked #3 on
Text-to-Video Generation
on MSR-VTT
Photorealistic Video Generation with Diffusion Models
We present W. A. L. T, a transformer-based approach for photorealistic video generation via diffusion modeling.
Ranked #1 on
Video Prediction
on Kinetics-600 12 frames, 64x64
Improve Supervised Representation Learning with Masked Image Modeling
On K-Nearest-Neighbor image retrieval evaluation with ImageNet-1k, the same model outperforms the baseline by 1. 32%.
Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation
While Large Language Models (LLMs) are the dominant models for generative tasks in language, they do not perform as well as diffusion models on image and video generation.
Ranked #2 on
Video Prediction
on Kinetics-600 12 frames, 64x64
Label Budget Allocation in Multi-Task Learning
How should the label budget (i. e. the amount of money spent on labeling) be allocated among different tasks to achieve optimal multi-task performance?
Collaborative Score Distillation for Consistent Visual Synthesis
Generative priors of large-scale text-to-image diffusion models enable a wide range of new generation and editing applications on diverse visual modalities.
Learning Disentangled Prompts for Compositional Image Synthesis
We study domain-adaptive image synthesis, the problem of teaching pretrained image generative models a new style or concept from as few as one image to synthesize novel images, to better understand the compositional image synthesis.
StyleDrop: Text-to-Image Generation in Any Style
Pre-trained large text-to-image models synthesize impressive images with an appropriate use of text prompts.
FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction
In FormNetV2, we introduce a centralized multimodal graph contrastive learning strategy to unify self-supervised pre-training for all modalities in one loss.
Video Probabilistic Diffusion Models in Projected Latent Space
Specifically, PVDM is composed of two components: (a) an autoencoder that projects a given video as 2D-shaped latent vectors that factorize the complex cubic structure of video pixels and (b) a diffusion model architecture specialized for our new factorized latent space and the training/sampling procedure to synthesize videos of arbitrary length with a single model.
MaskSketch: Unpaired Structure-guided Masked Image Generation
We show that intermediate self-attention maps of a masked generative transformer encode important structural information of the input image, such as scene layout and object shape, and we propose a novel sampling method based on this observation to enable structure-guided generation.
Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval
Existing methods rely on supervised learning of CIR models using labeled triplets consisting of the query image, text specification, and the target image.
Ranked #1 on
Zero-shot Image Retrieval
on ImageNet-R
MAGVIT: Masked Generative Video Transformer
We introduce the MAsked Generative VIdeo Transformer, MAGVIT, to tackle various video synthesis tasks with a single model.
Ranked #1 on
Video Prediction
on Something-Something V2
SPADE: Semi-supervised Anomaly Detection under Distribution Mismatch
Semi-supervised anomaly detection is a common problem, as often the datasets containing anomalies are partially labeled.
Semi-supervised Anomaly Detection
Supervised Anomaly Detection
Visual Prompt Tuning for Generative Transfer Learning
We base our framework on state-of-the-art generative vision transformers that represent an image as a sequence of visual tokens to the autoregressive or non-autoregressive transformers.
Prefix Conditioning Unifies Language and Label Supervision
In experiments, we show that this simple technique improves the performance in zero-shot image recognition accuracy and robustness to the image-level distribution shift.
Federated Semi-Supervised Learning with Prototypical Networks
Compared to a FSSL approach based on weight sharing, the prototype-based inter-client knowledge sharing significantly reduces both communication and computation costs, enabling more frequent knowledge sharing between more clients for better accuracy.
Towards Group Robustness in the presence of Partial Group Labels
Such a requirement is impractical in situations where the data labeling efforts for minority or rare groups are significantly laborious or where the individuals comprising the dataset choose to conceal sensitive information.
Anomaly Clustering: Grouping Images into Coherent Clusters of Anomaly Types
We define a distance function between images, each of which is represented as a bag of embeddings, by the Euclidean distance between weighted averaged embeddings.
Invariant Learning with Partial Group Labels
Such a requirement is impractical in situations where the data labelling efforts for minority or rare groups is significantly laborious or where the individuals comprising the dataset choose to conceal sensitive information.
Unifying Distribution Alignment as a Loss for Imbalanced Semi-supervised Learning
While remarkable progress in imbalanced supervised learning has been made recently, less attention has been given to the setting of imbalanced semi-supervised learning (SSL) where not only is a few labeled data provided, but the underlying data distribution can be severely imbalanced.
Object-aware Contrastive Learning for Debiased Scene Representation
Contrastive self-supervised learning has shown impressive results in learning visual representations from unlabeled images by enforcing invariance against different data augmentations.
Controlling Neural Networks with Rule Representations
The key aspect of DeepCTRL is that it does not require retraining to adapt the rule strength -- at inference, the user can adjust it based on the desired operation point on accuracy vs. rule verification ratio.
Self-supervise, Refine, Repeat: Improving Unsupervised Anomaly Detection
We demonstrate our method on various unsupervised AD tasks with image and tabular data.
AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation
We extend semi-supervised learning to the problem of domain adaptation to learn significantly higher-accuracy models that train on one data distribution and test on a different one.
Semi-supervised Domain Adaptation
Unsupervised Domain Adaptation
CutPaste: Self-Supervised Learning for Anomaly Detection and Localization
We aim at constructing a high performance model for defect detection that detects unknown anomalous patterns of an image without anomalous data.
Ranked #55 on
Anomaly Detection
on MVTec AD
CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning
Semi-supervised learning on class-imbalanced data, although a realistic problem, has been under studied.
Assessing Post-Disaster Damage from Satellite Imagery using Semi-Supervised Learning Techniques
To respond to disasters such as earthquakes, wildfires, and armed conflicts, humanitarian organizations require accurate and timely data in the form of damage assessments, which indicate what buildings and population centers have been most affected.
Learning and Evaluating Representations for Deep One-class Classification
We first learn self-supervised representations from one-class data, and then build one-class classifiers on learned representations.
Ranked #7 on
Anomaly Detection
on One-class CIFAR-100
i-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning
Contrastive representation learning has shown to be effective to learn representations from unlabeled data.
Improving Face Recognition by Clustering Unlabeled Faces in the Wild
While deep face recognition has benefited significantly from large-scale labeled data, current research is focused on leveraging unlabeled data to further boost performance, reducing the cost of human annotation.
A Simple Semi-Supervised Learning Framework for Object Detection
Semi-supervised learning (SSL) has a potential to improve the predictive performance of machine learning models using unlabeled data.
Ranked #13 on
Semi-Supervised Object Detection
on COCO 100% labeled data
(using extra training data)
ReMixMatch: Semi-Supervised Learning with Distribution Matching and Augmentation Anchoring
We improve the recently-proposed ``MixMatch semi-supervised learning algorithm by introducing two new techniques: distribution alignment and augmentation anchoring.
Towards Universal Representation Learning for Deep Face Recognition
Recognizing wild faces is extremely hard as they appear with all kinds of variations.
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
Semi-supervised learning (SSL) provides an effective means of leveraging unlabeled data to improve a model's performance.
Adversarial Learning of Privacy-Preserving and Task-Oriented Representations
For instance, there could be a potential privacy risk of machine learning systems via the model inversion attack, whose goal is to reconstruct the input data from the latent representation of deep networks.
ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
Distribution alignment encourages the marginal distribution of predictions on unlabeled data to be close to the marginal distribution of ground-truth labels.
Adaptation Across Extreme Variations using Unlabeled Domain Bridges
We tackle an unsupervised domain adaptation problem for which the domain discrepancy between labeled source and unlabeled target domains is large, due to many factors of inter and intra-domain variation.
Domain Adaptation for Structured Output via Disentangled Patch Representations
To this end, we propose to learn discriminative feature representations of patches based on label histograms in the source domain, through the construction of a disentangled space.
Unsupervised Domain Adaptation for Distance Metric Learning
Unsupervised domain adaptation is a promising avenue to enhance the performance of deep neural networks on a target domain, using labels only from a source domain.
Active Adversarial Domain Adaptation
Our approach, active adversarial domain adaptation (AADA), explores a duality between two related problems: adversarial domain alignment and importance sampling for adapting models across domains.
Domain Adaptation for Structured Output via Discriminative Patch Representations
Predicting structured outputs such as semantic segmentation relies on expensive per-pixel annotations to learn supervised models like convolutional neural networks.
Ranked #22 on
Image-to-Image Translation
on SYNTHIA-to-Cityscapes
Learning Gibbs-regularized GANs with variational discriminator reparameterization
We propose a novel approach to regularizing generative adversarial networks (GANs) leveraging learned {\em structured Gibbs distributions}.
Feature Transfer Learning for Deep Face Recognition with Under-Represented Data
In this paper, we propose a center-based feature transfer framework to augment the feature space of under-represented subjects from the regular subjects that have sufficiently diverse samples.
Learning to Adapt Structured Output Space for Semantic Segmentation
In this paper, we propose an adversarial learning method for domain adaptation in the context of semantic segmentation.
Ranked #3 on
Domain Adaptation
on Synscapes-to-Cityscapes
Gotta Adapt 'Em All: Joint Pixel and Feature-Level Domain Adaptation for Recognition in the Wild
Recent developments in deep domain adaptation have allowed knowledge transfer from a labeled source domain to an unlabeled target domain at the level of intermediate features or input pixels.
Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos
Despite rapid advances in face recognition, there remains a clear gap between the performance of still image-based face recognition and video-based face recognition, due to the vast difference in visual quality between the domains and the difficulty of curating diverse large-scale video datasets.
Channel-Recurrent Autoencoding for Image Modeling
Despite recent successes in synthesizing faces and bedrooms, existing generative models struggle to capture more complex image types, potentially due to the oversimplification of their latent space constructions.
Towards Large-Pose Face Frontalization in the Wild
Despite recent advances in face recognition using deep learning, severe accuracy drops are observed for large pose variations in unconstrained environments.
Reconstruction-Based Disentanglement for Pose-invariant Face Recognition
Finally, we propose a new feature reconstruction metric learning to explicitly disentangle identity and pose, by demanding alignment between the feature reconstructions through various combinations of identity and pose features, which is obtained from two images of the same subject.
Improved Deep Metric Learning with Multi-class N-pair Loss Objective
Deep metric learning has gained much popularity in recent years, following the success of deep learning.
Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units
Recently, convolutional neural networks (CNNs) have been used as a powerful tool to solve many problems of machine learning and computer vision.
Attribute2Image: Conditional Image Generation from Visual Attributes
This paper investigates a novel problem of generating images from visual attributes.
Learning Structured Output Representation using Deep Conditional Generative Models
The model is trained efficiently in the framework of stochastic gradient variational Bayes, and allows a fast prediction using stochastic feed-forward inference.
Ranked #1 on
Structured Prediction
on MNIST
Improving Object Detection with Deep Convolutional Networks via Bayesian Optimization and Structured Prediction
Object detection systems based on the deep convolutional neural network (CNN) have recently made ground- breaking advances on several object detection benchmarks.
Improved Multimodal Deep Learning with Variation of Information
Deep learning has been successfully applied to multimodal representation learning problems, with a common strategy to learning joint representations that are shared across multiple modalities on top of layers of modality-specific networks.
Augmenting CRFs with Boltzmann Machine Shape Priors for Image Labeling
Although the CRF is a good baseline labeler, we show how an RBM can be added to the architecture to provide a global shape bias that complements the local modeling provided by the CRF.
