Search Results for author:
Found 48 papers, 30 papers with code
PropTest: Automatic Property Testing for Improved Visual Programming
Visual Programming has emerged as an alternative to end-to-end black-box visual reasoning models.
Learning from Models and Data for Visual Grounding
We introduce SynGround, a novel framework that combines data-driven learning and knowledge transfer from various large-scale pretrained models to enhance the visual grounding capabilities of a pretrained vision-and-language model.
Grounding Language Models for Visual Entity Recognition
We introduce AutoVER, an Autoregressive model for Visual Entity Recognition.
Improved Visual Grounding through Self-Consistent Explanations
Vision-and-language models trained to match images with text can be combined with visual explanation methods to point to the locations of specific objects in an image.
ElasticDiffusion: Training-free Arbitrary Size Image Generation through Global-Local Content Separation
We propose ElasticDiffusion, a novel training-free decoding method that enables pretrained text-to-image diffusion models to generate images with various sizes.
Characterizing Video Question Answering with Sparsified Inputs
From our experiments, we have observed only 5. 2%-5. 8% loss of performance with only 10% of video lengths, which corresponds to 2-4 frames selected from each video.
SCoRD: Subject-Conditional Relation Detection with Text-Augmented Data
To solve this problem, we propose an auto-regressive model that given a subject, it predicts its relations, objects, and object locations by casting this output as a sequence of tokens.
Going Beyond Nouns With Vision & Language Models Using Synthetic Data
We contribute Synthetic Visual Concepts (SyViC) - a million-scale synthetic dataset and data generation codebase allowing to generate additional suitable data to improve VLC understanding and compositional reasoning of VL models.
Ranked #68 on
Visual Reasoning
on Winoground
ViC-MAE: Self-Supervised Representation Learning from Images and Video with Contrastive Masked Autoencoders
We show that visual representations learned under ViC-MAE generalize well to both video and image classification tasks.
Ranked #4 on
Image Classification
on Places365
(using extra training data)
Variation of Gender Biases in Visual Recognition Models Before and After Finetuning
We also find that (2) models finetuned on larger scale datasets are more likely to introduce new biased associations.
On the Transferability of Visual Features in Generalized Zero-Shot Learning
Generalized Zero-Shot Learning (GZSL) aims to train a classifier that can generalize to unseen classes, using a set of attributes as auxiliary information, and the visual features extracted from a pre-trained convolutional neural network.
Improving Visual Grounding by Encouraging Consistent Gradient-based Explanations
We propose a margin-based loss for tuning joint vision-language models so that their gradient-based explanations are consistent with region-level annotations provided by humans for relatively smaller grounding datasets.
Towards Understanding Gender-Seniority Compound Bias in Natural Language Generation
Our results show that GPT-2 amplifies bias by considering women as junior and men as senior more often than the ground truth in both domains.
SimVQA: Exploring Simulated Environments for Visual Question Answering
By exploiting 3D and physics simulation platforms, we provide a pipeline to generate synthetic data to expand and replace type-specific questions and answers without risking the exposure of sensitive or personal data that might be present in real images.
Repairing Group-Level Errors for DNNs Using Weighted Regularization
In particular, it can repair confusion error and bias error of DNN models for both single-label and multi-label image classifications.
CLIP-Lite: Information Efficient Visual Representation Learning with Language Supervision
We propose CLIP-Lite, an information efficient method for visual representation learning by feature alignment with textual annotations.
Estimating and Maximizing Mutual Information for Knowledge Distillation
Our empirical results show that MIMKD outperforms competing approaches across a wide range of student-teacher pairs with different capacities, with different architectures, and when student networks are with extremely low capacity.
Evolving Image Compositions for Feature Representation Learning
This paper proposes PatchMix, a data augmentation method that creates new samples by composing patches from pairs of images in a grid-like pattern.
Instance-level Image Retrieval using Reranking Transformers
Instance-level image retrieval is the task of searching in a large database for images that match an object in a query image.
Ranked #3 on
Image Retrieval
on RParis (Medium)
Chair Segments: A Compact Benchmark for the Study of Object Segmentation
We aim for ChairSegments to be the equivalent of the CIFAR-10 dataset but for quickly designing and iterating over novel model architectures for segmentation.
General Multi-label Image Classification with Transformers
Multi-label image classification is the task of predicting a set of labels corresponding to objects, attributes or other entities present in an image.
Using Visual Feature Space as a Pivot Across Languages
Our work aims to leverage visual feature space to pass information across languages.
Visual News: Benchmark and Challenges in News Image Captioning
We propose Visual News Captioner, an entity-aware model for the task of news image captioning.
Black-box Explanation of Object Detectors via Saliency Maps
We propose D-RISE, a method for generating visual explanations for the predictions of object detectors.
Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation
Word embeddings derived from human-generated corpora inherit strong gender bias which can be further amplified by downstream models.
Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning
Pseudo-labeling works by applying pseudo-labels to samples in the unlabeled set by using a model trained on the combination of the labeled samples and any previously pseudo-labeled samples, and iteratively repeating this process in a self-training cycle.
MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning
Inspired by human visual attention, we propose a novel inverse reinforcement learning formulation using Maximum Entropy Deep Inverse Reinforcement Learning (MEDIRL) for predicting the visual attention of drivers in accident-prone situations.
Drill-down: Interactive Retrieval of Complex Scenes using Natural Language Queries
We show that using multiple rounds of natural language queries as input can be surprisingly effective to find arbitrarily specific images of complex scenes.
Moviescope: Large-scale Analysis of Movies using Multiple Modalities
Film media is a rich form of artistic expression.
Testing DNN Image Classifiers for Confusion & Bias Errors
We found that many of the reported erroneous cases in popular DNN image classifiers occur because the trained models confuse one class with another or show biases towards some classes over others.
Gender Bias in Contextualized Word Embeddings
In this paper, we quantify, analyze and mitigate gender bias exhibited in ELMo's contextualized word vectors.
Chat-crowd: A Dialog-based Platform for Visual Layout Composition
In this paper we introduce Chat-crowd, an interactive environment for visual layout composition via conversational interactions.
Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations
In this work, we present a framework to measure and mitigate intrinsic biases with respect to protected variables --such as gender-- in visual recognition tasks.
Text2Scene: Generating Compositional Scenes from Textual Descriptions
In this paper, we propose Text2Scene, a model that generates various forms of compositional scene representations from natural language descriptions.
Deep Feature Aggregation and Image Re-ranking with Heat Diffusion for Image Retrieval
We show that by considering each deep feature as a heat source, our unsupervised aggregation method is able to avoid over-representation of \emph{bursty} features.
Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods
We introduce a new benchmark, WinoBias, for coreference resolution focused on gender bias.
Feedback-prop: Convolutional Neural Network Inference under Partial Evidence
We propose an inference procedure for deep convolutional neural networks (CNNs) when partial evidence is available.
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Language is increasingly being used to define rich visual recognition problems with supporting image collections sourced from the web.
OBJ2TEXT: Generating Visually Descriptive Language from Object Layouts
Generating captions for images is a task that has recently received considerable attention.
Where and Who? Automatic Semantic-Aware Person Composition
Image compositing is a method used to generate realistic yet fake imagery by inserting contents from one image to another.
Commonly Uncommon: Semantic Sparsity in Situation Recognition
Semantic sparsity is a common challenge in structured visual classification problems; when the output space is complex, the vast majority of the possible predictions are rarely, if ever, seen in the training set.
Ranked #11 on
Situation Recognition
on imSitu
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
We propose two efficient approximations to standard convolutional neural networks: Binary-Weight-Networks and XNOR-Networks.
Ranked #10 on
Classification with Binary Neural Network
on ImageNet
TreeTalk: Composition and Compression of Trees for Image Descriptions
We present a new tree based approach to composing expressive image descriptions that makes use of naturally occuring web images with captions.
Im2Text: Describing Images Using 1 Million Captioned Photographs
We develop and demonstrate automatic image description methods using a large captioned photo collection.
