Search Results for author:
Found 19 papers, 11 papers with code
Improving Selective Visual Question Answering by Learning from Your Peers
In this work, we explore Selective VQA in both in-distribution (ID) and OOD scenarios, where models are presented with mixtures of ID and OOD data.
Hyperbolic Image-Text Representations
Visual and linguistic concepts naturally organize themselves in a hierarchy, where a textual concept "dog" entails all images that contain dogs.
An Empirical Investigation of Domain Generalization with Empirical Risk Minimizers
Recent work demonstrates that deep neural networks trained using Empirical Risk Minimization (ERM) can generalize under distribution shift, outperforming specialized training algorithms for domain generalization.
CURI: A Benchmark for Productive Concept Learning Under Uncertainty
Humans can learn and reason under substantial uncertainty in a space of infinitely many concepts, including structured relational concepts ("a scene with objects that have the same color") and ad-hoc categories defined through goals ("objects that could fall on one's head").
Learning Optimal Representations with the Decodable Information Bottleneck
We address the question of characterizing and finding optimal representations for supervised learning.
DS-VIC: Unsupervised Discovery of Decision States for Transfer in RL
We learn to identify decision states, namely the parsimonious set of states where decisions meaningfully affect the future states an agent can reach in an environment.
IR-VIC: Unsupervised Discovery of Sub-goals for Transfer in RL
We propose a novel framework to identify sub-goals useful for exploration in sequential decision making tasks under partial observability.
Probabilistic Neural-symbolic Models for Interpretable Visual Question Answering
We propose a new class of probabilistic neural-symbolic models, that have symbolic functional programs as a latent, stochastic variable.
Generative Models of Visually Grounded Imagination
It is easy for people to imagine what a man with pink hair looks like, even if they have never seen such a person before.
Sound-Word2Vec: Learning Word Representations Grounded in Sounds
In this work, we treat sound as a first-class citizen, studying downstream textual tasks which require aural grounding.
Context-aware Captions from Context-agnostic Supervision
We introduce an inference technique to produce discriminative context-aware image captions (captions that describe differences between images or visual concepts) using only generic context-agnostic training data (captions that describe a concept or an image in isolation).
Grad-CAM: Why did you say that?
We propose a technique for making Convolutional Neural Network (CNN)-based models more transparent by visualizing input regions that are 'important' for predictions -- or visual explanations.
Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
For captioning and VQA, we show that even non-attention based models can localize inputs.
Counting Everyday Objects in Everyday Scenes
In this work, we build dedicated models for counting designed to tackle the large variance in counts, appearances, and scales of objects found in natural scenes.
Ranked #1 on
Object Counting
on COCO count-test
Learning Common Sense Through Visual Abstraction
We show that the commonsense knowledge we learn is complementary to what can be learnt from sources of text.
Visual Word2Vec (vis-w2v): Learning Visually Grounded Word Embeddings Using Abstract Scenes
While word embeddings trained using text have been extremely successful, they cannot uncover notions of semantic relatedness implicit in our visual world.
Microsoft COCO Captions: Data Collection and Evaluation Server
In this paper we describe the Microsoft COCO Caption dataset and evaluation server.
CIDEr: Consensus-based Image Description Evaluation
We propose a novel paradigm for evaluating image descriptions that uses human consensus.
Collecting Image Description Datasets using Crowdsourcing
We describe our two new datasets with images described by humans.
