Search Results for author:
Found 17 papers, 8 papers with code
TraveLER: A Multi-LMM Agent Framework for Video Question-Answering
Specifically, we propose TraveLER, a model that can create a plan to "Traverse" through the video, ask questions about individual frames to "Locate" and store key information, and then "Evaluate" if there is enough information to answer the question.
Recursive Visual Programming
Visual Programming (VP) has emerged as a powerful framework for Visual Question Answering (VQA).
From Wrong To Right: A Recursive Approach Towards Vision-Language Explanation
Addressing the challenge of adapting pre-trained vision-language models for generating insightful explanations for visual reasoning tasks with limited annotations, we present ReVisE: a $\textbf{Re}$cursive $\textbf{Vis}$ual $\textbf{E}$xplanation algorithm.
Can Language Models Learn to Listen?
We present a framework for generating appropriate facial responses from a listener in dyadic social interactions based on the speaker's words.
Modular Visual Question Answering via Code Generation
We present a framework that formulates visual question answering as modular code generation.
ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension
Training a referring expression comprehension (ReC) model for a new visual domain requires collecting referring expressions, and potentially corresponding bounding boxes, for images in the domain.
MedICaT: A Dataset of Medical Images, Captions, and Textual References
To address challenges in figure retrieval and figure-to-text alignment, we introduce MedICaT, a dataset of medical images in context.
Evaluating NLP Models via Contrast Sets
Unfortunately, when a dataset has systematic gaps (e. g., annotation artifacts), these evaluations are misleading: a model can learn simple decision rules that perform well on the test set but do not capture a dataset's intended capabilities.
Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answering
However, state-of-the-art models in grounded question answering often do not explicitly perform decomposition, leading to difficulties in generalization to out-of-distribution examples.
Obtaining Faithful Interpretations from Compositional Neural Networks
Neural module networks (NMNs) are a popular approach for modeling compositionality: they achieve high accuracy when applied to problems in language and vision, while reflecting the compositional structure of the problem in the network architecture.
Evaluating Models' Local Decision Boundaries via Contrast Sets
Unfortunately, when a dataset has systematic gaps (e. g., annotation artifacts), these evaluations are misleading: a model can learn simple decision rules that perform well on the test set but do not capture a dataset's intended capabilities.
AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models
Neural NLP models are increasingly accurate but are imperfect and opaque---they break in counterintuitive ways and leave end users puzzled at their behavior.
An Improved Neural Baseline for Temporal Relation Extraction
Determining temporal relations (e. g., before or after) between events has been a challenging natural language understanding task, partly due to the difficulty to generate large amounts of high-quality training data.
Correlation Clustering with Same-Cluster Queries Bounded by Optimal Cost
An algorithm in this setting has access to an oracle with full knowledge of an optimal clustering, and the algorithm can ask the oracle queries of the form, "Does the optimal clustering put vertices $ u $ and $ v $ in the same cluster?"
Improving Generalization in Coreference Resolution via Adversarial Training
In order for coreference resolution systems to be useful in practice, they must be able to generalize to new text.
Evaluation of named entity coreference
It is therefore important that coreference resolution systems are able to link these different types of mentions to the correct entity name.
Named Person Coreference in English News
Here, we evaluate two state of the art coreference resolution systems on the subtask of Named Person Coreference, in which we are interested in identifying a person mentioned by name, along with all other mentions of the person, by pronoun or generic noun phrase.
