Visual Question Answering (VQA)
767 papers with code • 62 benchmarks • 112 datasets
Visual Question Answering (VQA) is a task in computer vision that involves answering questions about an image. The goal of VQA is to teach machines to understand the content of an image and answer questions about it in natural language.
Image Source: visualqa.org
Benchmarks
These leaderboards are used to track progress in Visual Question Answering (VQA)
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Visual Question Answering (VQA) models and implementationsDatasets
MS COCO
Visual Genome
CLEVR
GQA
Visual Question Answering v2.0
Conceptual Captions
OK-VQA
TextVQA
VCR
VisDial
Subtasks
Latest papers
OmniFusion Technical Report
We propose an \textit{OmniFusion} model based on a pretrained LLM and adapters for visual modality.
MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding
boheumd/MA-LMM
•
•
However, existing LLM-based large multimodal models (e. g., Video-LLaMA, VideoChat) can only take in a limited number of frames for short video understanding.
Joint Visual and Text Prompting for Improved Object-Centric Perception with Multimodal Large Language Models
In this paper, we present a novel approach, Joint Visual and Text Prompting (VTPrompt), that employs fine-grained visual information to enhance the capability of MLLMs in VQA, especially for object-oriented perception.
Evaluating Text-to-Visual Generation with Image-to-Text Generation
For instance, the widely-used CLIPScore measures the alignment between a (generated) image and text prompt, but it fails to produce reliable scores for complex prompts involving compositions of objects, attributes, and relations.
Unsolvable Problem Detection: Evaluating Trustworthiness of Vision Language Models
atsumiyai/upd
•
•
This paper introduces a novel and significant challenge for Vision Language Models (VLMs), termed Unsolvable Problem Detection (UPD).
A Gaze-grounded Visual Question Answering Dataset for Clarifying Ambiguous Japanese Questions
Such ambiguities in questions are often clarified by the contexts in conversational situations, such as joint attention with a user or user gaze information.
Intrinsic Subgraph Generation for Interpretable Graph based Visual Question Answering
In this work, we introduce an interpretable approach for graph-based VQA and demonstrate competitive performance on the GQA dataset.
IllusionVQA: A Challenging Optical Illusion Dataset for Vision Language Models
GPT4V, the best-performing VLM, achieves 62. 99% accuracy (4-shot) on the comprehension task and 49. 7% on the localization task (4-shot and Chain-of-Thought).
MedPromptX: Grounded Multimodal Prompting for Chest X-ray Diagnosis
biomedia-mbzuai/medpromptx
•
•
Chest X-ray images are commonly used for predicting acute and chronic cardiopulmonary conditions, but efforts to integrate them with structured clinical data face challenges due to incomplete electronic health records (EHR).
Multi-Agent VQA: Exploring Multi-Agent Foundation Models in Zero-Shot Visual Question Answering
bowen-upenn/Multi-Agent-VQA
•
•
This work explores the zero-shot capabilities of foundation models in Visual Question Answering (VQA) tasks.
