TallyQA: Answering Complex Counting Questions

29 Oct 2018  ·  Manoj Acharya, Kushal Kafle, Christopher Kanan ·

Most counting questions in visual question answering (VQA) datasets are simple and require no more than object detection. Here, we study algorithms for complex counting questions that involve relationships between objects, attribute identification, reasoning, and more. To do this, we created TallyQA, the world's largest dataset for open-ended counting. We propose a new algorithm for counting that uses relation networks with region proposals. Our method lets relation networks be efficiently used with high-resolution imagery. It yields state-of-the-art results compared to baseline and recent systems on both TallyQA and the HowMany-QA benchmark.

PDF Abstract

Datasets


Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Object Counting HowMany-QA RCN Accuracy 60.3 # 3
RMSE 2.35 # 2
Object Counting TallyQA-Complex RCN Accuracy 56.2 # 5
RMSE 1.43 # 1
Object Counting TallyQA-Simple RCN Accuracy 71.8 # 5
RMSE 1.13 # 3

Methods


No methods listed for this paper. Add relevant methods here