Ensemble of MRR and NDCG models for Visual Dialog

15 Apr 2021 โ€ข Idan Schwartz

Assessing an AI agent that can converse in human language and understand visual content is challenging. Generation metrics, such as BLEU scores favor correct syntax over semantics... (read more)

PDF Abstract

Datasets


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Visual Dialog VisDial v1.0 test-std Two-Step MRR 0.7041 # 2
Mean Rank 3.66 # 1
NDCG 72.16 # 1
R@1 58.18 # 2
R@10 90.83 # 2
R@5 83.85 # 2
Visual Dialog VisDial v1.0 test-std 5xFGA + LS*+ MRR 0.7124 # 1
Mean Rank 2.96 # 2
R@1 58.28 # 1
R@10 94.45 # 1
R@5 87.55 # 1
Visual Dialog VisDial v1.0 test-std 5xFGA + LS NDCG 64.04 # 2
Visual Dialog Visual Dialog v1.0 test-std 2 Step: Factor Graph Attention + VD-Bert NDCG (x 100) 72.83 # 13
MRR (x 100) 69.92 # 4
R@1 58.3 # 1
R@5 81.55 # 14
R@10 89.6 # 23
Mean 3.84 # 40

Methods used in the Paper


METHOD TYPE
๐Ÿค– No Methods Found Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet