Large-scale Simple Question Answering with Memory Networks

5 Jun 2015  ·  Antoine Bordes, Nicolas Usunier, Sumit Chopra, Jason Weston ·

Training large-scale question answering systems is complicated because training sources usually cover a small portion of the range of possible questions. This paper studies the impact of multitask and transfer learning for simple question answering; a setting for which the reasoning required to answer is quite easy, as long as one can retrieve the correct evidence given a question, which can be difficult in large-scale conditions. To this end, we introduce a new dataset of 100k questions that we use in conjunction with existing benchmarks. We conduct our study within the framework of Memory Networks (Weston et al., 2015) because this perspective allows us to eventually scale up to more complex reasoning, and show that Memory Networks can be successfully trained to achieve excellent performance.

PDF Abstract

Datasets


Introduced in the Paper:

SimpleQuestions

Used in the Paper:

WebQuestions ReVerb Challenge

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Question Answering Reverb Memory Networks (ensemble) Accuracy 68% # 2
Question Answering SimpleQuestions Memory Networks (ensemble) F1 63.9% # 1
Question Answering WebQuestions Memory Networks (ensemble) F1 42.2% # 1

Methods


No methods listed for this paper. Add relevant methods here