SemEval-2018 Task 11: Machine Comprehension Using Commonsense Knowledge
This report summarizes the results of the SemEval 2018 task on machine comprehension using commonsense knowledge. For this machine comprehension task, we created a new corpus, MCScript. It contains a high number of questions that require commonsense knowledge for finding the correct answer. 11 teams from 4 different countries participated in this shared task, most of them used neural approaches. The best performing system achieves an accuracy of 83.95{\%}, outperforming the baselines by a large margin, but still far from the human upper bound, which was found to be at 98{\%}.
PDF Abstract
SQuAD
ConceptNet
TriviaQA
RACE
NewsQA
bAbI
MCTest
MCScript
Results from the Paper
Submit
results from this paper
to get state-of-the-art GitHub badges and help the
community compare results to other papers.
Methods
No methods listed for this paper. Add
relevant methods here
