Unsupervised Deep Structured Semantic Models for Commonsense Reasoning

NAACL 2019  ·  Shuohang Wang, Sheng Zhang, Yelong Shen, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, Jing Jiang ·

Commonsense reasoning is fundamental to natural language understanding. While traditional methods rely heavily on human-crafted features and knowledge bases, we explore learning commonsense knowledge from a large amount of raw text via unsupervised learning. We propose two neural network models based on the Deep Structured Semantic Models (DSSM) framework to tackle two classic commonsense reasoning tasks, Winograd Schema challenges (WSC) and Pronoun Disambiguation (PDP). Evaluation shows that the proposed models effectively capture contextual information in the sentence and co-reference information between pronouns and nouns, and achieve significant improvement over previous state-of-the-art approaches.

PDF Abstract NAACL 2019 PDF NAACL 2019 Abstract

Code
Add Remove Mark official

No code implementations yet. Submit your code now

Datasets

WSC
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Natural Language Understanding PDP60 UDSSM-II Accuracy 75 # 5
Compare
Natural Language Understanding PDP60 UDSSM-I (ensemble) Accuracy 76.7 # 4
Compare
Natural Language Understanding PDP60 DSSM Accuracy 75.0 # 5
Compare
Natural Language Understanding PDP60 UDSSM-II (ensemble) Accuracy 78.3 # 2
Compare
Coreference Resolution Winograd Schema Challenge UDSSM-II (ensemble) Accuracy 62.4 # 50
Compare
Coreference Resolution Winograd Schema Challenge UDSSM-I (ensemble) Accuracy 57.1 # 65
Compare
Coreference Resolution Winograd Schema Challenge UDSSM-II Accuracy 59.2 # 60
Compare
Coreference Resolution Winograd Schema Challenge UDSSM-I Accuracy 54.5 # 72
Compare

Results from Other Papers

Task Dataset Model Metric Name Metric Value Rank Source Paper Compare
Coreference Resolution Winograd Schema Challenge DSSM Accuracy 63.0 # 47
See all

Methods
Add Remove

No methods listed for this paper. Add relevant methods here
Contact us on: hello@paperswithcode.com . Papers With Code is a free resource with all data licensed under CC-BY-SA.
Terms Data policy Cookies policy