ETHAN at SemEval-2020 Task 5: Modelling Causal Reasoning inLanguage using neuro-symbolic cloud computing

COLING 2020  ·  Len Yabloko ·

Experimental Testing of Hybrid AI Node implemented entirely on free cloud computing infrastructure. The ultimate goal of this research is to create modular reusable hybrid neuro-symbolic architecture for Artificial Intelligence. As a test case I model natural language comprehension of causal relations from open domain text corpus that combines semi-supervised language model (Huggingface Transformers) with constituency and dependency parsers (Allen Institute for Artificial Intelligence). The experimental results presented in this paper show potential for SOTA-level performance and demonstrate hybrid Artificial Intelligence in action. On SemEval-2020 Task 5 Subtask 1 ETHAN achieves F1 0.878 which is the second best result at the time of my writing. On Subtask 2 ETHAN currently scores 0.673 which is within the top ten results. All code used for experiments is available as open source and can be executed directly from the web browser without any installation or configuration. I briefly discuss some theoretical approaches that led to the proposed solutions and future directions of this research.

PDF

Datasets


  Add Datasets introduced or used in this paper
Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Counterfactual Detection SemEval-2020 Task5 subtask 2 ETHAN F1 69.3 # 1
Recall 76.6 # 1
Precision 67 # 1
Exact Match 66.7 # 1

Methods


No methods listed for this paper. Add relevant methods here