FarsTail: A Persian Natural Language Inference Dataset

Natural language inference (NLI) is known as one of the central tasks in natural language processing (NLP) which encapsulates many fundamental aspects of language understanding. With the considerable achievements of data-hungry deep learning methods in NLP tasks, a great amount of effort has been devoted to develop more diverse datasets for different languages... (read more)

PDF Abstract

Datasets


Introduced in the Paper:

FarsTail

Mentioned in the Paper:

SNLI MIMIC-III MultiNLI XNLI
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Natural Language Inference FarsTail Translate-Source + fastText % Test Accuracy 78.13 # 1
Natural Language Inference FarsTail Decomposable Attention Model + word2vec % Test Accuracy 65.66 # 8
Natural Language Inference FarsTail ESIM + fastText % Test Accuracy 71.36 # 5
Natural Language Inference FarsTail HBMP + word2vec % Test Accuracy 66.04 # 7
Natural Language Inference FarsTail ULMFiT % Test Accuracy 72.44 # 4
Natural Language Inference FarsTail ESIM + BERT (FarsTail, MultiNLI) % Test Accuracy 74.62 # 3
Natural Language Inference FarsTail LSTM + BERT (concat) % Test Accuracy 75.83 # 2
Natural Language Inference FarsTail Translate-Target + fastText % Test Accuracy 70.46 # 6

Methods used in the Paper


METHOD TYPE
HBMP
Sequence To Sequence Models
ESIM
Sequence To Sequence Models
LSTM
Recurrent Neural Networks
BiGRU
Bidirectional Recurrent Neural Networks
Skip-gram Word2Vec
Word Embeddings
CBoW Word2Vec
Word Embeddings
fastText
Word Embeddings
ELMo
Word Embeddings
ULMFiT
Language Models
BERT
Language Models