Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets

WS 2019 Yifan PengShankai YanZhiyong Lu

TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Relation Extraction 2010 i2b2/VA NCBI_BERT(base) (P+M) F1 76.4 # 1
Medical Named Entity Recognition BC5CDR-chemical NCBI_BERT(base) (P) F1 93.5 # 1
Medical Named Entity Recognition BC5CDR-disease NCBI_BERT(base) (P) F1 86.6 # 1
Semantic Similarity BIOSSES NCBI_BERT(base) (P+M) Pearson Correlation 0.9159999999999999 # 1
Relation Extraction ChemProt NCBI_BERT(large) (P) F1 74.4 # 4
Medical Relation Extraction DDI extraction 2013 corpus NCBI_BERT(large) (P) F1 79.9 # 1
Document Classification HoC NCBI_BERT(large) (P) F1 87.3 # 1
Natural Language Inference MedNLI NCBI_BERT(base) (P+M) F1 84 # 1
Semantic Similarity MedSTS NCBI_BERT(base) (P+M) Pearson Correlation 0.848 # 1
Medical Named Entity Recognition ShARe/CLEF eHealth corpus NCBI_BERT(base) (P+M) F1 0.792 # 1

Methods used in the Paper


METHOD TYPE
🤖 No Methods Found Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet