TASK |
DATASET |
MODEL |
METRIC NAME |
METRIC VALUE |
GLOBAL RANK |
REMOVE |
Natural Language Inference
|
MultiNLI Dev
|
BERT-Large-uncased-PruneOFA (90% unstruct sparse)
|
Matched
|
83.74
|
# 2
|
|
Natural Language Inference
|
MultiNLI Dev
|
BERT-Large-uncased-PruneOFA (90% unstruct sparse)
|
Mismatched
|
84.2
|
# 2
|
|
Natural Language Inference
|
MultiNLI Dev
|
BERT-Base-uncased-PruneOFA (85% unstruct sparse, QAT Int8)
|
Matched
|
81.4
|
# 6
|
|
Natural Language Inference
|
MultiNLI Dev
|
BERT-Base-uncased-PruneOFA (85% unstruct sparse, QAT Int8)
|
Mismatched
|
82.51
|
# 5
|
|
Natural Language Inference
|
MultiNLI Dev
|
BERT-Base-uncased-PruneOFA (85% unstruct sparse)
|
Matched
|
82.71
|
# 4
|
|
Natural Language Inference
|
MultiNLI Dev
|
BERT-Base-uncased-PruneOFA (85% unstruct sparse)
|
Mismatched
|
83.67
|
# 4
|
|
Natural Language Inference
|
MultiNLI Dev
|
DistilBERT-uncased-PruneOFA (90% unstruct sparse, QAT Int8)
|
Matched
|
78.8
|
# 10
|
|
Natural Language Inference
|
MultiNLI Dev
|
DistilBERT-uncased-PruneOFA (90% unstruct sparse, QAT Int8)
|
Mismatched
|
80.4
|
# 10
|
|
Natural Language Inference
|
MultiNLI Dev
|
DistilBERT-uncased-PruneOFA (90% unstruct sparse)
|
Matched
|
80.68
|
# 8
|
|
Natural Language Inference
|
MultiNLI Dev
|
DistilBERT-uncased-PruneOFA (90% unstruct sparse)
|
Mismatched
|
81.47
|
# 8
|
|
Natural Language Inference
|
MultiNLI Dev
|
DistilBERT-uncased-PruneOFA (85% unstruct sparse, QAT Int8)
|
Matched
|
80.66
|
# 9
|
|
Natural Language Inference
|
MultiNLI Dev
|
DistilBERT-uncased-PruneOFA (85% unstruct sparse, QAT Int8)
|
Mismatched
|
81.14
|
# 9
|
|
Natural Language Inference
|
MultiNLI Dev
|
DistilBERT-uncased-PruneOFA (85% unstruct sparse)
|
Matched
|
81.35
|
# 7
|
|
Natural Language Inference
|
MultiNLI Dev
|
DistilBERT-uncased-PruneOFA (85% unstruct sparse)
|
Mismatched
|
82.03
|
# 7
|
|
Natural Language Inference
|
MultiNLI Dev
|
BERT-Large-uncased-PruneOFA (90% unstruct sparse, QAT Int8)
|
Matched
|
83.47
|
# 3
|
|
Natural Language Inference
|
MultiNLI Dev
|
BERT-Large-uncased-PruneOFA (90% unstruct sparse, QAT Int8)
|
Mismatched
|
84.08
|
# 3
|
|
Natural Language Inference
|
MultiNLI Dev
|
BERT-Base-uncased-PruneOFA (90% unstruct sparse)
|
Matched
|
81.45
|
# 5
|
|
Natural Language Inference
|
MultiNLI Dev
|
BERT-Base-uncased-PruneOFA (90% unstruct sparse)
|
Mismatched
|
82.43
|
# 6
|
|
Question Answering
|
SQuAD1.1 dev
|
BERT-Base-uncased-PruneOFA (85% unstruct sparse)
|
EM
|
81.1
|
# 12
|
|
Question Answering
|
SQuAD1.1 dev
|
BERT-Base-uncased-PruneOFA (85% unstruct sparse)
|
F1
|
88.42
|
# 14
|
|
Question Answering
|
SQuAD1.1 dev
|
BERT-Large-uncased-PruneOFA (90% unstruct sparse)
|
EM
|
83.35
|
# 10
|
|
Question Answering
|
SQuAD1.1 dev
|
BERT-Large-uncased-PruneOFA (90% unstruct sparse)
|
F1
|
90.2
|
# 12
|
|
Question Answering
|
SQuAD1.1 dev
|
DistilBERT-uncased-PruneOFA (90% unstruct sparse, QAT Int8)
|
EM
|
75.62
|
# 25
|
|
Question Answering
|
SQuAD1.1 dev
|
DistilBERT-uncased-PruneOFA (90% unstruct sparse, QAT Int8)
|
F1
|
83.87
|
# 28
|
|
Question Answering
|
SQuAD1.1 dev
|
DistilBERT-uncased-PruneOFA (90% unstruct sparse)
|
EM
|
76.91
|
# 22
|
|
Question Answering
|
SQuAD1.1 dev
|
DistilBERT-uncased-PruneOFA (90% unstruct sparse)
|
F1
|
84.82
|
# 26
|
|
Question Answering
|
SQuAD1.1 dev
|
DistilBERT-uncased-PruneOFA (85% unstruct sparse, QAT Int8)
|
EM
|
77.03
|
# 21
|
|
Question Answering
|
SQuAD1.1 dev
|
DistilBERT-uncased-PruneOFA (85% unstruct sparse, QAT Int8)
|
F1
|
85.13
|
# 24
|
|
Question Answering
|
SQuAD1.1 dev
|
DistilBERT-uncased-PruneOFA (85% unstruct sparse)
|
EM
|
78.1
|
# 19
|
|
Question Answering
|
SQuAD1.1 dev
|
DistilBERT-uncased-PruneOFA (85% unstruct sparse)
|
F1
|
85.82
|
# 21
|
|
Question Answering
|
SQuAD1.1 dev
|
BERT-Large-uncased-PruneOFA (90% unstruct sparse, QAT Int8)
|
EM
|
83.22
|
# 11
|
|
Question Answering
|
SQuAD1.1 dev
|
BERT-Large-uncased-PruneOFA (90% unstruct sparse, QAT Int8)
|
F1
|
90.02
|
# 13
|
|
Question Answering
|
SQuAD1.1 dev
|
BERT-Base-uncased-PruneOFA (90% unstruct sparse)
|
EM
|
79.83
|
# 14
|
|
Question Answering
|
SQuAD1.1 dev
|
BERT-Base-uncased-PruneOFA (90% unstruct sparse)
|
F1
|
87.25
|
# 17
|
|
Question Answering
|
SQuAD1.1 dev
|
BERT-Base-uncased-PruneOFA (85% unstruct sparse, QAT Int8)
|
EM
|
80.84
|
# 13
|
|
Question Answering
|
SQuAD1.1 dev
|
BERT-Base-uncased-PruneOFA (85% unstruct sparse, QAT Int8)
|
F1
|
88.24
|
# 15
|
|