TASK
DATASET
MODEL
METRIC NAME
METRIC VALUE
GLOBAL RANK
EXTRA DATA
REMOVE
Arithmetic Reasoning
GSM8K
Minerva 62B (8-shot)
Accuracy
52.4
# 118
Arithmetic Reasoning
GSM8K
Minerva 62B (8-shot)
Parameters (Billion)
62
# 79
Arithmetic Reasoning
GSM8K
Minerva 540B (CoT)
Accuracy
78.5
# 68
Arithmetic Reasoning
GSM8K
Minerva 540B (CoT)
Parameters (Billion)
540
# 111
Arithmetic Reasoning
GSM8K
PaLM 540B (8-shot)
Accuracy
56.5
# 112
Arithmetic Reasoning
GSM8K
PaLM 540B (8-shot)
Parameters (Billion)
540
# 111
Arithmetic Reasoning
GSM8K
Minerva 8B (maj5@100)
Accuracy
56.8
# 110
Arithmetic Reasoning
GSM8K
Minerva 8B (maj5@100)
Parameters (Billion)
8
# 48
Arithmetic Reasoning
GSM8K
Minerva 62B (maj5@100)
Accuracy
89
# 23
Arithmetic Reasoning
GSM8K
Minerva 62B (maj5@100)
Parameters (Billion)
62
# 79
Arithmetic Reasoning
GSM8K
Minerva 62B (maj1@100)
Accuracy
68.5
# 95
Arithmetic Reasoning
GSM8K
Minerva 62B (maj1@100)
Parameters (Billion)
62
# 79
Arithmetic Reasoning
GSM8K
PaLM 62B (8-shot)
Accuracy
33.0
# 133
Arithmetic Reasoning
GSM8K
PaLM 62B (8-shot)
Parameters (Billion)
62
# 79
Arithmetic Reasoning
GSM8K
Minerva 8B (8-shot)
Accuracy
16.2
# 145
Arithmetic Reasoning
GSM8K
Minerva 8B (8-shot)
Parameters (Billion)
8
# 48
Arithmetic Reasoning
GSM8K
Minerva 8B-maj1@k (8-shot)
Accuracy
28.4
# 136
Arithmetic Reasoning
GSM8K
Minerva 8B-maj1@k (8-shot)
Parameters (Billion)
8
# 48
Arithmetic Reasoning
GSM8K
PaLM 8B (8-shot)
Accuracy
4.1
# 151
Arithmetic Reasoning
GSM8K
PaLM 8B (8-shot)
Parameters (Billion)
8
# 48
Math Word Problem Solving
MATH
davinci-002 175B
Accuracy
19.1
# 79
Math Word Problem Solving
MATH
davinci-002 175B
Parameters (Billions)
175
# 5
Math Word Problem Solving
MATH
Minerva 540B (maj1@k, k=64)
Accuracy
50.3
# 26
Math Word Problem Solving
MATH
Minerva 8B (maj1@k, k=64)
Accuracy
25.4
# 71
Math Word Problem Solving
MATH
Minerva 8B (maj1@k, k=64)
Parameters (Billions)
8
# 53
Math Word Problem Solving
MATH
Minerva 540B
Accuracy
33.6
# 56
Math Word Problem Solving
MATH
Minerva 540B
Parameters (Billions)
540
# 1
Math Word Problem Solving
MATH
Minerva 62B (maj1@k, k=64)
Accuracy
43.4
# 47
Math Word Problem Solving
MATH
Minerva 62B (maj1@k, k=64)
Parameters (Billions)
62
# 22
Math Word Problem Solving
MATH
Minerva 62B (maj5@256)
Accuracy
64.9
# 5
Math Word Problem Solving
MATH
Minerva 62B (maj5@256)
Parameters (Billions)
62
# 22
Math Word Problem Solving
MATH
Minerva 8B (maj5@256)
Accuracy
47.6
# 35
Math Word Problem Solving
MATH
Minerva 8B (maj5@256)
Parameters (Billions)
8
# 53
Math Word Problem Solving
MATH
PaLM 540B
Accuracy
8.8
# 91
Math Word Problem Solving
MATH
PaLM 540B
Parameters (Billions)
540
# 1
Math Word Problem Solving
MATH
PaLM 8B (fine-tuned)
Accuracy
5.6
# 99
Math Word Problem Solving
MATH
PaLM 8B (fine-tuned)
Parameters (Billions)
8
# 53
Math Word Problem Solving
MATH
Minerva 62B (4-shot)
Accuracy
27.6
# 68
Math Word Problem Solving
MATH
Minerva 62B (4-shot)
Parameters (Billions)
62
# 22
Math Word Problem Solving
MATH
PaLM 8B
Accuracy
1.5
# 109
Math Word Problem Solving
MATH
PaLM 8B
Parameters (Billions)
8
# 53
Math Word Problem Solving
MATH
PaLM 62B
Accuracy
4.4
# 104
Math Word Problem Solving
MATH
PaLM 62B
Parameters (Billions)
62
# 22
Math Word Problem Solving
MATH
Minerva 8B
Accuracy
14.1
# 83
Math Word Problem Solving
MATH
Minerva 8B
Parameters (Billions)
8
# 53