TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
Common Sense Reasoning	ARC (Challenge)	Mistral 7B (0-shot)	Accuracy	55.5	# 24
Common Sense Reasoning	ARC (Easy)	Mistral 7B (0-shot)	Accuracy	80.0	# 12
Arithmetic Reasoning	GSM8K	Mistral 7B (maj@8)	Accuracy	52.2	# 119
Arithmetic Reasoning	GSM8K	Mistral 7B (maj@8)	Parameters (Billion)	7	# 10
Sentence Completion	HellaSwag	Mistral 7B (0-shot)	Accuracy	81.3	# 37
Code Generation	HumanEval	Mistral 7B (0-shot)	Pass@1	30.5	# 78
Zero-Shot Video Question Answer	IntentQA	Mistral (7B)	Accuracy	50.4	# 6
Math Word Problem Solving	MATH	Mistral 7B (maj@4)	Accuracy	13.1	# 85
Math Word Problem Solving	MATH	Mistral 7B (maj@4)	Parameters (Billions)	7	# 58
Code Generation	MBPP	Mistral 7B (3-shot)	Accuracy	47.5	# 55
Multi-task Language Understanding	MMLU	Mistral 7B (5-shot)	Average (%)	60.1	# 50
Question Answering	Natural Questions	Mistral 7B (5-shot)	EM	28.8	# 29
Zero-Shot Video Question Answer	NExT-GQA	Mistral (7B)	Acc@GQA	9.2	# 4
Zero-Shot Video Question Answer	NExT-QA	Mistral (7B)	Accuracy	51.1	# 13
Question Answering	PIQA	Mistral 7B (0-shot)	Accuracy	83.0	# 11
Question Answering	TriviaQA	Mistral 7B (5-shot)	EM	69.9	# 24
Common Sense Reasoning	WinoGrande	Mistral 7B (0-shot)	Accuracy	75.3	# 22

Badge	Markdown
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mistral-7b/zero-shot-video-question-answer-on-next-gqa)](https://paperswithcode.com/sota/zero-shot-video-question-answer-on-next-gqa?p=mistral-7b)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mistral-7b/zero-shot-video-question-answer-on-intentqa)](https://paperswithcode.com/sota/zero-shot-video-question-answer-on-intentqa?p=mistral-7b)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mistral-7b/question-answering-on-piqa)](https://paperswithcode.com/sota/question-answering-on-piqa?p=mistral-7b)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mistral-7b/common-sense-reasoning-on-arc-easy)](https://paperswithcode.com/sota/common-sense-reasoning-on-arc-easy?p=mistral-7b)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mistral-7b/zero-shot-video-question-answer-on-next-qa)](https://paperswithcode.com/sota/zero-shot-video-question-answer-on-next-qa?p=mistral-7b)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mistral-7b/common-sense-reasoning-on-winogrande)](https://paperswithcode.com/sota/common-sense-reasoning-on-winogrande?p=mistral-7b)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mistral-7b/common-sense-reasoning-on-arc-challenge)](https://paperswithcode.com/sota/common-sense-reasoning-on-arc-challenge?p=mistral-7b)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mistral-7b/question-answering-on-triviaqa)](https://paperswithcode.com/sota/question-answering-on-triviaqa?p=mistral-7b)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mistral-7b/question-answering-on-natural-questions)](https://paperswithcode.com/sota/question-answering-on-natural-questions?p=mistral-7b)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mistral-7b/sentence-completion-on-hellaswag)](https://paperswithcode.com/sota/sentence-completion-on-hellaswag?p=mistral-7b)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mistral-7b/multi-task-language-understanding-on-mmlu)](https://paperswithcode.com/sota/multi-task-language-understanding-on-mmlu?p=mistral-7b)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mistral-7b/code-generation-on-mbpp)](https://paperswithcode.com/sota/code-generation-on-mbpp?p=mistral-7b)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mistral-7b/code-generation-on-humaneval)](https://paperswithcode.com/sota/code-generation-on-humaneval?p=mistral-7b)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mistral-7b/math-word-problem-solving-on-math)](https://paperswithcode.com/sota/math-word-problem-solving-on-math?p=mistral-7b)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mistral-7b/arithmetic-reasoning-on-gsm8k)](https://paperswithcode.com/sota/arithmetic-reasoning-on-gsm8k?p=mistral-7b)`

Mistral 7B

10 Oct 2023 · Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, William El Sayed ·

We introduce Mistral 7B v0.1, a 7-billion-parameter language model engineered for superior performance and efficiency. Mistral 7B outperforms Llama 2 13B across all evaluated benchmarks, and Llama 1 34B in reasoning, mathematics, and code generation. Our model leverages grouped-query attention (GQA) for faster inference, coupled with sliding window attention (SWA) to effectively handle sequences of arbitrary length with a reduced inference cost. We also provide a model fine-tuned to follow instructions, Mistral 7B -- Instruct, that surpasses the Llama 2 13B -- Chat model both on human and automated benchmarks. Our models are released under the Apache 2.0 license.

PDF Abstract

Code

Add Remove Mark official

mistralai/mistral-src official

↳ Quickstart in

Replicate

8,698

skypilot-org/skypilot official

5,656

facebookresearch/fairseq2

574

epfllm/megatron-llm

462

knowlab/bi-weekly-paper-presentation

Tasks

Add Remove

Arithmetic Reasoning

Chatbot

Code Generation

Common Sense Reasoning

Language Modelling

Math

Mathematical Reasoning

Math Word Problem Solving

Multi-task Language Understanding

Question Answering

Sentence Completion

World Knowledge

Zero-Shot Video Question Answer

Datasets

Natural Questions

MMLU

GSM8K

TriviaQA

HumanEval

HellaSwag

MATH

PIQA

WinoGrande MBPP

ARC (AI2 Reasoning Challenge)

NExT-QA IntentQA

NExT-GQA

Results from the Paper

Edit

Ranked #4 on Zero-Shot Video Question Answer on NExT-GQA

Get a GitHub badge

Task	Dataset	Model	Metric Name	Metric Value	Global Rank	Benchmark
Common Sense Reasoning	ARC (Challenge)	Mistral 7B (0-shot)	Accuracy	55.5	# 24	Compare
Common Sense Reasoning	ARC (Easy)	Mistral 7B (0-shot)	Accuracy	80.0	# 12	Compare
Arithmetic Reasoning	GSM8K	Mistral 7B (maj@8)	Accuracy	52.2	# 119	Compare
Arithmetic Reasoning	GSM8K	Mistral 7B (maj@8)	Parameters (Billion)	7	# 10	Compare
Sentence Completion	HellaSwag	Mistral 7B (0-shot)	Accuracy	81.3	# 37	Compare
Code Generation	HumanEval	Mistral 7B (0-shot)	Pass@1	30.5	# 78	Compare
Zero-Shot Video Question Answer	IntentQA	Mistral (7B)	Accuracy	50.4	# 6	Compare
Math Word Problem Solving	MATH	Mistral 7B (maj@4)	Accuracy	13.1	# 85	Compare
Math Word Problem Solving	MATH	Mistral 7B (maj@4)	Parameters (Billions)	7	# 58	Compare
Code Generation	MBPP	Mistral 7B (3-shot)	Accuracy	47.5	# 55	Compare
Multi-task Language Understanding	MMLU	Mistral 7B (5-shot)	Average (%)	60.1	# 50	Compare
Question Answering	Natural Questions	Mistral 7B (5-shot)	EM	28.8	# 29	Compare
Zero-Shot Video Question Answer	NExT-GQA	Mistral (7B)	Acc@GQA	9.2	# 4	Compare
Zero-Shot Video Question Answer	NExT-QA	Mistral (7B)	Accuracy	51.1	# 13	Compare
Question Answering	PIQA	Mistral 7B (0-shot)	Accuracy	83.0	# 11	Compare
Question Answering	TriviaQA	Mistral 7B (5-shot)	EM	69.9	# 24	Compare
Common Sense Reasoning	WinoGrande	Mistral 7B (0-shot)	Accuracy	75.3	# 22	Compare

Methods

Add Remove

Dense Connections • Feedforward Network • Grouped-query attention • LLaMA • Scaled Dot-Product Attention • Sliding Window Attention • Softmax

Edit Social Preview

Mistral 7B

Code Edit Add Remove Mark official

Tasks Edit Add Remove

Datasets Edit

Results from the Paper Edit

Methods Edit Add Remove

Code

Add Remove Mark official

Tasks

Add Remove

Datasets

Results from the Paper

Edit

Methods

Add Remove