TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
Few-Shot Learning	CR	DART	Acc	91.8(0.5)	# 1
Few-Shot Learning	GLUE QQP	DART	F1-score	67.8(3.2)	# 1
Few-Shot Learning	MR	DART	Acc	88.2(1.0)	# 1
Few-Shot Learning	MRPC	DART	F1-score	78.3(4.5)	# 1
Few-Shot Learning	SST-2 Binary classification	DART	Acc	93.5(0.5)	# 1

Badge	Markdown
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/differentiable-prompt-makes-pre-trained/few-shot-learning-on-cr)](https://paperswithcode.com/sota/few-shot-learning-on-cr?p=differentiable-prompt-makes-pre-trained)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/differentiable-prompt-makes-pre-trained/few-shot-learning-on-glue-qqp)](https://paperswithcode.com/sota/few-shot-learning-on-glue-qqp?p=differentiable-prompt-makes-pre-trained)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/differentiable-prompt-makes-pre-trained/few-shot-learning-on-mr)](https://paperswithcode.com/sota/few-shot-learning-on-mr?p=differentiable-prompt-makes-pre-trained)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/differentiable-prompt-makes-pre-trained/few-shot-learning-on-mrpc)](https://paperswithcode.com/sota/few-shot-learning-on-mrpc?p=differentiable-prompt-makes-pre-trained)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/differentiable-prompt-makes-pre-trained/few-shot-learning-on-sst-2-binary)](https://paperswithcode.com/sota/few-shot-learning-on-sst-2-binary?p=differentiable-prompt-makes-pre-trained)`

Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners

ICLR 2022 · Ningyu Zhang, Luoqiu Li, Xiang Chen, Shumin Deng, Zhen Bi, Chuanqi Tan, Fei Huang, Huajun Chen ·

Large-scale pre-trained language models have contributed significantly to natural language processing by demonstrating remarkable abilities as few-shot learners. However, their effectiveness depends mainly on scaling the model parameters and prompt design, hindering their implementation in most real-world applications. This study proposes a novel pluggable, extensible, and efficient approach named DifferentiAble pRompT (DART), which can convert small language models into better few-shot learners without any prompt engineering. The main principle behind this approach involves reformulating potential natural language processing tasks into the task of a pre-trained language model and differentially optimizing the prompt template as well as the target label with backpropagation. Furthermore, the proposed approach can be: (i) Plugged to any pre-trained language models; (ii) Extended to widespread classification tasks. A comprehensive evaluation of standard NLP tasks demonstrates that the proposed approach achieves a better few-shot performance. Code is available in https://github.com/zjunlp/DART.

PDF Abstract ICLR 2022 PDF ICLR 2022 Abstract

Code

Add Remove Mark official

zjunlp/DART official

126

zhengxiangshi/powerfulpromptft

paperspapers/badprompt

zhaohan-xi/plm-prompt-defense

Tasks

Add Remove

Language Modelling

Prompt Engineering

Datasets

GLUE

SST SST-2

MRPC

Results from the Paper

Edit

Ranked #1 on Few-Shot Learning on CR

Get a GitHub badge

Task	Dataset	Model	Metric Name	Metric Value	Global Rank	Benchmark
Few-Shot Learning	CR	DART	Acc	91.8(0.5)	# 1	Compare
Few-Shot Learning	GLUE QQP	DART	F1-score	67.8(3.2)	# 1	Compare
Few-Shot Learning	MR	DART	Acc	88.2(1.0)	# 1	Compare
Few-Shot Learning	MRPC	DART	F1-score	78.3(4.5)	# 1	Compare
Few-Shot Learning	SST-2 Binary classification	DART	Acc	93.5(0.5)	# 1	Compare

Methods

Add Remove

No methods listed for this paper. Add relevant methods here

Edit Social Preview

Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners

Code Edit Add Remove Mark official

Tasks Edit Add Remove

Datasets Edit

Results from the Paper Edit

Methods Edit Add Remove

Code

Add Remove Mark official

Tasks

Add Remove

Datasets

Results from the Paper

Edit

Methods

Add Remove