TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
Unsupervised Reinforcement Learning	URLB (pixels, 10^5 frames)	DDPG (DrQ-v2)	Walker (mean normalized return)	14.18±8.68	# 6
Unsupervised Reinforcement Learning	URLB (pixels, 10^5 frames)	DDPG (DrQ-v2)	Quadruped (mean normalized return)	25.07±7.80	# 2
Unsupervised Reinforcement Learning	URLB (pixels, 10^5 frames)	DDPG (DrQ-v2)	Jaco (mean normalized return)	15.33±4.29	# 3
Unsupervised Reinforcement Learning	URLB (pixels, 10^6 frames)	DDPG (DrQ-v2)	Walker (mean normalized return)	14.18±8.68	# 6
Unsupervised Reinforcement Learning	URLB (pixels, 10^6 frames)	DDPG (DrQ-v2)	Quadruped (mean normalized return)	25.07±7.80	# 4
Unsupervised Reinforcement Learning	URLB (pixels, 10^6 frames)	DDPG (DrQ-v2)	Jaco (mean normalized return)	15.33±4.29	# 5
Unsupervised Reinforcement Learning	URLB (pixels, 2*10^6 frames)	DDPG (DrQ-v2)	Walker (mean normalized return)	14.18±8.68	# 7
Unsupervised Reinforcement Learning	URLB (pixels, 2*10^6 frames)	DDPG (DrQ-v2)	Quadruped (mean normalized return)	25.07±7.80	# 5
Unsupervised Reinforcement Learning	URLB (pixels, 2*10^6 frames)	DDPG (DrQ-v2)	Jaco (mean normalized return)	15.33±4.29	# 6
Unsupervised Reinforcement Learning	URLB (pixels, 5*10^5 frames)	DDPG (DrQ-v2)	Walker (mean normalized return)	14.18±8.68	# 6
Unsupervised Reinforcement Learning	URLB (pixels, 5*10^5 frames)	DDPG (DrQ-v2)	Quadruped (mean normalized return)	25.07±7.80	# 4
Unsupervised Reinforcement Learning	URLB (pixels, 5*10^5 frames)	DDPG (DrQ-v2)	Jaco (mean normalized return)	15.33±4.29	# 5
Unsupervised Reinforcement Learning	URLB (states, 10^5 frames)	DDPG (DrQ-v2)	Walker (mean normalized return)	73.68±31.29	# 7
Unsupervised Reinforcement Learning	URLB (states, 10^5 frames)	DDPG (DrQ-v2)	Quadruped (mean normalized return)	28.33±9.01	# 7
Unsupervised Reinforcement Learning	URLB (states, 10^5 frames)	DDPG (DrQ-v2)	Jaco (mean normalized return)	49.14±8.22	# 6
Unsupervised Reinforcement Learning	URLB (states, 10^6 frames)	DDPG (DrQ-v2)	Walker (mean normalized return)	73.68±31.29	# 7
Unsupervised Reinforcement Learning	URLB (states, 10^6 frames)	DDPG (DrQ-v2)	Quadruped (mean normalized return)	28.33±9.01	# 9
Unsupervised Reinforcement Learning	URLB (states, 10^6 frames)	DDPG (DrQ-v2)	Jaco (mean normalized return)	49.14±8.22	# 7
Unsupervised Reinforcement Learning	URLB (states, 2*10^6 frames)	DDPG (DrQ-v2)	Walker (mean normalized return)	73.68±31.29	# 5
Unsupervised Reinforcement Learning	URLB (states, 2*10^6 frames)	DDPG (DrQ-v2)	Quadruped (mean normalized return)	22.63±8.29	# 9
Unsupervised Reinforcement Learning	URLB (states, 2*10^6 frames)	DDPG (DrQ-v2)	Jaco (mean normalized return)	49.14±8.22	# 6
Unsupervised Reinforcement Learning	URLB (states, 5*10^5 frames)	DDPG (DrQ-v2)	Walker (mean normalized return)	73.68±31.29	# 7
Unsupervised Reinforcement Learning	URLB (states, 5*10^5 frames)	DDPG (DrQ-v2)	Quadruped (mean normalized return)	28.33±9.01	# 9
Unsupervised Reinforcement Learning	URLB (states, 5*10^5 frames)	DDPG (DrQ-v2)	Jaco (mean normalized return)	49.14±8.22	# 7

Badge	Markdown
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mastering-visual-continuous-control-improved/unsupervised-reinforcement-learning-on-urlb-3)](https://paperswithcode.com/sota/unsupervised-reinforcement-learning-on-urlb-3?p=mastering-visual-continuous-control-improved)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mastering-visual-continuous-control-improved/unsupervised-reinforcement-learning-on-urlb-4)](https://paperswithcode.com/sota/unsupervised-reinforcement-learning-on-urlb-4?p=mastering-visual-continuous-control-improved)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mastering-visual-continuous-control-improved/unsupervised-reinforcement-learning-on-urlb-6)](https://paperswithcode.com/sota/unsupervised-reinforcement-learning-on-urlb-6?p=mastering-visual-continuous-control-improved)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mastering-visual-continuous-control-improved/unsupervised-reinforcement-learning-on-urlb-5)](https://paperswithcode.com/sota/unsupervised-reinforcement-learning-on-urlb-5?p=mastering-visual-continuous-control-improved)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mastering-visual-continuous-control-improved/unsupervised-reinforcement-learning-on-urlb-7)](https://paperswithcode.com/sota/unsupervised-reinforcement-learning-on-urlb-7?p=mastering-visual-continuous-control-improved)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mastering-visual-continuous-control-improved/unsupervised-reinforcement-learning-on-urlb)](https://paperswithcode.com/sota/unsupervised-reinforcement-learning-on-urlb?p=mastering-visual-continuous-control-improved)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mastering-visual-continuous-control-improved/unsupervised-reinforcement-learning-on-urlb-2)](https://paperswithcode.com/sota/unsupervised-reinforcement-learning-on-urlb-2?p=mastering-visual-continuous-control-improved)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/mastering-visual-continuous-control-improved/unsupervised-reinforcement-learning-on-urlb-1)](https://paperswithcode.com/sota/unsupervised-reinforcement-learning-on-urlb-1?p=mastering-visual-continuous-control-improved)`

Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning

ICLR 2022 · Denis Yarats, Rob Fergus, Alessandro Lazaric, Lerrel Pinto ·

We present DrQ-v2, a model-free reinforcement learning (RL) algorithm for visual continuous control. DrQ-v2 builds on DrQ, an off-policy actor-critic approach that uses data augmentation to learn directly from pixels. We introduce several improvements that yield state-of-the-art results on the DeepMind Control Suite. Notably, DrQ-v2 is able to solve complex humanoid locomotion tasks directly from pixel observations, previously unattained by model-free RL. DrQ-v2 is conceptually simple, easy to implement, and provides significantly better computational footprint compared to prior work, with the majority of tasks taking just 8 hours to train on a single GPU. Finally, we publicly release DrQ-v2's implementation to provide RL practitioners with a strong and computationally efficient baseline.

PDF Abstract ICLR 2022 PDF ICLR 2022 Abstract

Code

Add Remove Mark official

facebookresearch/drqv2 official

326

denisyarats/drq

398

mazpie/mastering-urlb

Asap7772/understanding-rlhf

zhaoyi11/tcrl

See all 8 implementations

Tasks

Add Remove

Continuous Control

Data Augmentation

reinforcement-learning

Reinforcement Learning (RL)

Unsupervised Reinforcement Learning

Datasets

DeepMind Control Suite URLB

Results from the Paper

Edit

Ranked #5 on Unsupervised Reinforcement Learning on URLB (states, 2*10^6 frames)

Get a GitHub badge

Task	Dataset	Model	Metric Name	Metric Value	Global Rank	Benchmark
Unsupervised Reinforcement Learning	URLB (pixels, 10^5 frames)	DDPG (DrQ-v2)	Walker (mean normalized return)	14.18±8.68	# 6	Compare
			Quadruped (mean normalized return)	25.07±7.80	# 2	Compare
			Jaco (mean normalized return)	15.33±4.29	# 3	Compare
Unsupervised Reinforcement Learning	URLB (pixels, 10^6 frames)	DDPG (DrQ-v2)	Walker (mean normalized return)	14.18±8.68	# 6	Compare
			Quadruped (mean normalized return)	25.07±7.80	# 4	Compare
			Jaco (mean normalized return)	15.33±4.29	# 5	Compare
Unsupervised Reinforcement Learning	URLB (pixels, 2*10^6 frames)	DDPG (DrQ-v2)	Walker (mean normalized return)	14.18±8.68	# 7	Compare
			Quadruped (mean normalized return)	25.07±7.80	# 5	Compare
			Jaco (mean normalized return)	15.33±4.29	# 6	Compare
Unsupervised Reinforcement Learning	URLB (pixels, 5*10^5 frames)	DDPG (DrQ-v2)	Walker (mean normalized return)	14.18±8.68	# 6	Compare
			Quadruped (mean normalized return)	25.07±7.80	# 4	Compare
			Jaco (mean normalized return)	15.33±4.29	# 5	Compare
Unsupervised Reinforcement Learning	URLB (states, 10^5 frames)	DDPG (DrQ-v2)	Walker (mean normalized return)	73.68±31.29	# 7	Compare
			Quadruped (mean normalized return)	28.33±9.01	# 7	Compare
			Jaco (mean normalized return)	49.14±8.22	# 6	Compare
Unsupervised Reinforcement Learning	URLB (states, 10^6 frames)	DDPG (DrQ-v2)	Walker (mean normalized return)	73.68±31.29	# 7	Compare
			Quadruped (mean normalized return)	28.33±9.01	# 9	Compare
			Jaco (mean normalized return)	49.14±8.22	# 7	Compare
Unsupervised Reinforcement Learning	URLB (states, 2*10^6 frames)	DDPG (DrQ-v2)	Walker (mean normalized return)	73.68±31.29	# 5	Compare
			Quadruped (mean normalized return)	22.63±8.29	# 9	Compare
			Jaco (mean normalized return)	49.14±8.22	# 6	Compare
Unsupervised Reinforcement Learning	URLB (states, 5*10^5 frames)	DDPG (DrQ-v2)	Walker (mean normalized return)	73.68±31.29	# 7	Compare
			Quadruped (mean normalized return)	28.33±9.01	# 9	Compare
			Jaco (mean normalized return)	49.14±8.22	# 7	Compare

Methods

Add Remove

No methods listed for this paper. Add relevant methods here

Edit Social Preview

Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning

Code Edit Add Remove Mark official

Tasks Edit Add Remove

Datasets Edit

Results from the Paper Edit

Methods Edit Add Remove

Code

Add Remove Mark official

Tasks

Add Remove

Datasets

Results from the Paper

Edit

Methods

Add Remove