TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
Atari Games	Atari 2600 Alien	Reactor 500M	Score	12689.1	# 11
Atari Games	Atari 2600 Amidar	Reactor 500M	Score	1015.8	# 25
Atari Games	Atari 2600 Assault	Reactor 500M	Score	8323.3	# 20
Atari Games	Atari 2600 Asterix	Reactor 500M	Score	205914.0	# 19
Atari Games	Atari 2600 Asteroids	Reactor 500M	Score	3726.1	# 18
Atari Games	Atari 2600 Atlantis	Reactor 500M	Score	302831.0	# 34
Atari Games	Atari 2600 Bank Heist	Reactor 500M	Score	1259.7	# 14
Atari Games	Atari 2600 Battle Zone	Reactor 500M	Score	64070.0	# 11
Atari Games	Atari 2600 Beam Rider	Reactor 500M	Score	11033.4	# 31
Atari Games	Atari 2600 Berzerk	Reactor 500M	Score	2303.1	# 15
Atari Games	Atari 2600 Bowling	Reactor 500M	Score	81.0	# 16
Atari Games	Atari 2600 Boxing	Reactor 500M	Score	99.4	# 16
Atari Games	Atari 2600 Breakout	Reactor 500M	Score	514.8	# 20
Atari Games	Atari 2600 Centipede	Reactor 500M	Score	3422.0	# 42
Atari Games	Atari 2600 Chopper Command	Reactor 500M	Score	107779.0	# 9
Atari Games	Atari 2600 Crazy Climber	Reactor 500M	Score	236422.0	# 6
Atari Games	Atari 2600 Defender	Reactor 500M	Score	223025.0	# 10
Atari Games	Atari 2600 Demon Attack	Reactor 500M	Score	115154.0	# 16
Atari Games	Atari 2600 Double Dunk	Reactor 500M	Score	23.0	# 10
Atari Games	Atari 2600 Enduro	Reactor 500M	Score	2224.2	# 13

Badge	Markdown
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-crazy-climber)](https://paperswithcode.com/sota/atari-games-on-atari-2600-crazy-climber?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-chopper-command)](https://paperswithcode.com/sota/atari-games-on-atari-2600-chopper-command?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-defender)](https://paperswithcode.com/sota/atari-games-on-atari-2600-defender?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-double-dunk)](https://paperswithcode.com/sota/atari-games-on-atari-2600-double-dunk?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-alien)](https://paperswithcode.com/sota/atari-games-on-atari-2600-alien?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-battle-zone)](https://paperswithcode.com/sota/atari-games-on-atari-2600-battle-zone?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-enduro)](https://paperswithcode.com/sota/atari-games-on-atari-2600-enduro?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-bank-heist)](https://paperswithcode.com/sota/atari-games-on-atari-2600-bank-heist?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-berzerk)](https://paperswithcode.com/sota/atari-games-on-atari-2600-berzerk?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-bowling)](https://paperswithcode.com/sota/atari-games-on-atari-2600-bowling?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-boxing)](https://paperswithcode.com/sota/atari-games-on-atari-2600-boxing?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-demon-attack)](https://paperswithcode.com/sota/atari-games-on-atari-2600-demon-attack?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-asteroids)](https://paperswithcode.com/sota/atari-games-on-atari-2600-asteroids?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-asterix)](https://paperswithcode.com/sota/atari-games-on-atari-2600-asterix?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-assault)](https://paperswithcode.com/sota/atari-games-on-atari-2600-assault?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-breakout)](https://paperswithcode.com/sota/atari-games-on-atari-2600-breakout?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-amidar)](https://paperswithcode.com/sota/atari-games-on-atari-2600-amidar?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-beam-rider)](https://paperswithcode.com/sota/atari-games-on-atari-2600-beam-rider?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-atlantis)](https://paperswithcode.com/sota/atari-games-on-atari-2600-atlantis?p=the-reactor-a-fast-and-sample-efficient-actor)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/the-reactor-a-fast-and-sample-efficient-actor/atari-games-on-atari-2600-centipede)](https://paperswithcode.com/sota/atari-games-on-atari-2600-centipede?p=the-reactor-a-fast-and-sample-efficient-actor)`

The Reactor: A fast and sample-efficient Actor-Critic agent for Reinforcement Learning

ICLR 2018 · Audrunas Gruslys, Will Dabney, Mohammad Gheshlaghi Azar, Bilal Piot, Marc Bellemare, Remi Munos ·

In this work we present a new agent architecture, called Reactor, which combines multiple algorithmic and architectural contributions to produce an agent with higher sample-efficiency than Prioritized Dueling DQN (Wang et al., 2016) and Categorical DQN (Bellemare et al., 2017), while giving better run-time performance than A3C (Mnih et al., 2016). Our first contribution is a new policy evaluation algorithm called Distributional Retrace, which brings multi-step off-policy updates to the distributional reinforcement learning setting. The same approach can be used to convert several classes of multi-step policy evaluation algorithms designed for expected value evaluation into distributional ones. Next, we introduce the \b{eta}-leave-one-out policy gradient algorithm which improves the trade-off between variance and bias by using action values as a baseline. Our final algorithmic contribution is a new prioritized replay algorithm for sequences, which exploits the temporal locality of neighboring observations for more efficient replay prioritization. Using the Atari 2600 benchmarks, we show that each of these innovations contribute to both the sample efficiency and final agent performance. Finally, we demonstrate that Reactor reaches state-of-the-art performance after 200 million frames and less than a day of training.

PDF Abstract ICLR 2018 PDF ICLR 2018 Abstract

Code

Add Remove Mark official

No code implementations yet. Submit your code now

Tasks

Add Remove

Atari Games

Distributional Reinforcement Learning

Reinforcement Learning (RL)

Datasets

Arcade Learning Environment

Results from the Paper

Edit

Ranked #6 on Atari Games on Atari 2600 Crazy Climber

Get a GitHub badge

Task	Dataset	Model	Metric Name	Metric Value	Global Rank	Benchmark
Atari Games	Atari 2600 Alien	Reactor 500M	Score	12689.1	# 11	Compare
Atari Games	Atari 2600 Amidar	Reactor 500M	Score	1015.8	# 25	Compare
Atari Games	Atari 2600 Assault	Reactor 500M	Score	8323.3	# 20	Compare
Atari Games	Atari 2600 Asterix	Reactor 500M	Score	205914.0	# 19	Compare
Atari Games	Atari 2600 Asteroids	Reactor 500M	Score	3726.1	# 18	Compare
Atari Games	Atari 2600 Atlantis	Reactor 500M	Score	302831.0	# 34	Compare
Atari Games	Atari 2600 Bank Heist	Reactor 500M	Score	1259.7	# 14	Compare
Atari Games	Atari 2600 Battle Zone	Reactor 500M	Score	64070.0	# 11	Compare
Atari Games	Atari 2600 Beam Rider	Reactor 500M	Score	11033.4	# 31	Compare
Atari Games	Atari 2600 Berzerk	Reactor 500M	Score	2303.1	# 15	Compare
Atari Games	Atari 2600 Bowling	Reactor 500M	Score	81.0	# 16	Compare
Atari Games	Atari 2600 Boxing	Reactor 500M	Score	99.4	# 16	Compare
Atari Games	Atari 2600 Breakout	Reactor 500M	Score	514.8	# 20	Compare
Atari Games	Atari 2600 Centipede	Reactor 500M	Score	3422.0	# 42	Compare
Atari Games	Atari 2600 Chopper Command	Reactor 500M	Score	107779.0	# 9	Compare
Atari Games	Atari 2600 Crazy Climber	Reactor 500M	Score	236422.0	# 6	Compare
Atari Games	Atari 2600 Defender	Reactor 500M	Score	223025.0	# 10	Compare
Atari Games	Atari 2600 Demon Attack	Reactor 500M	Score	115154.0	# 16	Compare
Atari Games	Atari 2600 Double Dunk	Reactor 500M	Score	23.0	# 10	Compare
Atari Games	Atari 2600 Enduro	Reactor 500M	Score	2224.2	# 13	Compare

Methods

Add Remove

A3C • Convolution • Dense Connections • DQN • Entropy Regularization • Q-Learning • Retrace • Softmax

Edit Social Preview

The Reactor: A fast and sample-efficient Actor-Critic agent for Reinforcement Learning

Code Edit Add Remove Mark official

Tasks Edit Add Remove

Datasets Edit

Results from the Paper Edit

Methods Edit Add Remove

Code

Add Remove Mark official

Tasks

Add Remove

Datasets

Results from the Paper

Edit

Methods

Add Remove