Distributional Reinforcement Learning with Quantile Regression

27 Oct 2017 • Will Dabney • Mark Rowland • Marc G. Bellemare • Rémi Munos

In reinforcement learning an agent interacts with the environment by taking actions and observing the next state and reward. When sampled probabilistically, these state transitions, rewards, and actions can all induce randomness in the observed long-term return... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Atari Games Atari 2600 Alien QR-DQN-1 Score 4871 # 12
Atari Games Atari 2600 Amidar QR-DQN-1 Score 1641 # 12
Atari Games Atari 2600 Assault QR-DQN-1 Score 22012 # 5
Atari Games Atari 2600 Asterix QR-DQN-1 Score 261025 # 11
Atari Games Atari 2600 Asteroids QR-DQN-1 Score 4226 # 11
Atari Games Atari 2600 Atlantis QR-DQN-1 Score 971850 # 10
Atari Games Atari 2600 Bank Heist QR-DQN-1 Score 1249 # 9
Atari Games Atari 2600 Battle Zone QR-DQN-1 Score 39268 # 9
Atari Games Atari 2600 Beam Rider QR-DQN-1 Score 34821 # 6
Atari Games Atari 2600 Berzerk QR-DQN-1 Score 3117 # 7
Atari Games Atari 2600 Bowling QR-DQN-1 Score 77.2 # 11
Atari Games Atari 2600 Boxing QR-DQN-1 Score 99.9 # 3
Atari Games Atari 2600 Breakout QR-DQN-1 Score 742 # 9
Atari Games Atari 2600 Centipede QR-DQN-1 Score 12447 # 8
Atari Games Atari 2600 Chopper Command QR-DQN-1 Score 14667 # 10
Atari Games Atari 2600 Crazy Climber QR-DQN-1 Score 161196 # 10
Atari Games Atari 2600 Defender QR-DQN-1 Score 47887 # 8
Atari Games Atari 2600 Demon Attack QR-DQN-1 Score 121551 # 8
Atari Games Atari 2600 Double Dunk QR-DQN-1 Score 21.9 # 6
Atari Games Atari 2600 Enduro QR-DQN-1 Score 2355 # 5
Atari Games Atari 2600 Fishing Derby QR-DQN-1 Score 39 # 12
Atari Games Atari 2600 Freeway QR-DQN-1 Score 34 # 1
Atari Games Atari 2600 Frostbite QR-DQN-1 Score 4384 # 9
Atari Games Atari 2600 Gopher QR-DQN-1 Score 113585 # 5
Atari Games Atari 2600 Gravitar QR-DQN-1 Score 995 # 13
Atari Games Atari 2600 HERO QR-DQN-1 Score 21395 # 17
Atari Games Atari 2600 Ice Hockey QR-DQN-1 Score -1.7 # 18
Atari Games Atari 2600 James Bond QR-DQN-1 Score 4703 # 8
Atari Games Atari 2600 Kangaroo QR-DQN-1 Score 15356 # 4
Atari Games Atari 2600 Krull QR-DQN-1 Score 11447 # 6
Atari Games Atari 2600 Kung-Fu Master QR-DQN-1 Score 76642 # 5
Atari Games Atari 2600 Montezuma's Revenge QR-DQN-1 Score 0 # 34
Atari Games Atari 2600 Ms. Pacman QR-DQN-1 Score 5821 # 10
Atari Games Atari 2600 Name This Game QR-DQN-1 Score 21890 # 5
Atari Games Atari 2600 Phoenix QR-DQN-1 Score 16585 # 9
Atari Games Atari 2600 Pitfall! QR-DQN-1 Score 0 # 4
Atari Games Atari 2600 Pong QR-DQN-1 Score 21 # 1
Atari Games Atari 2600 Private Eye QR-DQN-1 Score 350 # 19
Atari Games Atari 2600 Q*Bert QR-DQN-1 Score 572510 # 1
Atari Games Atari 2600 River Raid QR-DQN-1 Score 17571 # 9
Atari Games Atari 2600 Road Runner QR-DQN-1 Score 64262 # 7
Atari Games Atari 2600 Robotank QR-DQN-1 Score 59.4 # 16
Atari Games Atari 2600 Seaquest QR-DQN-1 Score 8268 # 18
Atari Games Atari 2600 Skiing QR-DQN-1 Score -9324 # 7
Atari Games Atari 2600 Solaris QR-DQN-1 Score 6740 # 5
Atari Games Atari 2600 Space Invaders QR-DQN-1 Score 20972 # 8
Atari Games Atari 2600 Star Gunner QR-DQN-1 Score 77495 # 12
Atari Games Atari 2600 Surround QR-DQN-1 Score 8.2 # 5
Atari Games Atari 2600 Tennis QR-DQN-1 Score 23.6 # 2
Atari Games Atari 2600 Time Pilot QR-DQN-1 Score 10345 # 13
Atari Games Atari 2600 Tutankham QR-DQN-1 Score 297 # 4
Atari Games Atari 2600 Up and Down QR-DQN-1 Score 71260 # 11
Atari Games Atari 2600 Venture QR-DQN-1 Score 43.9 # 28
Atari Games Atari 2600 Video Pinball QR-DQN-1 Score 705662 # 6
Atari Games Atari 2600 Wizard of Wor QR-DQN-1 Score 25061 # 7
Atari Games Atari 2600 Yars Revenge QR-DQN-1 Score 26447 # 9
Atari Games Atari 2600 Zaxxon QR-DQN-1 Score 13112 # 13
Atari Games Atari-57 QR-DQN-1 Medium Human-Normalized Score 211% # 7

Methods used in the Paper