Implicit Quantile Networks for Distributional Reinforcement Learning

ICML 2018 • Will Dabney • Georg Ostrovski • David Silver • Rémi Munos

In this work, we build on recent advances in distributional reinforcement learning to give a generally applicable, flexible, and state-of-the-art distributional variant of DQN. We achieve this by using quantile regression to approximate the full quantile function for the state-action return distribution... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Atari Games Atari 2600 Alien IQN Score 7022 # 8
Atari Games Atari 2600 Amidar IQN Score 2946 # 6
Atari Games Atari 2600 Assault IQN Score 29091 # 3
Atari Games Atari 2600 Asterix IQN Score 342016 # 7
Atari Games Atari 2600 Asteroids IQN Score 2898 # 14
Atari Games Atari 2600 Atlantis IQN Score 978200 # 8
Atari Games Atari 2600 Bank Heist IQN Score 1416 # 5
Atari Games Atari 2600 Battle Zone IQN Score 42244 # 8
Atari Games Atari 2600 Beam Rider IQN Score 42776 # 4
Atari Games Atari 2600 Berzerk IQN Score 1053 # 20
Atari Games Atari 2600 Bowling IQN Score 86.5 # 7
Atari Games Atari 2600 Boxing IQN Score 99.8 # 4
Atari Games Atari 2600 Breakout IQN Score 734 # 10
Atari Games Atari 2600 Centipede IQN Score 11561 # 9
Atari Games Atari 2600 Chopper Command IQN Score 16836 # 8
Atari Games Atari 2600 Crazy Climber IQN Score 179082 # 7
Atari Games Atari 2600 Defender IQN Score 53537 # 7
Atari Games Atari 2600 Demon Attack IQN Score 128580 # 7
Atari Games Atari 2600 Double Dunk IQN Score 5.6 # 10
Atari Games Atari 2600 Enduro IQN Score 2359 # 4
Atari Games Atari 2600 Fishing Derby IQN Score 33.8 # 14
Atari Games Atari 2600 Freeway IQN Score 34 # 1
Atari Games Atari 2600 Frostbite IQN Score 4324 # 11
Atari Games Atari 2600 Gopher IQN Score 118365 # 4
Atari Games Atari 2600 Gravitar IQN Score 911 # 14
Atari Games Atari 2600 HERO IQN Score 28386 # 13
Atari Games Atari 2600 Ice Hockey IQN Score 0.2 # 11
Atari Games Atari 2600 James Bond IQN Score 35108 # 3
Atari Games Atari 2600 Kangaroo IQN Score 15487 # 3
Atari Games Atari 2600 Krull IQN Score 10707 # 8
Atari Games Atari 2600 Kung-Fu Master IQN Score 73512 # 6
Atari Games Atari 2600 Montezuma's Revenge IQN Score 0 # 34
Atari Games Atari 2600 Ms. Pacman IQN Score 6349 # 8
Atari Games Atari 2600 Name This Game IQN Score 22682 # 4
Atari Games Atari 2600 Phoenix IQN Score 56599 # 7
Atari Games Atari 2600 Pitfall! IQN Score 0 # 4
Atari Games Atari 2600 Pong IQN Score 21 # 1
Atari Games Atari 2600 Private Eye IQN Score 200 # 26
Atari Games Atari 2600 Q*Bert IQN Score 25750 # 8
Atari Games Atari 2600 River Raid IQN Score 17765 # 8
Atari Games Atari 2600 Road Runner IQN Score 57900 # 10
Atari Games Atari 2600 Robotank IQN Score 62.5 # 13
Atari Games Atari 2600 Seaquest IQN Score 30140 # 7
Atari Games Atari 2600 Skiing IQN Score -9289 # 6
Atari Games Atari 2600 Solaris IQN Score 8007 # 4
Atari Games Atari 2600 Space Invaders IQN Score 28888 # 6
Atari Games Atari 2600 Star Gunner IQN Score 74677 # 14
Atari Games Atari 2600 Surround IQN Score 9.4 # 4
Atari Games Atari 2600 Tennis IQN Score 23.6 # 2
Atari Games Atari 2600 Time Pilot IQN Score 12236 # 9
Atari Games Atari 2600 Tutankham IQN Score 293 # 5
Atari Games Atari 2600 Up and Down IQN Score 88148 # 8
Atari Games Atari 2600 Venture IQN Score 1318 # 6
Atari Games Atari 2600 Video Pinball IQN Score 698045 # 7
Atari Games Atari 2600 Wizard of Wor IQN Score 31190 # 6
Atari Games Atari 2600 Yars Revenge IQN Score 28379 # 8
Atari Games Atari 2600 Zaxxon IQN Score 21772 # 8

Methods used in the Paper