Noisy Networks for Exploration

ICLR 2018 Meire FortunatoMohammad Gheshlaghi AzarBilal PiotJacob MenickIan OsbandAlex GravesVlad MnihRemi MunosDemis HassabisOlivier PietquinCharles BlundellShane Legg

We introduce NoisyNet, a deep reinforcement learning agent with parametric noise added to its weights, and show that the induced stochasticity of the agent's policy can be used to aid efficient exploration. The parameters of the noise are learned with gradient descent along with the remaining network weights... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Atari Games Atari 2600 Alien NoisyNet-Dueling Score 5778 # 9
Atari Games Atari 2600 Amidar NoisyNet-Dueling Score 3537 # 4
Atari Games Atari 2600 Assault NoisyNet-Dueling Score 11231 # 9
Atari Games Atari 2600 Asterix NoisyNet-Dueling Score 28350 # 14
Atari Games Atari 2600 Asteroids NoisyNet-Dueling Score 86700 # 5
Atari Games Atari 2600 Atlantis NoisyNet-Dueling Score 972175 # 9
Atari Games Atari 2600 Bank Heist NoisyNet-Dueling Score 1318 # 6
Atari Games Atari 2600 Battle Zone NoisyNet-Dueling Score 52262 # 7
Atari Games Atari 2600 Beam Rider NoisyNet-Dueling Score 23134 # 13
Atari Games Atari 2600 Berzerk NoisyNet-Dueling Score 1896 # 10
Atari Games Atari 2600 Boxing NoisyNet-Dueling Score 100 # 1
Atari Games Atari 2600 Breakout NoisyNet-Dueling Score 263 # 34
Atari Games Atari 2600 Centipede NoisyNet-Dueling Score 7596 # 16
Atari Games Atari 2600 Chopper Command NoisyNet-Dueling Score 11477 # 12
Atari Games Atari 2600 Crazy Climber NoisyNet-Dueling Score 171171 # 8
Atari Games Atari 2600 Defender NoisyNet-Dueling Score 42253 # 9
Atari Games Atari 2600 Demon Attack NoisyNet-Dueling Score 69311 # 19
Atari Games Atari 2600 Double Dunk NoisyNet-Dueling Score 1 # 14
Atari Games Atari 2600 Enduro NoisyNet-Dueling Score 2013 # 13
Atari Games Atari 2600 Fishing Derby NoisyNet-Dueling Score 57 # 3
Atari Games Atari 2600 Freeway NoisyNet-Dueling Score 34 # 1
Atari Games Atari 2600 Frostbite NoisyNet-Dueling Score 2923 # 18
Atari Games Atari 2600 Gopher NoisyNet-Dueling Score 38909 # 10
Atari Games Atari 2600 Gravitar NoisyNet-Dueling Score 2209 # 7
Atari Games Atari 2600 HERO NoisyNet-Dueling Score 31533 # 9
Atari Games Atari 2600 Ice Hockey NoisyNet-Dueling Score 3 # 8
Atari Games Atari 2600 Kangaroo NoisyNet-Dueling Score 15227 # 5
Atari Games Atari 2600 Krull NoisyNet-Dueling Score 10754 # 7
Atari Games Atari 2600 Kung-Fu Master NoisyNet-Dueling Score 41672 # 12
Atari Games Atari 2600 Montezuma's Revenge NoisyNet-Dueling Score 57 # 23
Atari Games Atari 2600 Ms. Pacman NoisyNet-Dueling Score 5546 # 11
Atari Games Atari 2600 Name This Game NoisyNet-Dueling Score 12211 # 14
Atari Games Atari 2600 Phoenix NoisyNet-Dueling Score 10379 # 11
Atari Games Atari 2600 Pitfall! NoisyNet-Dueling Score 0 # 4
Atari Games Atari 2600 Pong NoisyNet-Dueling Score 21 # 1
Atari Games Atari 2600 Private Eye NoisyNet-Dueling Score 279 # 22
Atari Games Atari 2600 Q*Bert NoisyNet-Dueling Score 27121 # 7
Atari Games Atari 2600 Road Runner NoisyNet-Dueling Score 234352 # 3
Atari Games Atari 2600 Robotank NoisyNet-Dueling Score 64 # 10
Atari Games Atari 2600 Seaquest NoisyNet-Dueling Score 16754 # 10
Atari Games Atari 2600 Skiing NoisyNet-Dueling Score -7550 # 3
Atari Games Atari 2600 Solaris NoisyNet-Dueling Score 6522 # 6
Atari Games Atari 2600 Space Invaders NoisyNet-Dueling Score 5909 # 14
Atari Games Atari 2600 Star Gunner NoisyNet-Dueling Score 75867 # 13
Atari Games Atari 2600 Surround NoisyNet-Dueling Score 10 # 1
Atari Games Atari 2600 Tennis NoisyNet-Dueling Score 0 # 11
Atari Games Atari 2600 Time Pilot NoisyNet-Dueling Score 17301 # 7
Atari Games Atari 2600 Tutankham NoisyNet-Dueling Score 269 # 9
Atari Games Atari 2600 Up and Down NoisyNet-Dueling Score 61326 # 13
Atari Games Atari 2600 Venture NoisyNet-Dueling Score 815 # 10
Atari Games Atari 2600 Video Pinball NoisyNet-Dueling Score 870954 # 4
Atari Games Atari 2600 Wizard of Wor NoisyNet-Dueling Score 9149 # 16
Atari Games Atari 2600 Yars Revenge NoisyNet-Dueling Score 86101 # 4
Atari Games Atari 2600 Zaxxon NoisyNet-Dueling Score 14874 # 10

Methods used in the Paper