Rainbow: Combining Improvements in Deep Reinforcement Learning

The deep reinforcement learning community has made several independent improvements to the DQN algorithm. However, it is unclear which of these extensions are complementary and can be fruitfully combined... (read more)

PDF Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Montezuma's Revenge Atari 2600 Montezuma's Revenge Rainbow Average Return (NoOp) 384 # 3
Atari Games Atari 2600 Ms. Pacman Rainbow Score 2,570.2 # 41
Atari Games Atari 2600 Space Invaders Rainbow Score 12,629.0 # 44
Atari Games Atari-57 Rainbow DQN Medium Human-Normalized Score 223.0% # 6

Methods used in the Paper


METHOD TYPE
Adam
Stochastic Optimization
Double Q-learning
Off-Policy TD Control
Prioritized Experience Replay
Replay Memory
Noisy Linear Layer
Randomized Value Functions
Dueling Network
Q-Learning Networks
N-step Returns
Value Function Estimation
Rainbow DQN
Q-Learning Networks
Q-Learning
Off-Policy TD Control
Dense Connections
Feedforward Networks
Convolution
Convolutions
DQN
Q-Learning Networks