Methods > Reinforcement Learning > Q-Learning Networks

Double DQN

Introduced by Hasselt et al. in Deep Reinforcement Learning with Double Q-learning

A Double Deep Q-Network, or Double DQN utilises Double Q-learning to reduce overestimation by decomposing the max operation in the target into action selection and action evaluation. We evaluate the greedy policy according to the online network, but we use the target network to estimate its value. The update is the same as for DQN, but replacing the target $Y^{DQN}_{t}$ with:

$$ Y^{DoubleDQN}_{t} = R_{t+1}+\gamma{Q}\left(S_{t+1}, \arg\max_{a}Q\left(S_{t+1}, a; \theta_{t}\right);\theta_{t}^{-}\right) $$

Compared to the original formulation of Double Q-Learning, in Double DQN the weights of the second network $\theta^{'}_{t}$ are replaced with the weights of the target network $\theta_{t}^{-}$ for the evaluation of the current greedy policy.

Source: Deep Reinforcement Learning with Double Q-learning

Latest Papers

PAPER DATE
Recurrent Rational Networks
| Quentin DelfossePatrick SchramowskiAlejandro MolinaKristian Kersting
2021-02-18
Self-correcting Q-Learning
Rong ZhuMattia Rigotti
2020-12-02
Chrome Dino Run using Reinforcement Learning
Divyanshu MarwahSneha SrivastavaAnusha GuptaShruti Verma
2020-08-15
Reinforcement Learning with Quantum Variational Circuits
Owen LockwoodMei Si
2020-08-15
Noise, overestimation and exploration in Deep Reinforcement Learning
Rafael Stekolshchik
2020-06-25
Learning Heuristic Selection with Dynamic Algorithm Configuration
| David SpeckAndré BiedenkappFrank HutterRobert MattmüllerMarius Lindauer
2020-06-15
Balancing a CartPole System with Reinforcement Learning -- A Tutorial
Swagat Kumar
2020-06-08
Dynamic Algorithm Configuration: Foundation of a New Meta-Algorithmic Framework
| André BiedenkappH. Furkan BozkurtTheresa EimerFrank HutterMarius Lindauer
2020-06-01
Disentangling Controllable Object through Video Prediction Improves Visual Reinforcement Learning
Yuanyi ZhongAlexander SchwingJian Peng
2020-02-21
Fast Reinforcement Learning for Anti-jamming Communications
Pei-Gen YeYuan-Gen WangJin LiLiang Xiao
2020-02-13
SLM Lab: A Comprehensive Benchmark and Modular Software Framework for Reproducible Deep Reinforcement Learning
| Keng Wah LoonLaura GraesserMilan Cvitkovic
2019-12-28
Exploiting the potential of deep reinforcement learning for classification tasks in high-dimensional and unstructured data
Johan S. Obando-CeronVictor Romero CanoWalter Mayor Toro
2019-12-20
Task-Oriented Language Grounding for Language Input with Multiple Sub-Goals of Non-Linear Order
| Vladislav KurenkovBulat MaksudovAdil Khan
2019-10-27
Reverse Experience Replay
Egor Rotinov
2019-10-19
Double Deep Q-Learning for Optimal Execution
Brian NingFranco Ho Ting LinSebastian Jaimungal
2018-12-17
Weighted Double Deep Multiagent Reinforcement Learning in Stochastic Cooperative Environments
Yan ZhengJianye HaoZongzhang Zhang
2018-02-23
Efficient Exploration through Bayesian Deep Q-Networks
| Kamyar AzizzadenesheliAnimashree Anandkumar
2018-02-13
Faster Deep Q-learning using Neural Episodic Control
Daichi NishioSatoshi Yamane
2018-01-06
Deep Reinforcement Learning with Double Q-learning
| Hado van HasseltArthur GuezDavid Silver
2015-09-22

Tasks

TASK PAPERS SHARE
Atari Games 5 41.67%
OpenAI Gym 3 25.00%
Image Classification 1 8.33%
Video Prediction 1 8.33%
Feature Selection 1 8.33%
Efficient Exploration 1 8.33%

Categories