Search Results for author:
Found 26 papers, 18 papers with code
The ODE Method for Stochastic Approximation and Reinforcement Learning with Markovian Noise
Stochastic approximation is a class of algorithms that update a vector iteratively, incrementally, and stochastically, including, e. g., stochastic gradient descent and temporal difference learning.
AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning
StarCraft II is one of the most challenging simulated reinforcement learning environments; it is partially observable, stochastic, multi-agent, and mastering StarCraft II requires strategic planning over long time horizons with real-time low-level execution.
Direct Gradient Temporal Difference Learning
Gradient Temporal Difference (GTD) is one powerful tool to solve the deadly triad.
Improving Monte Carlo Evaluation with Offline Data
Most reinforcement learning practitioners evaluate their policies with online Monte Carlo estimators for either hyperparameter tuning or testing different algorithmic design choices, where the policy is repeatedly executed in the environment to get the average outcome.
On the Convergence of SARSA with Linear Function Approximation
SARSA, a classical on-policy control algorithm for reinforcement learning, is known to chatter when combined with linear function approximation: SARSA does not diverge but oscillates in a bounded region.
Global Optimality and Finite Sample Analysis of Softmax Off-Policy Actor Critic under State Distribution Mismatch
In this paper, we establish the global optimality and convergence rate of an off-policy actor critic algorithm in the tabular setting without using density ratio to correct the discrepancy between the state distribution of the behavior policy and that of the target policy.
Truncated Emphatic Temporal Difference Methods for Prediction and Control
Despite the theoretical success of emphatic TD methods in addressing the notorious deadly triad of off-policy RL, there are still two open problems.
Learning Expected Emphatic Traces for Deep RL
We develop a multi-step emphatic weighting that can be combined with replay, and a time-reversed $n$-step TD learning algorithm to learn the required emphatic weighting.
Breaking the Deadly Triad with a Target Network
The deadly triad refers to the instability of a reinforcement learning algorithm when it employs off-policy learning, function approximation, and bootstrapping simultaneously.
Average-Reward Off-Policy Policy Evaluation with Function Approximation
We consider off-policy policy evaluation with function approximation (FA) in average-reward MDPs, where the goal is to estimate both the reward rate and the differential value function.
A Deeper Look at Discounting Mismatch in Actor-Critic Algorithms
In the second scenario, we consider optimizing a discounted objective ($\gamma < 1$) and propose to interpret the omission of the discounting in the actor update from an auxiliary task perspective and provide supporting empirical results.
Learning Retrospective Knowledge with Reverse Reinforcement Learning
We present a Reverse Reinforcement Learning (Reverse RL) approach for representing retrospective knowledge.
Mean-Variance Policy Iteration for Risk-Averse Reinforcement Learning
We present a mean-variance policy iteration (MVPI) framework for risk-averse control in a discounted infinite horizon MDP optimizing the variance of a per-step reward random variable.
GradientDICE: Rethinking Generalized Offline Estimation of Stationary Values
Namely, the optimization problem in GenDICE is not a convex-concave saddle-point problem once nonlinearity in optimization variable parameterization is introduced to ensure positivity, so any primal-dual algorithm is not guaranteed to converge or find the desired solution.
Provably Convergent Two-Timescale Off-Policy Actor-Critic with Function Approximation
With the help of the emphasis critic and the canonical value function critic, we show convergence for COF-PAC, where the critics are linear and the actor can be nonlinear.
Distributional Reinforcement Learning for Efficient Exploration
In distributional reinforcement learning (RL), the estimated distribution of value function models both the parametric and intrinsic uncertainties.
Mega-Reward: Achieving Human-Level Play without Extrinsic Rewards
Intrinsic rewards were introduced to simulate how human intelligence works; they are usually evaluated by intrinsically-motivated play, i. e., playing games without extrinsic rewards but evaluated with extrinsic rewards.
Deep Residual Reinforcement Learning
We revisit residual algorithms in both model-free and model-based reinforcement learning settings.
Model-based Reinforcement Learning
reinforcement-learning
+1
DAC: The Double Actor-Critic Architecture for Learning Options
We reformulate the option framework as two parallel augmented MDPs.
Generalized Off-Policy Actor-Critic
We propose a new objective, the counterfactual objective, unifying existing objectives for off-policy policy gradient algorithms in the continuing reinforcement learning (RL) setting.
ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Search
In this paper, we propose an actor ensemble algorithm, named ACE, for continuous control with a deterministic policy in reinforcement learning.
QUOTA: The Quantile Option Architecture for Reinforcement Learning
In this paper, we propose the Quantile Option Architecture (QUOTA) for exploration based on recent advances in distributional reinforcement learning (RL).
mlpack 3: a fast, flexible machine learning library
In the past several years, the field of machine learning has seen an explosion of interest and excitement, with hundreds or thousands of algorithms developed for different tasks every year.
A Deeper Look at Experience Replay
Recently experience replay is widely used in various deep reinforcement learning (RL) algorithms, in this paper we rethink the utility of experience replay.
Comparing Deep Reinforcement Learning and Evolutionary Methods in Continuous Control
Reinforcement Learning and the Evolutionary Strategy are two major approaches in addressing complicated control problems.
Learning Representations by Stochastic Meta-Gradient Descent in Neural Networks
In this paper, we introduce a new incremental learning algorithm called crossprop, which learns incoming weights of hidden units based on the meta-gradient descent approach, that was previously introduced by Sutton (1992) and Schraudolph (1999) for learning step-sizes.
