Search Results for author:
Found 24 papers, 10 papers with code
An MRP Formulation for Supervised Learning: Generalized Temporal Difference Learning Models
In traditional statistical learning, data points are usually assumed to be independently and identically distributed (i. i. d.)
A Simple Mixture Policy Parameterization for Improving Sample Efficiency of CVaR Optimization
Reinforcement learning algorithms utilizing policy gradients (PG) to optimize Conditional Value at Risk (CVaR) face significant challenges with sample inefficiency, hindering their practical applications.
Improving Adversarial Transferability via Model Alignment
During the alignment process, the parameters of the source model are fine-tuned to minimize an alignment loss.
Understanding the robustness difference between stochastic gradient descent and adaptive gradient methods
In the context of deep learning, our experiments show that SGD-trained neural networks have smaller Lipschitz constants, explaining the better robustness to input perturbations than those trained with adaptive gradient methods.
Conditionally Optimistic Exploration for Cooperative Deep Multi-Agent Reinforcement Learning
Efficient exploration is critical in cooperative deep Multi-Agent Reinforcement Learning (MARL).
The In-Sample Softmax for Offline Reinforcement Learning
We highlight a simple fact: it is more straightforward to approximate an in-sample \emph{softmax} using only actions in the dataset.
Label Alignment Regularization for Distribution Shift
Recent work has highlighted the label alignment property (LAP) in supervised learning, where the vector of all labels in the dataset is mostly in the span of the top few singular vectors of the data matrix.
Memory-efficient Reinforcement Learning with Value-based Knowledge Consolidation
The experience replay buffer, a standard component in deep reinforcement learning, is often used to reduce forgetting and improve sample efficiency by storing experiences in a large buffer and using them for training later.
STOPS: Short-Term-based Volatility-controlled Policy Search and its Global Convergence
This paper proposes Short-Term VOlatility-controlled Policy Search (STOPS), a novel algorithm that solves risk-averse problems by learning from short-term trajectories instead of long-term trajectories.
An Alternate Policy Gradient Estimator for Softmax Policies
Policy gradient (PG) estimators are ineffective in dealing with softmax policies that are sub-optimally saturated, which refers to the situation when the policy concentrates its probability mass on sub-optimal actions.
Beyond Prioritized Replay: Sampling States in Model-Based RL via Simulated Priorities
The prioritized Experience Replay (ER) method has attracted great attention; however, there is little theoretical understanding of such prioritization strategy and why they help.
Understanding and Mitigating the Limitations of Prioritized Experience Replay
Prioritized Experience Replay (ER) has been empirically shown to improve sample efficiency across many domains and attracted great attention; however, there is little theoretical understanding of why such prioritized sampling helps and its limitations.
Maxmin Q-learning: Controlling the Estimation Bias of Q-learning
Q-learning suffers from overestimation bias, because it approximates the maximum action value using the maximum estimated action value.
An implicit function learning approach for parametric modal regression
We empirically demonstrate on several synthetic problems that our method (i) can learn multi-valued functions and produce the conditional modes, (ii) scales well to high-dimensional inputs, and (iii) can even be more effective for certain uni-modal problems, particularly for high-frequency functions.
Frequency-based Search-control in Dyna
This suggests a search-control strategy: we should use states from high frequency regions of the value function to query the model to acquire more samples.
Fuzzy Tiling Activations: A Simple Approach to Learning Sparse Representations Online
Recent work has shown that sparse representations -- where only a small percentage of units are active -- can significantly reduce interference.
Hill Climbing on Value Estimates for Search-control in Dyna
In this work, we propose to generate such states by using the trajectory obtained from Hill Climbing (HC) the current estimate of the value function.
Model-based Reinforcement Learning
Reinforcement Learning (RL)
Greedy Actor-Critic: A New Conditional Cross-Entropy Method for Policy Improvement
We first provide a policy improvement result in an idealized setting, and then prove that our conditional CEM (CCEM) strategy tracks a CEM update per state, even with changing action-values.
Reinforcement Learning with Function-Valued Action Spaces for Partial Differential Equation Control
Recent work has shown that reinforcement learning (RL) is a promising approach to control dynamical systems described by partial differential equations (PDE).
Organizing Experience: A Deeper Look at Replay Mechanisms for Sample-based Planning in Continuous State Domains
We show that a model, as opposed to a replay buffer, is particularly useful for specifying which states to sample from during planning, such as predecessor states that propagate information in reverse from a state more quickly.
Effective sketching methods for value function approximation
As a remedy, we demonstrate how to use sketching more sparingly, with only a left-sided sketch, that can still enable significant computational gains and the use of these matrix-based learning algorithms that are less sensitive to parameters.
Adapting Kernel Representations Online Using Submodular Maximization
In this work, we develop an approximately submodular criterion for this setting, and an efficient online greedy submodular maximization algorithm for optimizing the criterion.
Accelerated Gradient Temporal Difference Learning
The family of temporal difference (TD) methods span a spectrum from computationally frugal linear methods like TD({\lambda}) to data efficient least squares methods.
Incremental Truncated LSTD
Balancing between computational efficiency and sample efficiency is an important goal in reinforcement learning.
