Search Results for author:
Found 35 papers, 6 papers with code
Percentile Criterion Optimization in Offline Reinforcement Learning
The percentile criterion is approximately solved by constructing an \emph{ambiguity set} that contains the true model with high probability and optimizing the policy for the worst model in the set.
Data Poisoning Attacks on Off-Policy Policy Evaluation Methods
Off-policy Evaluation (OPE) methods are a crucial tool for evaluating policies in high-stakes domains such as healthcare, where exploration is often infeasible, unethical, or expensive.
Bayesian Regret Minimization in Offline Bandits
We study how to make decisions that minimize Bayesian regret in offline linear bandits.
On Dynamic Programming Decompositions of Static Risk Measures in Markov Decision Processes
However, we show that these popular decompositions for Conditional-Value-at-Risk (CVaR) and Entropic-Value-at-Risk (EVaR) are inherently suboptimal regardless of the discretization level.
Policy Gradient in Robust MDPs with Global Convergence Guarantee
In contrast with prior robust policy gradient algorithms, DRPG monotonically reduces approximation errors to guarantee convergence to a globally optimal policy in tabular RMDPs.
On the convex formulations of robust Markov decision processes
Our work opens a new research direction for RMDPs and can serve as a first step toward obtaining a tractable convex formulation of RMDPs.
RASR: Risk-Averse Soft-Robust MDPs with EVaR and Entropic Risk
We propose and analyze a new framework to jointly model the risk associated with epistemic and aleatory uncertainties in finite-horizon and discounted infinite-horizon MDPs.
Robust Phi-Divergence MDPs
In recent years, robust Markov decision processes (MDPs) have emerged as a prominent modeling framework for dynamic decision problems affected by uncertainty.
Fast Algorithms for $L_\infty$-constrained S-rectangular Robust MDPs
Robust Markov decision processes (RMDPs) are a useful building block of robust reinforcement learning algorithms but can be hard to solve.
Policy Gradient Bayesian Robust Optimization for Imitation Learning
Results suggest that PG-BROIL can produce a family of behaviors ranging from risk-neutral to risk-averse and outperforms state-of-the-art imitation learning algorithms when learning from ambiguous demonstrations by hedging against uncertainty, rather than seeking to uniquely identify the demonstrator's reward function.
Robust Maximum Entropy Behavior Cloning
Imitation learning (IL) algorithms use expert demonstrations to learn a specific task.
Soft-Robust Algorithms for Batch Reinforcement Learning
In reinforcement learning, robust policies for high-stakes decision-making problems with limited data are usually computed by optimizing the percentile criterion, which minimizes the probability of a catastrophic failure.
Bayesian Robust Optimization for Imitation Learning
Existing safe imitation learning approaches based on IRL deal with this uncertainty using a maxmin framework that optimizes a policy under the assumption of an adversarial reward function, whereas risk-neutral IRL approaches either optimize a policy for the mean or MAP reward function.
Entropic Risk Constrained Soft-Robust Policy Optimization
Having a perfect model to compute the optimal policy is often infeasible in reinforcement learning.
Partial Policy Iteration for L1-Robust Markov Decision Processes
Robust Markov decision processes (MDPs) allow to compute reliable solutions for dynamic decision problems whose evolution is modeled by rewards and partially-known transition probabilities.
Finite-Sample Analysis of Proximal Gradient TD Algorithms
In this paper, we analyze the convergence rate of the gradient temporal difference learning (GTD) family of algorithms.
Proximal Gradient Temporal Difference Learning: Stable Reinforcement Learning with Polynomial Sample Complexity
In this paper, we introduce proximal gradient temporal difference learning, which provides a principled way of designing and analyzing true stochastic gradient temporal difference learning algorithms.
Optimizing Norm-Bounded Weighted Ambiguity Sets for Robust MDPs
Our proposed method computes a weight parameter from the value functions, and these weights then drive the shape of the ambiguity sets.
Optimizing Percentile Criterion Using Robust MDPs
We then propose new algorithms that minimize the span of ambiguity sets defined by weighted $L_1$ and $L_\infty$ norms.
Robust Exploration with Tight Bayesian Plausibility Sets
Optimism about the poorly understood states and actions is the main driving force of exploration for many provably-efficient reinforcement learning algorithms.
Beyond Confidence Regions: Tight Bayesian Ambiguity Sets for Robust MDPs
Robust MDPs (RMDPs) can be used to compute policies with provable worst-case guarantees in reinforcement learning.
Policy-Conditioned Uncertainty Sets for Robust Markov Decision Processes
What policy should be employed in a Markov decision process with uncertain parameters?
Tight Bayesian Ambiguity Sets for Robust MDPs
Robustness is important for sequential decision making in a stochastic dynamic environment with uncertain probabilistic parameters.
Interpretable Reinforcement Learning with Ensemble Methods
We propose to use boosted regression trees as a way to compute human-interpretable solutions to reinforcement learning problems.
BIG-bench Machine Learning
Interpretable Machine Learning
+3
Fast Bellman Updates for Robust MDPs
The first algorithm uses a homotopy continuation method to compute updates for L1-constrained s, a-rectangular ambiguity sets.
A Practical Method for Solving Contextual Bandit Problems Using Decision Trees
We propose a new method for the contextual bandit problem that is simple, practical, and can be applied with little or no domain expertise.
Value Directed Exploration in Multi-Armed Bandits with Structured Priors
Multi-armed bandits are a quintessential machine learning problem requiring the balancing of exploration and exploitation.
Safe Policy Improvement by Minimizing Robust Baseline Regret
We show that our formulation is NP-hard and propose an approximate algorithm.
Building an Interpretable Recommender via Loss-Preserving Transformation
We propose a method for building an interpretable recommender system for personalizing online content and promotions.
Robust Partially-Compressed Least-Squares
While maintaining computational efficiency, our models provide robust solutions that are more accurate--relative to solutions of uncompressed least-squares--than those of classical compressed variants.
RAAM: The Benefits of Robustness in Approximating Aggregated MDPs in Reinforcement Learning
We describe how to use robust Markov decision processes for value function approximation with state aggregation.
A Bilinear Programming Approach for Multiagent Planning
Because the algorithm is formulated for bilinear programs, it is more general and simpler to implement.
Solution Methods for Constrained Markov Decision Process with Continuous Probability Modulation
We propose solution methods for previously-unsolved constrained MDPs in which actions can continuously modify the transition probabilities within some acceptable sets.
Robust Value Function Approximation Using Bilinear Programming
Existing value function approximation methods have been successfully used in many applications, but they often lack useful a priori error bounds.
Biasing Approximate Dynamic Programming with a Lower Discount Factor
We thus propose another justification: when the rewards are received only sporadically (as it is the case in Tetris), we can derive tighter bounds, which support a significant performance increase with a decrease in the discount factor.
