Search Results for author:
Found 25 papers, 4 papers with code
Sample-efficient Learning of Infinite-horizon Average-reward MDPs with General Function Approximation
Moreover, for AMDPs, we propose a novel complexity measure -- average-reward generalized eluder coefficient (AGEC) -- which captures the challenge of exploration in AMDPs with general function approximation.
Distributionally Robust Reinforcement Learning with Interactive Data Collection: Fundamental Hardness and Near-Optimal Algorithm
Unlike previous work, which relies on a generative model or a pre-collected offline dataset enjoying good coverage of the deployment environment, we tackle robust RL via interactive data collection, where the learner interacts with the training environment only and refines the policy through trial and error.
Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment
We consider the problem of multi-objective alignment of foundation models with human preferences, which is a critical step towards helpful and harmless AI systems.
Rethinking Model-based, Policy-based, and Value-based Reinforcement Learning via the Lens of Representation Complexity
We first demonstrate that, for a broad class of Markov decision processes (MDPs), the model can be represented by constant-depth circuits with polynomial size or Multi-Layer Perceptrons (MLPs) with constant layers and polynomial hidden dimension.
Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraint
This includes an iterative version of the Direct Preference Optimization (DPO) algorithm for online settings, and a multi-step rejection sampling strategy for offline scenarios.
Horizon-Free and Instance-Dependent Regret Bounds for Reinforcement Learning with General Function Approximation
To tackle long planning horizon problems in reinforcement learning with general function approximation, we propose the first algorithm, termed as UCRL-WVTR, that achieves both \emph{horizon-free} and \emph{instance-dependent}, since it eliminates the polynomial dependency on the planning horizon.
Towards Robust Offline Reinforcement Learning under Diverse Data Corruption
Offline reinforcement learning (RL) presents a promising approach for learning reinforced policies from offline datasets without the need for costly or unsafe interactions with the environment.
Tackling Heavy-Tailed Rewards in Reinforcement Learning with Function Approximation: Minimax Optimal and Instance-Dependent Regret Bounds
Our algorithm, termed as \textsc{Heavy-LSVI-UCB}, achieves the \emph{first} computationally efficient \emph{instance-dependent} $K$-episode regret of $\tilde{O}(d \sqrt{H \mathcal{U}^*} K^\frac{1}{1+\epsilon} + d \sqrt{H \mathcal{V}^* K})$.
Maximize to Explore: One Objective Function Fusing Estimation, Planning, and Exploration
To achieve this, existing sample-efficient online RL algorithms typically consist of three components: estimation, planning, and exploration.
Provably Efficient Exploration in Quantum Reinforcement Learning with Logarithmic Worst-Case Regret
While quantum reinforcement learning (RL) has attracted a surge of attention recently, its theoretical understanding is limited.
A Reduction-based Framework for Sequential Decision Making with Delayed Feedback
We study stochastic delayed feedback in general multi-agent sequential decision making, which includes bandits, single-agent Markov decision processes (MDPs), and Markov games (MGs).
GEC: A Unified Framework for Interactive Decision Making in MDP, POMDP, and Beyond
The proposed algorithm modifies the standard posterior sampling algorithm in two aspects: (i) we use an optimistic prior distribution that biases towards hypotheses with higher values and (ii) a loglikelihood function is set to be the empirical loss evaluated on the historical data, where the choice of loss function supports both model-free and model-based learning.
Provable Sim-to-real Transfer in Continuous Domain with Partial Observations
Sim-to-real transfer trains RL agents in the simulated environments and then deploys them in the real world.
A Self-Play Posterior Sampling Algorithm for Zero-Sum Markov Games
Existing studies on provably efficient algorithms for Markov games (MGs) almost exclusively build on the "optimism in the face of uncertainty" (OFU) principle.
Nearly Minimax Optimal Offline Reinforcement Learning with Linear Function Approximation: Single-Agent MDP and Markov Game
We also extend our techniques to the two-player zero-sum Markov games (MGs), and establish a new performance lower bound for MGs, which tightens the existing result, and verifies the nearly minimax optimality of the proposed algorithm.
Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Power
Moreover, we establish an improved upper bound of $\exp({\mathcal{O}}(k))$ for the network size to achieve low robust generalization error when the data lies on a manifold with intrinsic dimension $k$ ($k \ll d$).
Human-in-the-loop: Provably Efficient Preference-based Reinforcement Learning with General Function Approximation
To the best of our knowledge, this is the first theoretical result for PbRL with (general) function approximation.
Pessimistic Minimax Value Iteration: Provably Efficient Equilibrium Learning from Offline Datasets
When the dataset does not have uniform coverage over all policy pairs, finding an approximate NE involves challenges in three aspects: (i) distributional shift between the behavior policy and the optimal policy, (ii) function approximation to handle large state space, and (iii) minimax optimization for equilibrium solving.
Can Reinforcement Learning Find Stackelberg-Nash Equilibria in General-Sum Markov Games with Myopic Followers?
We develop sample-efficient reinforcement learning (RL) algorithms for solving for an SNE in both online and offline settings.
Nearly Optimal Policy Optimization with Stable at Any Time Guarantee
Policy optimization methods are one of the most widely used classes of Reinforcement Learning (RL) algorithms.
Breaking the Moments Condition Barrier: No-Regret Algorithm for Bandits with Super Heavy-Tailed Payoffs
Despite a large amount of effort in dealing with heavy-tailed error in machine learning, little is known when moments of the error can become non-existential: the random noise $\eta$ satisfies Pr$\left[|\eta| > |y|\right] \le 1/|y|^{\alpha}$ for some $\alpha > 0$.
Optimistic Policy Optimization is Provably Efficient in Non-stationary MDPs
We study episodic reinforcement learning (RL) in non-stationary linear kernel Markov decision processes (MDPs).
Can Reinforcement Learning Efficiently Find Stackelberg-Nash Equilibria in General-Sum Markov Games?
To our best knowledge, we establish the first provably efficient RL algorithms for solving SNE in general-sum Markov games with leader-controlled state transitions.
A Reduction-Based Framework for Conservative Bandits and Reinforcement Learning
We also obtain a new upper bound for conservative low-rank MDP.
Risk-Sensitive Deep RL: Variance-Constrained Actor-Critic Provably Finds Globally Optimal Policy
In particular, we focus on a variance-constrained policy optimization problem where the goal is to find a policy that maximizes the expected value of the long-run average reward, subject to a constraint that the long-run variance of the average reward is upper bounded by a threshold.
