Search Results for author:
Found 12 papers, 1 papers with code
Beyond Expectations: Learning with Stochastic Dominance Made Practical
Stochastic dominance models risk-averse preferences for decision making with uncertain outcomes, which naturally captures the intrinsic structure of the underlying uncertainty, in contrast to simply resorting to the expectations.
Federated Natural Policy Gradient Methods for Multi-task Reinforcement Learning
Federated reinforcement learning (RL) enables collaborative decision making of multiple distributed agents without sharing local data trajectories.
Global Convergence of Policy Gradient Methods in Reinforcement Learning, Games and Control
Policy gradient methods, where one searches for the policy of interest by maximizing the value functions using first-order information, become increasingly popular for sequential decision making in reinforcement learning, games, and control.
Asynchronous Gradient Play in Zero-Sum Multi-agent Games
Moving beyond, we demonstrate entropy-regularized OMWU -- by adopting two-timescale learning rates in a delay-aware manner -- enjoys faster last-iterate convergence under fixed delays, and continues to converge provably even when the delays are arbitrarily bounded in an average-iterate manner.
Faster Last-iterate Convergence of Policy Optimization in Zero-Sum Markov Games
Multi-Agent Reinforcement Learning (MARL) -- where multiple agents learn to interact in a shared dynamic environment -- permeates across a wide range of critical applications.
Independent Natural Policy Gradient Methods for Potential Games: Finite-time Global Convergence with Entropy Regularization
We show that the proposed method converges to the quantal response equilibrium (QRE) -- the equilibrium to the entropy-regularized game -- at a sublinear rate, which is independent of the size of the action space and grows at most sublinearly with the number of agents.
Fast Policy Extragradient Methods for Competitive Games with Entropy Regularization
Motivated by the algorithmic role of entropy regularization in single-agent reinforcement learning and game theory, we develop provably efficient extragradient methods to find the quantal response equilibrium (QRE) -- which are solutions to zero-sum two-player matrix games with entropy regularization -- at a linear rate.
Policy Mirror Descent for Regularized Reinforcement Learning: A Generalized Framework with Linear Convergence
These can often be accounted for via regularized RL, which augments the target value function with a structure-promoting regularizer.
Fast Global Convergence of Natural Policy Gradient Methods with Entropy Regularization
This class of methods is often applied in conjunction with entropy regularization -- an algorithmic scheme that encourages exploration -- and is closely related to soft policy iteration and trust region policy optimization.
Communication-Efficient Distributed Optimization in Networks with Gradient Tracking and Variance Reduction
There is growing interest in large-scale machine learning and optimization over decentralized networks, e. g. in the context of multi-agent learning and federated learning.
Convergence of Distributed Stochastic Variance Reduced Methods without Sampling Extra Data
Our theory captures how the convergence of distributed algorithms behaves as the number of machines and the size of local data vary.
A Stochastic Semismooth Newton Method for Nonsmooth Nonconvex Optimization
In this work, we present a globalized stochastic semismooth Newton method for solving stochastic optimization problems involving smooth nonconvex and nonsmooth convex terms in the objective function.
