Search Results for author:
Found 18 papers, 1 papers with code
Fast Nonlinear Two-Time-Scale Stochastic Approximation: Achieving $O(1/k)$ Finite-Sample Complexity
This paper proposes to develop a new variant of the two-time-scale stochastic approximation to find the roots of two coupled nonlinear operators, assuming only noisy samples of these operators can be observed.
Convergence and Price of Anarchy Guarantees of the Softmax Policy Gradient in Markov Potential Games
Our focus in this paper is to study the convergence of the policy gradient method for solving MPGs under softmax policy parameterization, both tabular and parameterized with general function approximators such as neural networks.
Regularized Gradient Descent Ascent for Two-Player Zero-Sum Markov Games
We study the problem of finding the Nash equilibrium in a two-player zero-sum Markov game.
Convergence Rates of Two-Time-Scale Gradient Descent-Ascent Dynamics for Solving Nonconvex Min-Max Problems
Perhaps, the most popular first-order method in solving min-max optimization is the so-called simultaneous (or single-loop) gradient descent-ascent algorithm due to its simplicity in implementation.
Finite-Time Complexity of Online Primal-Dual Natural Actor-Critic Algorithm for Constrained Markov Decision Processes
To solve this constrained optimization program, we study an online actor-critic variant of a classic primal-dual method where the gradients of both the primal and dual functions are estimated using samples from a single trajectory generated by the underlying time-varying Markov processes.
A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learning
In our two-time-scale approach, one scale is to estimate the true gradient from these samples, which is then used to update the estimate of the optimal solution.
Byzantine Fault-Tolerance in Federated Local SGD under 2f-Redundancy
However, we do not know of any such techniques for the federated local SGD algorithm - a more commonly used method for federated machine learning.
Improved Convergence Rate for a Distributed Two-Time-Scale Gradient Method under Random Quantization
In particular, we show that the method converges at a rate $O(log_2 k/\sqrt k)$ to the optimal solution, when the underlying objective function is strongly convex and smooth.
Finite-Time Convergence Rates of Nonlinear Two-Time-Scale Stochastic Approximation under Markovian Noise
Such dependent data result to biased observations of the underlying operators.
Finite Sample Analysis of Two-Time-Scale Natural Actor-Critic Algorithm
In this paper, we characterize the \emph{global} convergence of an online natural actor-critic algorithm in the tabular setting using a single trajectory of samples.
Nonlinear Two-Time-Scale Stochastic Approximation: Convergence and Finite-Time Performance
Under some fairly standard assumptions, we provide a formula that characterizes the rate of convergence of the main iterates to the desired solutions.
Finite-Time Convergence Rates of Decentralized Stochastic Approximation with Applications in Multi-Agent and Multi-Task Learning
We study a decentralized variant of stochastic approximation, a data-driven approach for finding the root of an operator under noisy measurements.
Local Stochastic Approximation: A Unified View of Federated Learning and Distributed Multi-Task Reinforcement Learning Algorithms
Motivated by broad applications in reinforcement learning and federated learning, we study local stochastic approximation over a network of agents, where their goal is to find the root of an operator composed of the local operators at the agents.
Finite-Time Analysis of Stochastic Gradient Descent under Markov Randomness
Motivated by broad applications in reinforcement learning and machine learning, this paper considers the popular stochastic gradient descent (SGD) when the gradients of the underlying objective function are sampled from Markov processes.
Finite-Time Analysis and Restarting Scheme for Linear Two-Time-Scale Stochastic Approximation
Motivated by their broad applications in reinforcement learning, we study the linear two-time-scale stochastic approximation, an iterative method using two different step sizes for finding the solutions of a system of two equations.
Finite-Time Performance of Distributed Temporal Difference Learning with Linear Function Approximation
Our main contribution is to provide a finite-analysis on the performance of this distributed {\sf TD}$(\lambda)$ algorithm for both constant and time-varying step sizes.
Finite-Sample Analysis of Nonlinear Stochastic Approximation with Applications in Reinforcement Learning
To demonstrate the generality of our theoretical results on Markovian SA, we use it to derive the finite-sample bounds of the popular $Q$-learning with linear function approximation algorithm, under a condition on the behavior policy.
Finite-Time Analysis of Distributed TD(0) with Linear Function Approximation for Multi-Agent Reinforcement Learning
In this problem, a group of agents works cooperatively to evaluate the value function for the global discounted accumulative reward problem, which is composed of local rewards observed by the agents.
Optimization and Control
