Search Results for author:
Found 47 papers, 5 papers with code
Control Theoretic Approach to Fine-Tuning and Transfer Learning
Given a training set in the form of a paired $(\mathcal{X},\mathcal{Y})$, we say that the control system $\dot{x} = f(x, u)$ has learned the paired set via the control $u^*$ if the system steers each point of $\mathcal{X}$ to its corresponding target in $\mathcal{Y}$.
Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction
Large language models (LLMs) have been driving a new wave of interactive AI applications across numerous domains.
Decision Transformer as a Foundation Model for Partially Observable Continuous Control
Closed-loop control of nonlinear dynamical systems with partial-state observability demands expert knowledge of a diverse, less standardized set of theoretical tools.
Policy Optimization finds Nash Equilibrium in Regularized General-Sum LQ Games
In this paper, we investigate the impact of introducing relative entropy regularization on the Nash Equilibria (NE) of General-Sum $N$-agent games, revealing the fact that the NE of such games conform to linear Gaussian policies.
Independent RL for Cooperative-Competitive Agents: A Mean-Field Perspective
This MFTG NE is then shown to be $\mathcal{O}(1/M)$-NE for the finite population game where $M$ is a lower bound on the number of agents in each team.
Learning How to Strategically Disclose Information
In this work, we consider an online version of information design where a sender interacts with a receiver of an unknown type who is adversarially chosen at each round.
Policy Optimization for PDE Control with a Warm Start
The PO step fine-tunes the model-based controller to compensate for the modeling error from dimensionality reduction.
$\widetilde{O}(T^{-1})$ Convergence to (Coarse) Correlated Equilibria in Full-Information General-Sum Markov Games
No-regret learning has a long history of being closely connected to game theory.
Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms
This project serves the learning for dynamics & control (L4DC) community, aiming to explore key questions: the convergence of RL algorithms in learning control policies; the stability and robustness issues of learning-based controllers; and the scalability of RL algorithms to high- and potentially infinite-dimensional systems.
Global Convergence of Receding-Horizon Policy Search in Learning Estimator Designs
We introduce the receding-horizon policy gradient (RHPG) algorithm, the first PG algorithm with provable global convergence in learning the optimal linear estimator designs, i. e., the Kalman filter (KF).
Large Population Games on Constrained Unreliable Networks
A Base station (BS) actively schedules agent communications over the network by minimizing a weighted Age of Information (WAoI) based cost function under a capacity limit $\mathcal{C} < N$ on the number of transmission attempts at each instant.
Revisiting LQR Control from the Perspective of Receding-Horizon Policy Gradient
We revisit in this paper the discrete-time linear quadratic regulator (LQR) problem from the perspective of receding-horizon policy gradient (RHPG), a newly developed model-free learning framework for control applications.
Learning the Kalman Filter with Fine-Grained Sample Complexity
We develop the first end-to-end sample complexity of model-free policy gradient (PG) methods in discrete-time infinite-horizon Kalman filtering.
Robust Reinforcement Learning for Risk-Sensitive Linear Quadratic Gaussian Control
This paper proposes a novel robust reinforcement learning framework for discrete-time linear systems with model mismatch that may arise from the sim-to-real gap.
An Improved Analysis of (Variance-Reduced) Policy Gradient and Natural Policy Gradient Methods
In this paper, we revisit and improve the convergence of policy gradient (PG), natural PG (NPG) methods, and their variance-reduced variants, under general smooth policy parametrizations.
Towards a Theoretical Foundation of Policy Optimization for Learning Control Policies
Gradient-based methods have been widely used for system design and optimization in diverse application domains.
Weighted Age of Information based Scheduling for Large Population Games on Networks
Due to a hard bandwidth constraint on the number of transmissions through the network, at most $R_d < N$ agents can concurrently access their state information through the network.
Oracle-free Reinforcement Learning in Mean-Field Games along a Single Sample Path
Given that the underlying Markov Decision Process (MDP) of the agent is communicating, we provide finite sample convergence guarantees in terms of convergence of the mean-field and control policy to the mean-field equilibrium.
Modeling and Analysis of a Coupled SIS Bi-Virus Model
We establish a sufficient condition and multiple necessary conditions for local exponential convergence to the boundary equilibrium (i. e., one virus persists, the other one dies out) of each virus.
Convergence and sample complexity of natural policy gradient primal-dual methods for constrained MDPs
We study sequential decision making problems aimed at maximizing the expected total reward while satisfying a constraint on the expected total utility.
Linear Quadratic Mean-Field Games with Communication Constraints
Since the complexity of solving the game increases with the number of agents, we use the Mean-Field Game paradigm to solve it.
Finite-Sample Analysis of Decentralized Q-Learning for Stochastic Games
With linear function approximation, the results are for convergence to a linear approximated equilibrium - a new notion of equilibrium that we propose - which describes that each agent's policy is a best reply (to other agents) within a linear space.
Provably Efficient Reinforcement Learning in Decentralized General-Sum Markov Games
We show that the agents can find an $\epsilon$-approximate CCE in at most $\widetilde{O}( H^6S A /\epsilon^2)$ episodes, where $S$ is the number of states, $A$ is the size of the largest individual action space, and $H$ is the length of an episode.
On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learning
Multi-agent reinforcement learning (MARL) algorithms often suffer from an exponential sample complexity dependence on the number of agents, a phenomenon known as \emph{the curse of multiagents}.
Adversarial Linear-Quadratic Mean-Field Games over Multigraphs
In this paper, we propose a game between an exogenous adversary and a network of agents connected via a multigraph.
Partial Observability Approach for the Optimal Transparency Problem in Multi-agent Systems
A principal is tasked to optimize the network's performance by controlling the information available to each agent with regard to other agents' latest actions.
Dynamical Systems Systems and Control Systems and Control Optimization and Control
Derivative-Free Policy Optimization for Linear Risk-Sensitive and Robust Control Design: Implicit Regularization and Sample Complexity
Direct policy search serves as one of the workhorses in modern reinforcement learning (RL), and its applications in continuous control tasks have recently attracted increasing attention.
Model-Free Non-Stationary RL: Near-Optimal Regret and Applications in Multi-Agent RL and Inventory Control
We consider model-free reinforcement learning (RL) in non-stationary Markov decision processes.
Reinforcement Learning in Non-Stationary Discrete-Time Linear-Quadratic Mean-Field Games
We propose an actor-critic algorithm to iteratively compute the mean-field equilibrium (MFE) of the LQ-MFG.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Model-Based Multi-Agent RL in Zero-Sum Markov Games with Near-Optimal Sample Complexity
This is in contrast to the usual reward-aware setting, with a $\tilde\Omega(|S|(|A|+|B|)(1-\gamma)^{-3}\epsilon^{-2})$ lower bound, where this model-based approach is near-optimal with only a gap on the $|A|,|B|$ dependence.
Model-based Reinforcement Learning
Reinforcement Learning (RL)
POLY-HOOT: Monte-Carlo Planning in Continuous Space MDPs with Non-Asymptotic Analysis
Monte-Carlo planning, as exemplified by Monte-Carlo Tree Search (MCTS), has demonstrated remarkable performance in applications with finite spaces.
Information State Embedding in Partially Observable Cooperative Multi-Agent Reinforcement Learning
To enable the development of tractable algorithms, we introduce the concept of an information state embedding that serves to compress agents' histories.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Fully Asynchronous Policy Evaluation in Distributed Reinforcement Learning over Networks
This paper proposes a \emph{fully asynchronous} scheme for the policy evaluation problem of distributed reinforcement learning (DisRL) over directed peer-to-peer networks.
Decentralized Multi-Agent Reinforcement Learning with Networked Agents: Recent Advances
Multi-agent reinforcement learning (MARL) has long been a significant and everlasting research topic in both machine learning and control.
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
Orthogonal to the existing reviews on MARL, we highlight several new angles and taxonomies of MARL theory, including learning in extensive-form games, decentralized MARL with networked agents, MARL in the mean-field regime, (non-)convergence of policy-based methods for learning in games, etc.
Non-Cooperative Inverse Reinforcement Learning
Through interacting with the more informed player, the less informed player attempts to both infer, and act according to, the true objective function.
Policy Optimization for $\mathcal{H}_2$ Linear Control with $\mathcal{H}_\infty$ Robustness Guarantee: Implicit Regularization and Global Convergence
In this paper, we study the convergence theory of PO for $\mathcal{H}_2$ linear control with $\mathcal{H}_\infty$-norm robustness guarantee.
Online Planning for Decentralized Stochastic Control with Partial History Sharing
To demonstrate the applicability of the model, we propose a novel collaborative intrusion response model, where multiple agents (defenders) possessing asymmetric information aim to collaboratively defend a computer network.
A Communication-Efficient Multi-Agent Actor-Critic Algorithm for Distributed Reinforcement Learning
This paper considers a distributed reinforcement learning problem in which a network of multiple agents aim to cooperatively maximize the globally averaged return through communication with only local neighbors.
Global Convergence of Policy Gradient Methods to (Almost) Locally Optimal Policies
Under a further strict saddle points assumption, this result establishes convergence to essentially locally-optimal policies of the underlying problem, and thus bridges the gap in existing literature on the convergence of PG methods.
Policy Optimization Provably Converges to Nash Equilibria in Zero-Sum Linear Quadratic Games
To the best of our knowledge, this work appears to be the first one to investigate the optimization landscape of LQ games, and provably show the convergence of policy optimization methods to the Nash equilibria.
Communication-Efficient Policy Gradient Methods for Distributed Reinforcement Learning
This paper deals with distributed policy optimization in reinforcement learning, which involves a central controller and a group of learners.
Finite-Sample Analysis For Decentralized Batch Multi-Agent Reinforcement Learning With Networked Agents
This work appears to be the first finite-sample analysis for batch MARL, a step towards rigorous theoretical understanding of general MARL algorithms in the finite-sample regime.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Distributed Learning of Average Belief Over Networks Using Sequential Observations
This paper addresses the problem of distributed learning of average belief with sequential observations, in which a network of $n>1$ agents aim to reach a consensus on the average value of their beliefs, by exchanging information only with their neighbors.
Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents
To this end, we propose two decentralized actor-critic algorithms with function approximation, which are applicable to large-scale MARL problems where both the number of states and the number of agents are massively large.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Reliable Intersection Control in Non-cooperative Environments
We propose a reliable intersection control mechanism for strategic autonomous and connected vehicles (agents) in non-cooperative environments.
Team-Optimal Distributed MMSE Estimation in General and Tree Networks
Finally, in the numerical examples, we demonstrate the superior performance of the introduced algorithms in the finite-horizon MSE sense due to optimal estimation.
