no code implementations • 1 Mar 2024 • Emile Anand, Guannan Qu
This work proposes the SUB-SAMPLE-Q algorithm where the global agent subsamples $k\leq n$ local agents to compute an optimal policy in time that is only exponential in $k$, providing an exponential speedup from standard methods that are exponential in $n$.
no code implementations • 2 Feb 2024 • Neharika Jali, Guannan Qu, Weina Wang, Gauri Joshi
Unlike homogeneous systems, a threshold policy, that routes jobs to the slow server(s) when the queue length exceeds a certain threshold, is known to be optimal for the one-fast-one-slow two-server system.
1 code implementation • 14 Jan 2024 • Zeji Yi, Chaoyi Pan, Guanqi He, Guannan Qu, Guanya Shi
Sampling-based Model Predictive Control (MPC) has been a practical and effective approach in many domains, notably model-based reinforcement learning, thanks to its flexibility and parallelizability.
no code implementations • 7 Dec 2023 • Han Xu, Jialin Zheng, Guannan Qu
This paper addresses the challenges associated with decentralized voltage control in power grids due to an increase in distributed generations (DGs).
1 code implementation • 25 Mar 2023 • Songyuan Zhang, Yumeng Xiu, Guannan Qu, Chuchu Fan
Specifically, we treat a large networked dynamical system as an interconnection of smaller subsystems and develop methods that can find each subsystem a decentralized controller and an ISS Lyapunov function; the latter can be collectively composed to prove the global stability of the system.
no code implementations • 30 Nov 2022 • Yizhou Zhang, Guannan Qu, Pan Xu, Yiheng Lin, Zaiwei Chen, Adam Wierman
In particular, we show that, despite restricting each agent's attention to only its $\kappa$-hop neighborhood, the agents are able to learn a policy with an optimality gap that decays polynomially in $\kappa$.
Multi-agent Reinforcement Learning reinforcement-learning +1
1 code implementation • 16 Sep 2022 • Jie Feng, Yuanyuan Shi, Guannan Qu, Steven H. Low, Anima Anandkumar, Adam Wierman
In this paper, we propose a stability-constrained reinforcement learning (RL) method for real-time voltage control, that guarantees system stability both during policy learning and deployment of the learned policy.
no code implementations • 3 Jun 2022 • Sahin Lale, Yuanyuan Shi, Guannan Qu, Kamyar Azizzadenesheli, Adam Wierman, Anima Anandkumar
However, current reinforcement learning (RL) methods lack stabilization guarantees, which limits their applicability for the control of safety-critical systems.
no code implementations • 2 Jun 2022 • Tongxin Li, Ruixiao Yang, Guannan Qu, Yiheng Lin, Steven Low, Adam Wierman
Machine-learned black-box policies are ubiquitous for nonlinear control problems.
1 code implementation • 12 Apr 2022 • Sungho Shin, Yiheng Lin, Guannan Qu, Adam Wierman, Mihai Anitescu
This paper studies the trade-off between the degree of decentralization and the performance of a distributed controller in a linear-quadratic control setting.
no code implementations • 30 Sep 2021 • Yuanyuan Shi, Guannan Qu, Steven Low, Anima Anandkumar, Adam Wierman
Deep reinforcement learning (RL) has been recognized as a promising tool to address the challenges in real-time control of power systems.
no code implementations • 30 Sep 2021 • Jueming Hu, Zhe Xu, Weichang Wang, Guannan Qu, Yutian Pang, Yongming Liu
Experimental results show that local information is sufficient for DGRM and agents can accomplish complex tasks with the help of RM.
Multi-agent Reinforcement Learning reinforcement-learning +1
no code implementations • NeurIPS 2021 • Tongxin Li, Ruixiao Yang, Guannan Qu, Guanya Shi, Chenkai Yu, Adam Wierman, Steven H. Low
Motivated by online learning methods, we design a self-tuning policy that adaptively learns the trust parameter $\lambda$ with a competitive ratio that depends on $\varepsilon$ and the variation of system perturbations and predictions.
no code implementations • 29 Apr 2021 • Guannan Qu, Yuanyuan Shi, Sahin Lale, Anima Anandkumar, Adam Wierman
In this work, we propose an efficient online control algorithm, COvariance Constrained Online Linear Quadratic (COCO-LQ) control, that guarantees input-to-state stability for a large class of LTV systems while also minimizing the control cost.
no code implementations • 27 Jan 2021 • Xin Chen, Guannan Qu, Yujie Tang, Steven Low, Na Li
With large-scale integration of renewable generation and distributed energy resources, modern power systems are confronted with new operational challenges, such as growing complexity, increasing uncertainty, and aggravating volatility.
no code implementations • 19 Jun 2020 • Andreas Venzke, Guannan Qu, Steven Low, Spyros Chatzivasileiadis
This paper introduces for the first time a framework to obtain provable worst-case guarantees for neural network performance, using learning for optimal power flow (OPF) problems as a guiding example.
no code implementations • 12 Jun 2020 • Guannan Qu, Chenkai Yu, Steven Low, Adam Wierman
Model-free learning-based control methods have seen great success recently.
1 code implementation • NeurIPS 2021 • Yiheng Lin, Guannan Qu, Longbo Huang, Adam Wierman
We study multi-agent reinforcement learning (MARL) in a stochastic network of agents.
Multi-agent Reinforcement Learning reinforcement-learning +1
no code implementations • NeurIPS 2020 • Guannan Qu, Yiheng Lin, Adam Wierman, Na Li
It has long been recognized that multi-agent reinforcement learning (MARL) faces significant scalability issues due to the fact that the size of the state and action spaces are exponentially large in the number of agents.
Multi-agent Reinforcement Learning reinforcement-learning +1
no code implementations • L4DC 2020 • Guannan Qu, Adam Wierman, Na Li
We study reinforcement learning (RL) in a setting with a network of agents whose states and actions interact in a local manner where the objective is to find localized policies such that the (discounted) global reward is maximized.
no code implementations • 1 Feb 2020 • Guannan Qu, Adam Wierman
We consider a general asynchronous Stochastic Approximation (SA) scheme featuring a weighted infinity-norm contractive operator, and prove a bound on its finite-time convergence rate on a single trajectory.
no code implementations • 5 Dec 2019 • Guannan Qu, Adam Wierman, Na Li
We study reinforcement learning (RL) in a setting with a network of agents whose states and actions interact in a local manner where the objective is to find localized policies such that the (discounted) global reward is maximized.
no code implementations • 15 Sep 2019 • Guannan Qu, Na Li
Further, under some special conditions, we prove that the gap between the approximated reward function and the true reward function is decaying exponentially fast as the length of the truncated Markov process gets longer.