Search Results for author:
Found 6 papers, 1 papers with code
From Stability to Chaos: Analyzing Gradient Descent Dynamics in Quadratic Regression
We conduct a comprehensive investigation into the dynamics of gradient descent using large-order constant step-sizes in the context of quadratic regression models.
Stochastic Nested Compositional Bi-level Optimization for Robust Feature Learning
We develop and analyze stochastic approximation algorithms for solving nested compositional bi-level optimization problems.
Optimal Algorithms for Stochastic Bilevel Optimization under Relaxed Smoothness Conditions
In this paper, we introduce a novel fully single-loop and Hessian-inversion-free algorithmic framework for stochastic bilevel optimization and present a tighter analysis under standard smoothness assumptions (first-order Lipschitzness of the UL function and second-order Lipschitzness of the LL function).
A One-Sample Decentralized Proximal Algorithm for Non-Convex Stochastic Composite Optimization
We focus on decentralized stochastic non-convex optimization, where $n$ agents work together to optimize a composite objective function which is a sum of a smooth term and a non-smooth convex term.
Decentralized Stochastic Bilevel Optimization with Improved per-Iteration Complexity
Bilevel optimization recently has received tremendous attention due to its great success in solving important machine learning problems like meta learning, reinforcement learning, and hyperparameter optimization.
Efficiently Escaping Saddle Points in Bilevel Optimization
Moreover, we propose an inexact NEgative-curvature-Originated-from-Noise Algorithm (iNEON), a pure first-order algorithm that can escape saddle point and find local minimum of stochastic bilevel optimization.
