Search Results for author:
Found 5 papers, 1 papers with code
Last Iterate Convergence of Incremental Methods and Applications in Continual Learning
We further obtain generalizations of our results to weighted averaging of the iterates with increasing weights, which can be seen as interpolating between the last iterate and the average iterate guarantees.
Variance Reduced Halpern Iteration for Finite-Sum Monotone Inclusions
Our main contributions are variants of the classical Halpern iteration that employ variance reduction to obtain improved complexity guarantees in which $n$ component operators in the finite sum are ``on average'' either cocoercive or Lipschitz continuous and monotone, with parameter $L$.
Empirical Risk Minimization with Shuffled SGD: A Primal-Dual Perspective and Improved Bounds
Contrary to the empirical practice of sampling from the datasets without replacement and with (possible) reshuffling at each epoch, the theoretical counterpart of SGD usually relies on the assumption of sampling with replacement.
Cyclic Block Coordinate Descent With Variance Reduction for Composite Nonconvex Optimization
Our convergence analysis is based on a gradient Lipschitz condition with respect to a Mahalanobis norm, inspired by a recent progress on cyclic block coordinate methods.
Stochastic Halpern Iteration with Variance Reduction for Stochastic Monotone Inclusions
We study stochastic monotone inclusion problems, which widely appear in machine learning applications, including robust regression and adversarial learning.
