Search Results for author:
Found 5 papers, 3 papers with code
SGD with Clipping is Secretly Estimating the Median Gradient
We then derive iterative methods based on the stochastic proximal point method for computing the geometric median and generalizations thereof.
Function Value Learning: Adaptive Learning Rates Based on the Polyak Stepsize and Function Splitting in ERM
We then move onto to develop $\texttt{FUVAL}$, a variant of $\texttt{SPS}_+$ where the loss values at optimality are gradually learned, as opposed to being given.
MoMo: Momentum Models for Adaptive Learning Rates
MoMo uses momentum estimates of the batch losses and gradients sampled at each iteration to build a model of the loss function.
A Stochastic Proximal Polyak Step Size
Developing a proximal variant of SPS is particularly important, since SPS requires a lower bound of the objective function to work well.
A Semismooth Newton Stochastic Proximal Point Algorithm with Variance Reduction
We develop an implementable stochastic proximal point (SPP) method for a class of weakly convex, composite optimization problems.
