Search Results for author:
Found 11 papers, 0 papers with code
Adaptive Consensus: A network pruning approach for decentralized optimization
We consider network-based decentralized optimization problems, where each node in the network possesses a local function and the objective is to collectively attain a consensus solution that minimizes the sum of all the local functions.
On the fast convergence of minibatch heavy ball momentum
Simple stochastic momentum methods are widely used in machine learning optimization, but their good practical performance is at odds with an absence of theoretical guarantees of acceleration in the literature.
Scalable Unidirectional Pareto Optimality for Multi-Task Learning with Constraints
Multi-objective optimization (MOO) problems require balancing competing objectives, often under constraints.
Adaptive Sampling Quasi-Newton Methods for Zeroth-Order Stochastic Optimization
We consider unconstrained stochastic optimization problems with no available gradient information.
A Retrospective Approximation Approach for Smooth Stochastic Optimization
Our investigation leads naturally to generalizing SG into Retrospective Approximation (RA) where, during each iteration, a "deterministic solver" executes possibly multiple steps on a subsampled deterministic problem and stops when further solving is deemed unnecessary from the standpoint of statistical efficiency.
Constrained and Composite Optimization via Adaptive Sampling Methods
The motivation for this paper stems from the desire to develop an adaptive sampling method for solving constrained optimization problems in which the objective function is stochastic and the constraints are deterministic.
Adaptive Sampling Quasi-Newton Methods for Derivative-Free Stochastic Optimization
We consider stochastic zero-order optimization problems, which arise in settings from simulation optimization to reinforcement learning.
A Progressive Batching L-BFGS Method for Machine Learning
The standard L-BFGS method relies on gradient approximations that are not dominated by noise, so that search directions are descent directions, the line search is reliable, and quasi-Newton updating yields useful quadratic models of the objective function.
Adaptive Sampling Strategies for Stochastic Optimization
In this paper, we propose a stochastic optimization method that adaptively controls the sample size used in the computation of gradient approximations.
An Investigation of Newton-Sketch and Subsampled Newton Methods
Sketching, a dimensionality reduction technique, has received much attention in the statistics community.
Exact and Inexact Subsampled Newton Methods for Optimization
The paper studies the solution of stochastic optimization problems in which approximations to the gradient and Hessian are obtained through subsampling.
