Search Results for author:
Found 10 papers, 3 papers with code
Constrained and Composite Optimization via Adaptive Sampling Methods
The motivation for this paper stems from the desire to develop an adaptive sampling method for solving constrained optimization problems in which the objective function is stochastic and the constraints are deterministic.
A Noise-Tolerant Quasi-Newton Algorithm for Unconstrained Optimization
This paper describes an extension of the BFGS and L-BFGS methods for the minimization of a nonlinear function subject to errors.
Optimization and Control
A Progressive Batching L-BFGS Method for Machine Learning
The standard L-BFGS method relies on gradient approximations that are not dominated by noise, so that search directions are descent directions, the line search is reliable, and quasi-Newton updating yields useful quadratic models of the objective function.
Adaptive Sampling Strategies for Stochastic Optimization
In this paper, we propose a stochastic optimization method that adaptively controls the sample size used in the computation of gradient approximations.
An Investigation of Newton-Sketch and Subsampled Newton Methods
Sketching, a dimensionality reduction technique, has received much attention in the statistics community.
Exact and Inexact Subsampled Newton Methods for Optimization
The paper studies the solution of stochastic optimization problems in which approximations to the gradient and Hessian are obtained through subsampling.
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
The stochastic gradient descent (SGD) method and its variants are algorithms of choice for many Deep Learning tasks.
Optimization Methods for Large-Scale Machine Learning
This paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications.
A Multi-Batch L-BFGS Method for Machine Learning
The question of how to parallelize the stochastic gradient descent (SGD) method has received much attention in the literature.
Newton-Like Methods for Sparse Inverse Covariance Estimation
The second approach, which we call the Orthant-Based Newton method, is a two-phase algorithm that first identifies an orthant face and then minimizes a smooth quadratic approximation of the objective function using the conjugate gradient method.
