Search Results for author:
Found 5 papers, 1 papers with code
GOALS: Gradient-Only Approximations for Line Searches Towards Robust and Consistent Training of Deep Neural Networks
The results show that training a model with the recommended learning rate for a class of search directions helps to reduce the model errors in multimodal cases.
Gradient-only line searches to automatically determine learning rates for a variety of stochastic training algorithms
Gradient-only and probabilistic line searches have recently reintroduced the ability to adaptively determine learning rates in dynamic mini-batch sub-sampled neural network training.
Resolving learning rates adaptively by locating Stochastic Non-Negative Associated Gradient Projection Points using line searches
This study proposes gradient-only line searches to resolve the learning rate for neural network training algorithms.
Gradient-only line searches: An Alternative to Probabilistic Line Searches
Line searches are capable of adaptively resolving learning rate schedules.
Traversing the noise of dynamic mini-batch sub-sampled loss functions: A visual guide
Mini-batch sub-sampling in neural network training is unavoidable, due to growing data demands, memory-limited computational resources such as graphical processing units (GPUs), and the dynamics of on-line learning.
