Gradient-only line searches to automatically determine learning rates for a variety of stochastic training algorithms

1 code implementation • 29 Jun 2020 • Dominic Kafka, Daniel Nicolas Wilke

Gradient-only and probabilistic line searches have recently reintroduced the ability to adaptively determine learning rates in dynamic mini-batch sub-sampled neural network training.