Search Results for author:
Found 7 papers, 5 papers with code
Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures
In this work, we identify two different settings of linear weight-sharing layers which motivate two flavours of K-FAC -- $\textit{expand}$ and $\textit{reduce}$.
Accelerating Generalized Linear Models by Trading off Computation for Uncertainty
Bayesian Generalized Linear Models (GLMs) define a flexible probabilistic framework to model categorical, ordinal and continuous data, and are widely used in practice.
Benchmarking Neural Network Training Algorithms
In order to address these challenges, we introduce a new, competitive, time-to-result benchmark using multiple workloads running on fixed hardware, the AlgoPerf: Training Algorithms benchmark.
Cockpit: A Practical Debugging Tool for the Training of Deep Neural Networks
When engineers train deep learning models, they are very much 'flying blind'.
Descending through a Crowded Valley — Benchmarking Deep Learning Optimizers
(iii) While we can not discern an optimization method clearly dominating across all tested tasks, we identify a significantly reduced subset of specific algorithms and parameter choices that generally lead to competitive results in our experiments.
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers
Choosing the optimizer is considered to be among the most crucial design decisions in deep learning, and it is not an easy one.
DeepOBS: A Deep Learning Optimizer Benchmark Suite
We suggest routines and benchmarks for stochastic optimization, with special focus on the unique aspects of deep learning, such as stochasticity, tunability and generalization.
