Search Results for author:
Found 9 papers, 5 papers with code
Unbiased Kinetic Langevin Monte Carlo with Inexact Gradients
We exhibit similar bounds using both approximate and stochastic gradients, and our method's computational cost is shown to scale logarithmically with the size of the dataset.
Multirate Training of Neural Networks
We also discuss splitting choices for the neural network parameters which could enhance generalization performance when neural networks are trained from scratch.
Better Training using Weight-Constrained Stochastic Dynamics
We employ constraints to control the parameter space of deep neural networks throughout training.
Constraint-Based Regularization of Neural Networks
We propose a method for efficiently incorporating constraints into a stochastic gradient Langevin framework for the training of deep neural networks.
Partitioned integrators for thermodynamic parameterization of neural networks
We describe easy-to-implement hybrid partitioned numerical algorithms, based on discretized stochastic differential equations, which are adapted to feed-forward neural networks, including a multi-layer Langevin algorithm, AdLaLa (combining the adaptive Langevin and Langevin algorithms) and LOL (combining Langevin and Overdamped Langevin); we examine the convergence of these methods using numerical studies and compare their performance among themselves and in relation to standard alternatives such as stochastic gradient descent and ADAM.
TATi-Thermodynamic Analytics ToolkIt: TensorFlow-based software for posterior sampling in machine learning applications
With the advent of GPU-assisted hardware and maturing high-efficiency software platforms such as TensorFlow and PyTorch, Bayesian posterior sampling for neural networks becomes plausible.
Ensemble preconditioning for Markov chain Monte Carlo simulation
We describe parallel Markov chain Monte Carlo methods that propagate a collective ensemble of paths, with local covariance information calculated from neighboring replicas.
Covariance-Controlled Adaptive Langevin Thermostat for Large-Scale Bayesian Sampling
Monte Carlo sampling for Bayesian posterior inference is a common approach used in machine learning.
Rational Construction of Stochastic Numerical Methods for Molecular Sampling
We then compare Langevin dynamics integrators in terms of their invariant distributions and demonstrate a superconvergence property (4th order accuracy where only 2nd order would be expected) of one method in the high friction limit; this method, moreover, can be reduced to a simple modification of the Euler-Maruyama method for Brownian dynamics involving a non-Markovian (coloured noise) random process.
Numerical Analysis Statistical Mechanics Chemical Physics Computational Physics
