Search Results for author:
Found 9 papers, 1 papers with code
Unique Properties of Wide Minima in Deep Networks
In this paper, we characterize the wide minima in linear neural networks trained with a quadratic loss.
The Expected Loss of Preconditioned Langevin Dynamics Reveals the Hessian Rank
Langevin dynamics (LD) is widely used for sampling from distributions and for optimization.
The Implicit Bias of Minima Stability in Multivariate Shallow ReLU Networks
Finally, we prove that if a function is sufficiently smooth (in a Sobolev sense) then it can be approximated arbitrarily well using shallow ReLU networks that correspond to stable solutions of gradient descent.
Exact Mean Square Linear Stability Analysis for SGD
Furthermore, we show that SGD's stability threshold is equivalent to that of a mixture process which takes in each iteration a full batch gradient step w. p.
The Implicit Bias of Minima Stability: A View from Function Space
First, we extend the existing knowledge on minima stability to non-differentiable minima, which are common in ReLU nets.
Spectral Discovery of Jointly Smooth Features for Multimodal Data
We show analytically that our method is guaranteed to provide a set of orthogonal functions that are as jointly smooth as possible, ordered by increasing Dirichlet energy from the smoothest to the least smooth.
Unique Properties of Flat Minima in Deep Networks
In this paper, we characterize the flat minima in linear neural networks trained with a quadratic loss.
Revealing Common Statistical Behaviors in Heterogeneous Populations
In many areas of neuroscience and biological data analysis, it is desired to reveal common patterns among a group of subjects.
