Search Results for author:
Found 5 papers, 2 papers with code
Random matrix theory and the loss surfaces of neural networks
Informed by the historical applications of random matrix theory in physics and elsewhere, we establish the presence of local random matrix universality in real neural networks and then utilise this as a modeling assumption to derive powerful and novel results about the Hessians of neural network loss surfaces and their spectra.
Universal characteristics of deep neural network loss surfaces from random matrix theory
This paper considers several aspects of random matrix universality in deep neural networks.
A novel sampler for Gauss-Hermite determinantal point processes with application to Monte Carlo integration
Determinantal points processes are a promising but relatively under-developed tool in machine learning and statistical modelling, being the canonical statistical example of distributions with repulsion.
Appearance of Random Matrix Theory in Deep Learning
We further investigate the importance of the true loss surface in neural networks and find, in contrast to previous work, that the exponential hardness of locating the global minimum has practical consequences for achieving state of the art performance.
A spin-glass model for the loss surfaces of generative adversarial networks
Our model consists of two interacting spin glasses, and we conduct an extensive theoretical analysis of the complexity of the model's critical points using techniques from Random Matrix Theory.
