Search Results for author:
Found 7 papers, 1 papers with code
Non-asymptotic approximations of Gaussian neural networks via second-order Poincaré inequalities
There is a growing interest on large-width asymptotic properties of Gaussian neural networks (NNs), namely NNs whose weights are initialized according to Gaussian distributions.
Infinitely wide limits for deep Stable neural networks: sub-linear, linear and super-linear activation functions
As a novelty with respect to previous works, our results rely on the use of a generalized central limit theorem for heavy tails distributions, which allows for an interesting unified treatment of infinitely wide limits for deep Stable NNs.
Large-width asymptotics for ReLU neural networks with $α$-Stable initializations
As a difference with respect to the Gaussian setting, our result shows that the choice of the activation function affects the scaling of the NN, that is: to achieve the infinitely wide $\alpha$-Stable process, the ReLU activation requires an additional logarithmic term in the scaling with respect to sub-linear activations.
Deep Stable neural networks: large-width asymptotics and convergence rates
Then, we establish sup-norm convergence rates of the rescaled deep Stable NN to the Stable SP, under the ``joint growth" and a ``sequential growth" of the width over the NN's layers.
Large-width functional asymptotics for deep Gaussian neural networks
In this paper, we consider fully connected feed-forward deep neural networks where weights and biases are independent and identically distributed according to Gaussian distributions.
Infinite-channel deep stable convolutional neural networks
The interplay between infinite-width neural networks (NNs) and classes of Gaussian processes (GPs) is well known since the seminal work of Neal (1996).
Stable behaviour of infinitely wide deep neural networks
We consider fully connected feed-forward deep neural networks (NNs) where weights and biases are independent and identically distributed as symmetric centered stable distributions.
