Search Results for author:
Found 12 papers, 1 papers with code
Convergence guarantees for forward gradient descent in the linear regression model
Renewed interest in the relationship between artificial and biological neural networks motivates the study of gradient-free methods.
Hebbian learning inspired estimation of the linear regression parameters from queries
In this work, we study a variation of this Hebbian learning rule to recover the regression vector in the linear regression model.
Dropout Regularization Versus $\ell_2$-Penalization in the Linear Model
We investigate the statistical behavior of gradient descent iterates with dropout in the linear regression model.
Interpreting learning in biological neural networks as zero-order optimization method
In particular, the locality in the updating rule of the connection parameters in biological neural networks (BNNs) makes it biologically implausible that the learning of the brain is based on gradient descent.
On the inability of Gaussian process regression to optimally learn compositional functions
We rigorously prove that deep Gaussian process priors can outperform Gaussian process priors if the target function has a compositional structure.
On generalization bounds for deep networks based on loss surface implicit regularization
We argue that under reasonable assumptions, the local geometry forces SGD to stay close to a low dimensional subspace and that this induces another form of implicit regularization and results in tighter bounds on the generalization error for deep neural networks.
Convergence rates of deep ReLU networks for multiclass classification
For classification problems, trained deep neural networks return probabilities of class memberships.
The Kolmogorov-Arnold representation theorem revisited
There is a longstanding debate whether the Kolmogorov-Arnold representation theorem can explain the use of more than one hidden layer in neural networks.
On lower bounds for the bias-variance trade-off
In a second part of the article, the abstract lower bounds are applied to several statistical models including the Gaussian white noise model, a boundary estimation problem, the Gaussian sequence model and the high-dimensional linear regression model.
Deep ReLU network approximation of functions on a manifold
Whereas recovery of the manifold from data is a well-studied topic, approximation rates for functions defined on manifolds are less known.
A comparison of deep networks with ReLU activation function and linear spline-type methods
Deep neural networks (DNNs) generate much richer function spaces than shallow networks.
Nonparametric regression using deep neural networks with ReLU activation function
It is shown that estimators based on sparsely connected deep neural networks with ReLU activation function and properly chosen network architecture achieve the minimax rates of convergence (up to $\log n$-factors) under a general composition assumption on the regression function.
