Search Results for author:
Found 15 papers, 5 papers with code
An Ordering of Divergences for Variational Inference with Factorized Gaussian Approximations
Our analysis covers the KL divergence, the R\'enyi divergences, and a score-based divergence that compares $\nabla\log p$ and $\nabla\log q$.
Computational-Statistical Gaps in Gaussian Single-Index Models
Single-Index Models are high-dimensional regression problems with planted structure, whereby labels depend on an unknown one-dimensional projection of the input via a generic, non-linear, and potentially non-deterministic transformation.
Batch and match: black-box variational inference with a score-based divergence
We analyze the convergence of BaM when the target distribution is Gaussian, and we prove that in the limit of infinite batch size the variational parameter updates converge exponentially quickly to the target mean and covariance.
On Learning Gaussian Multi-index Models with Gradient Flow
We study gradient flow on the multi-index regression problem for high-dimensional Gaussian data.
On Single Index Models beyond Gaussian Data
Sparse high-dimensional functions have arisen as a rich framework to study the behavior of gradient-descent methods using shallow neural networks, showcasing their ability to perform feature learning beyond linear models.
Kernelized Diffusion maps
Spectral clustering and diffusion maps are celebrated dimensionality reduction algorithms built on eigen-elements related to the diffusive structure of the data.
SGD with Large Step Sizes Learns Sparse Features
We present empirical observations that commonly used large step sizes (i) lead the iterates to jump from one side of a valley to the other causing loss stabilization, and (ii) this stabilization induces a hidden stochastic dynamics orthogonal to the bouncing directions that biases it implicitly toward sparse predictors.
Label noise (stochastic) gradient descent implicitly solves the Lasso for quadratic parametrisation
Understanding the implicit bias of training algorithms is of crucial importance in order to explain the success of overparametrised neural networks.
Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs
The training of neural networks by gradient descent methods is a cornerstone of the deep learning revolution.
Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity
Understanding the implicit bias of training algorithms is of crucial importance in order to explain the success of overparametrised neural networks.
Last iterate convergence of SGD for Least-Squares in the Interpolation regime.
Motivated by the recent successes of neural networks that have the ability to fit the data perfectly \emph{and} generalize well, we study the noiseless model in the fundamental least-squares setup.
Last iterate convergence of SGD for Least-Squares in the Interpolation regime
Motivated by the recent successes of neural networks that have the ability to fit the data perfectly and generalize well, we study the noiseless model in the fundamental least-squares setup.
Overcoming the curse of dimensionality with Laplacian regularization in semi-supervised learning
As annotations of data can be scarce in large-scale practical problems, leveraging unlabelled examples is one of the most important aspects of machine learning.
Statistical Optimality of Stochastic Gradient Descent on Hard Learning Problems through Multiple Passes
We consider stochastic gradient descent (SGD) for least-squares regression with potentially several passes over the data.
Exponential convergence of testing error for stochastic gradient methods
We consider binary classification problems with positive definite kernels and square loss, and study the convergence rates of stochastic gradient methods.
