Search Results for author:
Found 7 papers, 3 papers with code
Neural Tangent Kernel: A Survey
A seminal work [Jacot et al., 2018] demonstrated that training a neural network under specific parameterization is equivalent to performing a particular kernel method as width goes to infinity.
Feature Learning in $L_{2}$-regularized DNNs: Attraction/Repulsion and Sparsity
This second reformulation allows us to prove a sparsity result for homogeneous DNNs: any local minimum of the $L_{2}$-regularized loss can be achieved with at most $N(N+1)$ neurons in each hidden layer (where $N$ is the size of the training set).
Dynamically Stable Infinite-Width Limits of Neural Classifiers
Existing MF and NTK limit models, as well as one novel limit model, satisfy most of the properties demonstrated by finite-width models.
An Essay on Optimization Mystery of Deep Learning
Despite the huge empirical success of deep learning, theoretical understanding of neural networks learning process is still lacking.
Embedding-reparameterization procedure for manifold-valued latent variables in generative models
Conventional prior for Variational Auto-Encoder (VAE) is a Gaussian distribution.
Differentiable lower bound for expected BLEU score
In natural language processing tasks performance of the models is often measured with some non-differentiable metric, such as BLEU score.
Using stochastic computation graphs formalism for optimization of sequence-to-sequence model
Variety of machine learning problems can be formulated as an optimization task for some (surrogate) loss function.
