Search Results for author:
Found 20 papers, 4 papers with code
Network Pruning by Greedy Subnetwork Selection
Theoretically, we show that the small networks pruned using our method achieve provably lower loss than small networks trained from scratch with the same size.
One-Dimensional Deep Image Prior for Curve Fitting of S-Parameters from Electromagnetic Solvers
DIP is a technique that optimizes the weights of a randomly-initialized convolutional neural network to fit a signal from noisy or under-determined measurements.
Ambient Diffusion: Learning Clean Distributions from Corrupted Data
We present the first diffusion-based framework that can learn an unknown distribution using only highly-corrupted samples.
Efficiently Learning One Hidden Layer ReLU Networks From Queries
While the problem of PAC learning neural networks from samples has received considerable attention in recent years, in certain settings like model extraction attacks, it is reasonable to imagine having more than just the ability to observe random labeled examples.
Tight Hardness Results for Training Depth-2 ReLU Networks
We are also able to obtain lower bounds on the running time in terms of the desired additive error $\epsilon$.
The Polynomial Method is Universal for Distribution-Free Correlational SQ Learning
Generalizing a beautiful work of Malach and Shalev-Shwartz (2022) that gave tight correlational SQ (CSQ) lower bounds for learning DNF formulas, we give new proofs that lower bounds on the threshold or approximate degree of any function class directly imply CSQ lower bounds for PAC or agnostic learning respectively.
From Boltzmann Machines to Neural Networks and Back Again
Graphical models are powerful tools for modeling high-dimensional data, but learning graphical models in the presence of latent variables is well-known to be difficult.
Statistical-Query Lower Bounds via Functional Gradients
We give the first statistical-query lower bounds for agnostically learning any non-polynomial activation with respect to Gaussian marginals (e. g., ReLU, sigmoid, sign).
Superpolynomial Lower Bounds for Learning One-Layer Neural Networks using Gradient Descent
Our lower bounds hold for broad classes of activations including ReLU and sigmoid.
Good Subnetworks Provably Exist: Pruning via Greedy Forward Selection
This differs from the existing methods based on backward elimination, which remove redundant neurons from the large network.
Time/Accuracy Tradeoffs for Learning a ReLU with respect to Gaussian Marginals
Let $\mathsf{opt} < 1$ be the population loss of the best-fitting ReLU.
Efficient Algorithms for Outlier-Robust Regression
We give the first polynomial-time algorithm for performing linear or polynomial regression resilient to adversarial corruptions in both examples and labels.
Learning One Convolutional Layer with Overlapping Patches
We give the first provably efficient algorithm for learning a one hidden layer convolutional network with respect to a general class of (potentially overlapping) patches.
Learning Neural Networks with Two Nonlinear Layers in Polynomial Time
We give a polynomial-time algorithm for learning neural networks with one layer of sigmoids feeding into any Lipschitz, monotone activation function (e. g., sigmoid or ReLU).
Eigenvalue Decay Implies Polynomial-Time Learnability for Neural Networks
In this work we show that a natural distributional assumption corresponding to {\em eigenvalue decay} of the Gram matrix yields polynomial-time algorithms in the non-realizable setting for expressive classes of networks (e. g. feed-forward networks of ReLUs).
Learning Graphical Models Using Multiplicative Weights
Our main application is an algorithm for learning the structure of t-wise MRFs with nearly-optimal sample complexity (up to polynomial losses in necessary terms that depend on the weights) and running time that is $n^{O(t)}$.
Hyperparameter Optimization: A Spectral Approach
In particular, we obtain the first quasi-polynomial time algorithm for learning noisy decision trees with polynomial sample complexity.
Exact MAP Inference by Avoiding Fractional Vertices
We require that the number of fractional vertices in the LP relaxation exceeding the optimal solution is bounded by a polynomial in the problem size.
Reliably Learning the ReLU in Polynomial Time
These results are in contrast to known efficient algorithms for reliably learning linear threshold functions, where $\epsilon$ must be $\Omega(1)$ and strong assumptions are required on the marginal distribution.
Sparse Polynomial Learning and Graph Sketching
We give an algorithm for exactly reconstructing f given random examples from the uniform distribution on $\{-1, 1\}^n$ that runs in time polynomial in $n$ and $2s$ and succeeds if the function satisfies the unique sign property: there is one output value which corresponds to a unique set of values of the participating parities.
