Search Results for author:
Found 14 papers, 1 papers with code
Uncertainty quantification for learned ISTA
Model-based deep learning solutions to inverse problems have attracted increasing attention in recent years as they bridge state-of-the-art numerical performance with interpretability.
Robust Implicit Regularization via Weight Normalization
By analyzing key invariants of the gradient flow and using Lojasiewicz Theorem, we show that weight normalization also has an implicit bias towards sparse solutions in the diagonal linear model, but that in contrast to plain gradient flow, weight normalization enables a robust bias that persists even if the weights are initialized at practically large scale.
More is Less: Inducing Sparsity via Overparameterization
In deep learning it is common to overparameterize neural networks, that is, to use more parameters than training samples.
Generalization Error Bounds for Iterative Recovery Algorithms Unfolded as Neural Networks
Motivated by the learned iterative soft thresholding algorithm (LISTA), we introduce a general class of neural networks suitable for sparse reconstruction from few linear measurements.
Spark Deficient Gabor Frames for Inverse Problems
In this paper, we apply star-Digital Gabor Transform in analysis Compressed Sensing and speech denoising.
ADMM-DAD net: a deep unfolding network for analysis compressed sensing
In this paper, we propose a new deep unfolding neural network based on the ADMM algorithm for analysis Compressed Sensing.
Path classification by stochastic linear recurrent neural networks
We investigate the functioning of a classifying biological neural network from the perspective of statistical learning theory, modelled, in a simplified setting, as a continuous-time stochastic recurrent neural network (RNN) with identity activation function.
Convergence of gradient descent for learning linear neural networks
In the case of three or more layers we show that gradient descent converges to a global minimum on the manifold matrices of some fixed rank, where the rank cannot be determined a priori.
Unfolding recurrence by Green’s functions for optimized reservoir computing
Cortical networks are strongly recurrent, and neurons have intrinsic temporal dynamics.
Gradient Descent for Deep Matrix Factorization: Dynamics and Implicit Bias towards Low Rank
This suggests that deep learning prefers trajectories whose complexity (measuredin terms of effective rank) is monotonically increasing, which we believe is a fundamental concept for thetheoretical understanding of deep learning.
Compressive Sensing and Neural Networks from a Statistical Learning Perspective
The neural networks in this class are obtained by unfolding iterations of ISTA and learning some of the weights.
Unfolding recurrence by Green's functions for optimized reservoir computing
Cortical networks are strongly recurrent, and neurons have intrinsic temporal dynamics.
Overparameterization and generalization error: weighted trigonometric interpolation
Motivated by surprisingly good generalization properties of learned deep neural networks in overparameterized scenarios and by the related double descent phenomenon, this paper analyzes the relation between smoothness and low generalization error in an overparameterized linear learning problem.
Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers
We study the convergence of gradient flows related to learning deep linear neural networks (where the activation function is the identity map) from data.
