Search Results for author:
Found 17 papers, 4 papers with code
Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks
In this work, we instantiate a regularized form of the gradient clipping algorithm and prove that it can converge to the global minima of deep neural network loss functions provided that the net is of sufficient width.
Investigating the Ability of PINNs To Solve Burgers' PDE Near Finite-Time BlowUp
Physics Informed Neural Networks (PINNs) have been achieving ever newer feats of solving complicated PDEs numerically while offering an attractive trade-off between accuracy and speed of inference.
LIPEx-Locally Interpretable Probabilistic Explanations-To Look Beyond The True Class
This data-efficiency is seen to manifest as LIPEx being able to compute its explanation matrix around 53% faster than all-class LIME, for classification experiments with text data.
Global Convergence of SGD For Logistic Loss on Two Layer Neural Nets
In this note, we demonstrate a first-of-its-kind provable convergence of SGD to the global minima of appropriately regularized logistic empirical risk of depth $2$ nets -- for arbitrary data and with any number of gates with adequately smooth and bounded activations like sigmoid and tanh.
Size Lowerbounds for Deep Operator Networks
Deep Operator Networks are an increasingly popular paradigm for solving regression in infinite dimensions and hence solve families of PDEs in one shot.
Global Convergence of SGD On Two Layer Neural Nets
In this note we demonstrate provable convergence of SGD to the global minima of appropriately regularized $\ell_2-$empirical risk of depth $2$ nets -- for arbitrary data and with any number of gates, if they are using adequately smooth and bounded activations like sigmoid and tanh.
Towards Size-Independent Generalization Bounds for Deep Operator Nets
In recent times machine learning methods have made significant advances in becoming a useful tool for analyzing physical systems.
An Empirical Study of the Occurrence of Heavy-Tails in Training a ReLU Gate
while training a $\relu$ gate (in the realizable and in the binary classification setup) and for a variant of S. G. D.
Dynamics of Local Elasticity During Training of Neural Nets
On various state-of-the-art neural network training on SVHN, CIFAR-10 and CIFAR-100 we demonstrate how our new proposal of $S_{\rm rel}$, as opposed to the original definition, much more sharply detects the property of the weight updates preferring to make prediction changes within the same class as the sampled data.
Investigating the Role of Overparameterization While Solving the Pendulum with DeepONets
DeepONets [1] are one of the most prominent ideas in this theme which entails an optimization over a space of inner-products of neural nets.
A Study of the Mathematics of Deep Learning
In chapter 2 we show new circuit complexity theorems for deep neural functions and prove classification theorems about these function spaces which in turn lead to exact algorithms for empirical risk minimization for depth 2 ReLU nets.
Provable Training of a ReLU Gate with an Iterative Non-Gradient Algorithm
In this work, we demonstrate provable guarantees on the training of a single ReLU gate in hitherto unexplored regimes.
Depth-2 Neural Networks Under a Data-Poisoning Attack
In this class of networks, we attempt to learn the network weights in the presence of a malicious oracle doing stochastic, bounded and additive adversarial distortions on the true output during training.
Convergence guarantees for RMSProp and ADAM in non-convex optimization and an empirical comparison to Nesterov acceleration
Through these experiments we demonstrate the interesting sensitivity that ADAM has to its momentum parameter $\beta_1$.
Lower bounds over Boolean inputs for deep neural networks with ReLU gates
We use the method of sign-rank to show exponential in dimension lower bounds for ReLU circuits ending in a LTF gate and of depths upto $O(n^{\xi})$ with $\xi < \frac{1}{8}$ with some restrictions on the weights in the bottom most layer.
Understanding Deep Neural Networks with Rectified Linear Units
In this paper we investigate the family of functions representable by deep neural networks (DNN) with rectified linear units (ReLU).
