Dynamics of Local Elasticity During Training of Neural Nets

1 code implementation • 1 Nov 2021 • Soham Dan, Anirbit Mukherjee, Avirup Das, Phanideep Gampa

On various state-of-the-art neural network training on SVHN, CIFAR-10 and CIFAR-100 we demonstrate how our new proposal of $S_{\rm rel}$, as opposed to the original definition, much more sharply detects the property of the weight updates preferring to make prediction changes within the same class as the sampled data.

regression