LRTuner: A Learning Rate Tuner for Deep Neural Networks

1 code implementation • ICML Workshop AutoML 2021 • Nikhil Iyer, V Thejas, Nipun Kwatra, Ramachandran Ramjee, Muthian Sivathanu

For example on ImageNet with Resnet-50, LRTuner shows up to 0. 2% absolute gains in test accuracy compared to the hand-tuned baseline schedule.

Wide-minima Density Hypothesis and the Explore-Exploit Learning Rate Schedule

2 code implementations • 9 Mar 2020 • Nikhil Iyer, V Thejas, Nipun Kwatra, Ramachandran Ramjee, Muthian Sivathanu

Several papers argue that wide minima generalize better than narrow minima.

Machine Translation

AutoLR: A Method for Automatic Tuning of Learning Rate

no code implementations • 25 Sep 2019 • Nipun Kwatra, V Thejas, Nikhil Iyer, Ramachandran Ramjee, Muthian Sivathanu

We compare favorably against state of the art learning rate schedules for the given dataset and models, including for ImageNet on Resnet-50, Cifar-10 on Resnet-18, and SQuAD fine-tuning on BERT.

Text-Based Joint Prediction of Numeric and Categorical Attributes of Entities in Knowledge Bases

no code implementations • RANLP 2019 • V Thejas, Abhijeet Gupta, Sebastian Pad{\'o}

Our analysis indicates that this is the case because categorical attributes, many of which describe membership in various classes, provide useful {`}background knowledge{'} for numeric prediction, while this is true to a lesser degree in the inverse direction.

Attribute Knowledge Base Completion +1