Exponential Decay

Exponential Decay is a learning rate schedule where we decay the learning rate with more iterations using an exponential function:

$$ \text{lr} = \text{lr}_{0}\exp\left(-kt\right) $$

Image Credit: Suki Lau

Latest Papers

PAPER DATE
Noise-Induced Barren Plateaus in Variational Quantum Algorithms
Samson WangEnrico FontanaM. CerezoKunal SharmaAkira SoneLukasz CincioPatrick J. Coles
2020-07-28
Context-Aware Attentive Knowledge Tracing
| Aritra GhoshNeil HeffernanAndrew S. Lan
2020-07-24
Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Reward
Guannan QuYiheng LinAdam WiermanNa Li
2020-06-11
Revisiting Regex Generation for Modeling Industrial Applications by Incorporating Byte Pair Encoder
Desheng WangJiawei LiuXiang QiBaolin SunPeng Zhang
2020-05-06
Supervised Contrastive Learning
| Prannay KhoslaPiotr TeterwakChen WangAaron SarnaYonglong TianPhillip IsolaAaron MaschinotCe LiuDilip Krishnan
2020-04-23
RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databases
DongHyun ChoiMyeong Cheol ShinEungGyun KimDong Ryeol Shin
2020-04-07
Scalable Gradients for Stochastic Differential Equations
| Xuechen LiTing-Kam Leonard WongRicky T. Q. ChenDavid Duvenaud
2020-01-05
Momentum Contrast for Unsupervised Visual Representation Learning
| Kaiming HeHaoqi FanYuxin WuSaining XieRoss Girshick
2019-11-13
Learning pairwise Markov network structures using correlation neighborhoods
Juri KuronenJukka CoranderJohan Pensar
2019-10-30
Quasiclassical Modeling of Cavity Quantum Electrodynamics
Tao E. LiHsing-Ta ChenAbraham NitzanJoseph E. Subotnik
2019-10-05
Global exponential stability of primal-dual gradient flow dynamics based on the proximal augmented Lagrangian: A Lyapunov-based approach
Dongsheng DingMihailo R. Jovanović
2019-10-02
Augmented Memory for Correlation Filters in Real-Time UAV Tracking
| Yiming LiChanghong FuFangqiang DingZiyuan HuangJia Pan
2019-09-24
Asymptotically Optimal One- and Two-Sample Testing with Kernels
Shengyu ZhuBiao ChenZhitang ChenPengfei Yang
2019-08-27
Learning the Model Update for Siamese Trackers
| Lichao ZhangAbel Gonzalez-GarciaJoost van de WeijerMartin DanelljanFahad Shahbaz Khan
2019-08-02
Supervised Symbolic Music Style Translation Using Synthetic Data
| Ondřej CífkaUmut ŞimşekliGaël Richard
2019-07-04
Measurement-driven entanglement transition in hybrid quantum circuits
Yaodong LiXiao ChenMatthew P. A. Fisher
2019-01-23
Concentration bounds for CVaR estimation: The cases of light-tailed and heavy-tailed distributions
Prashanth L. A.Krishna JagannathanRavi Kumar Kolla
2019-01-04
K-nearest Neighbor Search by Random Projection Forests
Donghui YanYingjie WangJin WangHonggang WangZhenpeng Li
2018-12-31
Shortcut Matrix Product States and its applications
Zhuan LiPan Zhang
2018-12-13
Predicting Recall Probability to Adaptively Prioritize Study
Shane MooneyKaren SunEric Bomgardner
2018-02-28
Exponentially Consistent Kernel Two-Sample Tests
Shengyu ZhuBiao ChenZhitang Chen
2018-02-23
Universal Hypothesis Testing with Kernels: Asymptotically Optimal Tests for Goodness of Fit
Shengyu ZhuBiao ChenPengfei YangZhitang Chen
2018-02-21
Approximation beats concentration? An approximation view on inference with smooth radial kernels
Mikhail Belkin
2018-01-10
Scale-invariant temporal history (SITH): optimal slicing of the past in an uncertain world
Tyler A. SpearsBrandon G. JacquesMarc W. HowardPer B. Sederberg
2017-12-19
Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions
| Jonathan ShenRuoming PangRon J. WeissMike SchusterNavdeep JaitlyZongheng YangZhifeng ChenYu ZhangYuxuan WangRJ Skerry-RyanRif A. SaurousYannis AgiomyrgiannakisYonghui Wu
2017-12-16
Learning Transferable Architectures for Scalable Image Recognition
| Barret ZophVijay VasudevanJonathon ShlensQuoc V. Le
2017-07-21
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
| Andrew G. HowardMenglong ZhuBo ChenDmitry KalenichenkoWeijun WangTobias WeyandMarco AndreettoHartwig Adam
2017-04-17
A generalized Bayesian framework for the analysis of subscription based businesses
Rahul MadhavanAnkit Baraskar
2017-04-12
Double spend races
Cyril GrunspanRicardo Pérez-Marco
2017-02-06
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
| Christian SzegedySergey IoffeVincent VanhouckeAlex Alemi
2016-02-23
Uniform {\varepsilon}-Stability of Distributed Nonlinear Filtering over DNAs: Gaussian-Finite HMMs
Dionysios S. KalogeriasAthina P. Petropulu
2016-02-16
Rethinking the Inception Architecture for Computer Vision
| Christian SzegedyVincent VanhouckeSergey IoffeJonathon ShlensZbigniew Wojna
2015-12-02
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
| Sergey IoffeChristian Szegedy
2015-02-11
Bayesian Inference for NMR Spectroscopy with Applications to Chemical Quantification
Andrew Gordon WilsonYuting WuDaniel J. HollandSebastian NowozinMick D. MantleLynn F. GladdenAndrew Blake
2014-02-14

Components

COMPONENT TYPE
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories