Bidirectional Looking with A Novel Double Exponential Moving Average to Adaptive and Non-adaptive Momentum Optimizers

1 code implementation • 2 Jul 2023 • Yineng Chen, Zuchao Li, Lefei Zhang, Bo Du, Hai Zhao

SGD and Adam are two classical and effective optimizers on which researchers have proposed many variants, such as SGDM and RAdam.