Balance is Essence: Accelerating Sparse Training via Adaptive Gradient Correction

1 code implementation • 9 Jan 2023 • Bowen Lei, Dongkuan Xu, Ruqi Zhang, Shuren He, Bani K. Mallick

To accelerate and stabilize the convergence of sparse training, we analyze the gradient changes and develop an adaptive gradient correction method.