L-GreCo: Layerwise-Adaptive Gradient Compression for Efficient and Accurate Deep Learning

1 code implementation • 31 Oct 2022 • Mohammadreza Alimohammadi, Ilia Markov, Elias Frantar, Dan Alistarh

Data-parallel distributed training of deep neural networks (DNN) has gained very widespread adoption, but can still experience communication bottlenecks.

Image Classification Language Modelling +1