MQGrad: Reinforcement Learning of Gradient Quantization in Parameter Server

no code implementations • 22 Apr 2018 • Guoxin Cui, Jun Xu, Wei Zeng, Yanyan Lan, Jiafeng Guo, Xue-Qi Cheng

One of the most significant bottleneck in training large scale machine learning models on parameter server (PS) is the communication overhead, because it needs to frequently exchange the model gradients between the workers and servers during the training iterations.

BIG-bench Machine Learning Quantization +2