PopSGD: Decentralized Stochastic Gradient Descent in the Population Model

no code implementations • 25 Sep 2019 • Giorgi Nadiradze, Amirmojtaba Sabour, Aditya Sharma, Ilia Markov, Vitaly Aksenov, Dan Alistarh.

We prove that, under standard assumptions, SGD can converge even in this extremely loose, decentralized setting, for both convex and non-convex objectives.

Distributed Optimization Scheduling