HetSeq: Distributed GPU Training on Heterogeneous Infrastructure

25 Sep 2020 Yifan Ding Nicholas Botzer Tim Weninger

Modern deep learning systems like PyTorch and Tensorflow are able to train enormous models with billions (or trillions) of parameters on a distributed infrastructure. These systems require that the internal nodes have the same memory capacity and compute performance... (read more)

PDF Abstract

Results from the Paper


  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.

Methods used in the Paper