DistIR: An Intermediate Representation and Simulator for Efficient Neural Network Distribution

no code implementations • 9 Nov 2021 • Keshav Santhanam, Siddharth Krishna, Ryota Tomioka, Tim Harris, Matei Zaharia

The rapidly growing size of deep neural network (DNN) models and datasets has given rise to a variety of distribution strategies such as data, tensor-model, pipeline parallelism, and hybrid combinations thereof.

Efficient Neural Network