Multilevel Initialization for Layer-Parallel Deep Neural Network Training

1 code implementation • 19 Dec 2019 • Eric C. Cyr, Stefanie Günther, Jacob B. Schroder

This paper investigates multilevel initialization strategies for training very deep neural networks with a layer-parallel multigrid solver.

Parallelizing Over Artificial Neural Network Training Runs with Multigrid

1 code implementation • 7 Aug 2017 • Jacob B. Schroder

This work considers a multigrid reduction in time (MGRIT) algorithm that is able to parallelize over the thousands of training runs and converge to the exact same solution as traditional training would provide.

Distributed Computing