Layer-Parallel Training of Residual Networks with Auxiliary Variables

no code implementations • NeurIPS Workshop DLDE 2021 • Qi Sun, Hexin Dong, Zewei Chen, Weizhen Dian, Jiacheng Sun, Yitong Sun, Zhenguo Li, Bin Dong

Backpropagation algorithm is indispensable for training modern residual networks (ResNets) and usually tends to be time-consuming due to its inherent algorithmic lockings.

Data Augmentation

A Practical Layer-Parallel Training Algorithm for Residual Networks

no code implementations • 3 Sep 2020 • Qi Sun, Hexin Dong, Zewei Chen, Weizhen Dian, Jiacheng Sun, Yitong Sun, Zhenguo Li, Bin Dong

Gradient-based algorithms for training ResNets typically require a forward pass of the input data, followed by back-propagating the objective gradient to update parameters, which are time-consuming for deep ResNets.

Data Augmentation