no code implementations • 29 Aug 2022 • Yuan Peiwen, Henan Liu, Zhu Changsheng, Yuyi Wang
First, We examine how activation functions affect the forward and backward propagation of neural networks and derive a general form for gradient variance that extends the previous work in this area.
