1 code implementation • 24 Mar 2022 • Cheng Kevin Qu, Asem Wardak, Pulin Gong
Deep neural networks (DNNs) have been successfully applied to many real-world problems, but a complete understanding of their dynamical and computational principles is still lacking.
1 code implementation • 22 Sep 2020 • Guozhang Chen, Cheng Kevin Qu, Pulin Gong
The anomalous superdiffusion process during the initial learning phase indicates that the motion of SGD along the loss landscape possesses intermittent, big jumps; this non-equilibrium property enables the SGD to escape from sharp local minima.
