no code implementations • 29 Mar 2021 • Yucong Zhou, Yunxiao Sun, Zhao Zhong
Based on this discovery, we propose a new training method called FixNorm, which discards weight decay and directly controls the two mechanisms.
