no code implementations • 6 Apr 2023 • Xuanzhe Xiao, Zeng Li, Chuanlong Xie, Fengwei Zhou
To capitalize on this discovery, we introduce a novel regularization technique, termed Heavy-Tailed Regularization, which explicitly promotes a more heavy-tailed spectrum in the weight matrix through regularization.
