1 code implementation • 28 Nov 2022 • Fengyu Zhang, Ashkan Panahi, Guangjun Gao
By ablation study, we show that low frequency self-attention can achieve very close or better performance relative to full frequency even without retraining the network.