Factorization Vision Transformer: Modeling Long Range Dependency with Local Window Cost

1 code implementation • 14 Dec 2023 • Haolin Qin, Daquan Zhou, Tingfa Xu, Ziyang Bian, Jianan Li

Accordingly, we propose a novel factorization self-attention mechanism (FaSA) that enjoys both the advantages of local window cost and long-range dependency modeling capability.