Bridging the Gap Between Vision Transformers and Convolutional Neural Networks on Small Datasets

1 code implementation • 12 Oct 2022 • Zhiying Lu, Hongtao Xie, Chuanbin Liu, Yongdong Zhang

On channel aspect, we introduce a dynamic feature aggregation module in MLP and a brand new "head token" design in multi-head self-attention module to help re-calibrate channel representation and make different channel group representation interacts with each other.

Inductive Bias