Make A Long Image Short: Adaptive Token Length for Vision Transformers

no code implementations • 5 Jul 2023 • Qiqi Zhou, Yichen Zhu

The TLA enables ReViT to process images with the minimum sufficient number of tokens, reducing token numbers in the ViT model and improving inference speed.

Action Recognition Image Classification

ScaleKD: Distilling Scale-Aware Knowledge in Small Object Detector

no code implementations • CVPR 2023 • Yichen Zhu, Qiqi Zhou, Ning Liu, Zhiyuan Xu, Zhicai Ou, Xiaofeng Mou, Jian Tang

Unlike existing works that struggle to balance the trade-off between inference speed and SOD performance, in this paper, we propose a novel Scale-aware Knowledge Distillation (ScaleKD), which transfers knowledge of a complex teacher model to a compact student model.

Knowledge Distillation object-detection +2