Distilling Global and Local Logits With Densely Connected Relations

1 code implementation • ICCV 2021 • Youmin Kim, Jinbae Park, YounHo Jang, Muhammad Ali, Tae-Hyun Oh, Sung-Ho Bae

In prevalent knowledge distillation, logits in most image recognition models are computed by global average pooling, then used to learn to encode the high-level and task-relevant knowledge.

Image Classification Knowledge Distillation +3