Search Results for author:
Found 7 papers, 5 papers with code
MixSKD: Self-Knowledge Distillation from Mixup for Image Recognition
MixSKD mutually distills feature maps and probability distributions between the random pair of original images and their mixup images in a meaningful way.
Prior Gradient Mask Guided Pruning-Aware Fine-Tuning
In detail, the proposed PGMPF selectively suppresses the gradient of those ”unimportant” parameters via a prior gradient mask generated by the pruning criterion during fine-tuning.
Knowledge Distillation Using Hierarchical Self-Supervision Augmented Distribution
Each auxiliary branch is guided to learn self-supervision augmented task and distill this distribution from teacher to student.
Hierarchical Self-supervised Augmented Knowledge Distillation
We therefore adopt an alternative self-supervised augmented task to guide the network to learn the joint distribution of the original recognition task and self-supervised auxiliary task.
Ranked #20 on
Knowledge Distillation
on ImageNet
Mutual Contrastive Learning for Visual Representation Learning
We present a collaborative learning method called Mutual Contrastive Learning (MCL) for general visual representation learning.
GHFP: Gradually Hard Filter Pruning
Filter pruning is widely used to reduce the computation of deep learning, enabling the deployment of Deep Neural Networks (DNNs) in resource-limited devices.
Softer Pruning, Incremental Regularization
Network pruning is widely used to compress Deep Neural Networks (DNNs).
