Search Results for author:
Found 8 papers, 3 papers with code
Learning to Project for Cross-Task Knowledge Distillation
In this setting, cross-task distillation can be used, enabling the use of any teacher model trained on a different task.
$V_kD:$ Improving Knowledge Distillation using Orthogonal Projections
Knowledge distillation is an effective method for training small and efficient deep learning models.
Ranked #2 on
Knowledge Distillation
on ImageNet
Understanding the Role of the Projector in Knowledge Distillation
We then show that the normalisation of representations is tightly coupled with the training dynamics of this projector, which can have a large impact on the students performance.
Ranked #4 on
Knowledge Distillation
on ImageNet
MobileVOS: Real-Time Video Object Segmentation Contrastive Learning meets Knowledge Distillation
This paper tackles the problem of semi-supervised video object segmentation on resource-constrained devices, such as mobile phones.
Ranked #6 on
Video Object Segmentation
on YouTube-VOS 2019
Information Theoretic Representation Distillation
Despite the empirical success of knowledge distillation, current state-of-the-art methods are computationally expensive to train, which makes them difficult to adopt in practice.
Classification with Binary Weight Network
Knowledge Distillation
+1
Reconstructing Pruned Filters using Cheap Spatial Transformations
We present an efficient alternative to the convolutional layer using cheap spatial transformations.
Cascaded channel pruning using hierarchical self-distillation
In this paper, we propose an approach for filter-level pruning with hierarchical knowledge distillation based on the teacher, teaching-assistant, and student framework.
Compression of descriptor models for mobile applications
Deep neural networks have demonstrated state-of-the-art performance for feature-based image matching through the advent of new large and diverse datasets.
