Search Results for author:
Found 11 papers, 9 papers with code
Effective Whole-body Pose Estimation with Two-stages Distillation
Different from the previous self-knowledge distillation, this stage finetunes the student's head with only 20% training time as a plug-and-play training strategy.
Ranked #1 on
2D Human Pose Estimation
on COCO-WholeBody
(using extra training data)
CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers
Although extensively studied for unimodal models, the acceleration for multimodal models, especially the vision-language Transformers, is relatively under-explored.
From Knowledge Distillation to Self-Knowledge Distillation: A Unified Approach with Normalized Loss and Customized Soft Labels
We decompose the KD loss and find the non-target loss from it forces the student's non-target logits to match the teacher's, but the sum of the two non-target logits is different, preventing them from being identical.
UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers
Real-world data contains a vast amount of multimodal information, among which vision and language are the two most representative modalities.
Accurate 3D Face Reconstruction with Facial Component Tokens
However, the current deep learning-based methods face significant challenges in achieving accurate reconstruction with disentangled facial parameters and ensuring temporal stability in single-frame methods for 3D face tracking on video data.
ViTKD: Practical Guidelines for ViT feature knowledge distillation
In this paper, we explore the way of feature-based distillation for ViT.
Rethinking Knowledge Distillation via Cross-Entropy
Furthermore, we smooth students' target output to treat it as the soft target for training without teachers and propose a teacher-free new KD loss (tf-NKD).
Masked Generative Distillation
The current distillation algorithm usually improves students' performance by imitating the output of the teacher.
Transparent Shape from a Single View Polarization Image
To train and test our method, we construct a dataset for transparent shape from polarization with paired polarization images and ground-truth normal maps.
Focal and Global Knowledge Distillation for Detectors
Global distillation rebuilds the relation between different pixels and transfers it from teachers to students, compensating for missing global information in focal distillation.
Ranked #1 on
Knowledge Distillation
on MS COCO
A Sentiment-Controllable Topic-to-Essay Generator with Topic Knowledge Graph
Therefore, we propose a novel Sentiment-Controllable topic-to-essay generator with a Topic Knowledge Graph enhanced decoder, named SCTKG, which is based on the conditional variational autoencoder (CVAE) framework.
