1 code implementation • 29 Jul 2023 • Zhendong Yang, Ailing Zeng, Chun Yuan, Yu Li
Different from the previous self-knowledge distillation, this stage finetunes the student's head with only 20% training time as a plug-and-play training strategy.
Ranked #1 on 2D Human Pose Estimation on COCO-WholeBody (using extra training data)
1 code implementation • 27 May 2023 • Dachuan Shi, Chaofan Tao, Anyi Rao, Zhendong Yang, Chun Yuan, Jiaqi Wang
Although extensively studied for unimodal models, the acceleration for multimodal models, especially the vision-language Transformers, is relatively under-explored.
1 code implementation • ICCV 2023 • Zhendong Yang, Ailing Zeng, Zhe Li, Tianke Zhang, Chun Yuan, Yu Li
We decompose the KD loss and find the non-target loss from it forces the student's non-target logits to match the teacher's, but the sum of the two non-target logits is different, preventing them from being identical.
1 code implementation • 31 Jan 2023 • Dachuan Shi, Chaofan Tao, Ying Jin, Zhendong Yang, Chun Yuan, Jiaqi Wang
Real-world data contains a vast amount of multimodal information, among which vision and language are the two most representative modalities.
no code implementations • ICCV 2023 • Tianke Zhang, Xuangeng Chu, Yunfei Liu, Lijian Lin, Zhendong Yang, Zhengzhuo Xu, Chengkun Cao, Fei Yu, Changyin Zhou, Chun Yuan, Yu Li
However, the current deep learning-based methods face significant challenges in achieving accurate reconstruction with disentangled facial parameters and ensuring temporal stability in single-frame methods for 3D face tracking on video data.
1 code implementation • 6 Sep 2022 • Zhendong Yang, Zhe Li, Ailing Zeng, Zexian Li, Chun Yuan, Yu Li
In this paper, we explore the way of feature-based distillation for ViT.
1 code implementation • 22 Aug 2022 • Zhendong Yang, Zhe Li, Yuan Gong, Tianke Zhang, Shanshan Lao, Chun Yuan, Yu Li
Furthermore, we smooth students' target output to treat it as the soft target for training without teachers and propose a teacher-free new KD loss (tf-NKD).
3 code implementations • 3 May 2022 • Zhendong Yang, Zhe Li, Mingqi Shao, Dachuan Shi, Zehuan Yuan, Chun Yuan
The current distillation algorithm usually improves students' performance by imitating the output of the teacher.
1 code implementation • ICCV 2023 • Mingqi Shao, Chongkun Xia, Zhendong Yang, Junnan Huang, Xueqian Wang
To train and test our method, we construct a dataset for transparent shape from polarization with paired polarization images and ground-truth normal maps.
1 code implementation • CVPR 2022 • Zhendong Yang, Zhe Li, Xiaohu Jiang, Yuan Gong, Zehuan Yuan, Danpei Zhao, Chun Yuan
Global distillation rebuilds the relation between different pixels and transfers it from teachers to students, compensating for missing global information in focal distillation.
Ranked #1 on Knowledge Distillation on MS COCO
no code implementations • Findings of the Association for Computational Linguistics 2020 • Lin Qiao, Jianhao Yan, Fandong Meng, Zhendong Yang, Jie zhou
Therefore, we propose a novel Sentiment-Controllable topic-to-essay generator with a Topic Knowledge Graph enhanced decoder, named SCTKG, which is based on the conditional variational autoencoder (CVAE) framework.