Search Results for author:
Found 14 papers, 2 papers with code
Nested-TNT: Hierarchical Vision Transformers with Multi-Scale Feature Processing
ViT divides an image into several local patches, known as "visual sentences".
DMT: Comprehensive Distillation with Multiple Self-supervised Teachers
Numerous self-supervised learning paradigms, such as contrastive learning and masked image modeling, have been proposed to acquire powerful and general representations from unlabeled data.
Language-guided Few-shot Semantic Segmentation
Few-shot learning is a promising way for reducing the label cost in new categories adaptation with the guidance of a small, well labeled support set.
ES-MVSNet: Efficient Framework for End-to-end Self-supervised Multi-View Stereo
to provide additional consistency constraints, which grows GPU memory consumption and complicates the model's structure and training pipeline.
Dynamic Token-Pass Transformers for Semantic Segmentation
Vision transformers (ViT) usually extract features via forwarding all the tokens in the self-attention layers from top to toe.
LMSeg: Language-guided Multi-dataset Segmentation
Instead of relabeling each dataset with the unified taxonomy, a category-guided decoding module is designed to dynamically guide predictions to each datasets taxonomy.
Nebula-I: A General Framework for Collaboratively Training Deep Learning Models on Low-Bandwidth Cloud Clusters
We took natural language processing (NLP) as an example to show how Nebula-I works in different training phases that include: a) pre-training a multilingual language model using two remote clusters; and b) fine-tuning a machine translation model using knowledge distilled from pre-trained models, which run through the most popular paradigm of recent deep learning.
Cross-Lingual Natural Language Inference
Distributed Computing
+2
Data-Free Knowledge Transfer: A Survey
In this paper, we provide a comprehensive survey on data-free knowledge transfer from the perspectives of knowledge distillation and unsupervised domain adaptation, to help readers have a better understanding of the current research status and ideas.
Dynamically Stable Poincaré Embeddings for Neural Manifolds
As a result, we prove that, if initial metrics have an $L^2$-norm perturbation which deviates from the Hyperbolic metric on the Poincar\'e ball, the scaled Ricci-DeTurck flow of such metrics smoothly and exponentially converges to the Hyperbolic metric.
Riemannian Manifold Embeddings for Straight-Through Estimator
Quantized Neural Networks (QNNs) aim at replacing full-precision weights $\boldsymbol{W}$ with quantized weights $\boldsymbol{\hat{W}}$, which make it possible to deploy large models to mobile and miniaturized devices easily.
Source-Free Domain Adaptation for Semantic Segmentation
To cope with this issue, we propose a source-free domain adaptation framework for semantic segmentation, namely SFDA, in which only a well-trained source model and an unlabeled target domain dataset are available for adaptation.
Zero-shot Adversarial Quantization
To address the above issues, we propose a zero-shot adversarial quantization (ZAQ) framework, facilitating effective discrepancy estimation and knowledge transfer from a full-precision model to its quantized model.
Ranked #2 on
Data Free Quantization
on CIFAR-100
(CIFAR-100 W5A5 Top-1 Accuracy metric)
Adaptive Multi-Teacher Multi-level Knowledge Distillation
Knowledge distillation~(KD) is an effective learning paradigm for improving the performance of lightweight student networks by utilizing additional supervision knowledge distilled from teacher networks.
Learning from a Lightweight Teacher for Efficient Knowledge Distillation
Knowledge Distillation (KD) is an effective framework for compressing deep learning models, realized by a student-teacher paradigm requiring small student networks to mimic the soft target generated by well-trained teachers.
