Search Results for author:
Found 6 papers, 2 papers with code
SLaM: Student-Label Mixing for Distillation with Unlabeled Examples
Knowledge distillation with unlabeled examples is a powerful training paradigm for generating compact and lightweight student models in applications where the amount of labeled data is limited but one has access to a large pool of unlabeled data.
Weighted Distillation with Unlabeled Examples
Our method is hyper-parameter free, data-agnostic, and simple to implement.
Robust Active Distillation
Distilling knowledge from a large teacher model to a lightweight one is a widely successful approach for generating compact, powerful models in the semi-supervised learning setting where a limited amount of labeled data is available.
Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better
Our hope is that this survey would provide the reader with the mental model and the necessary understanding of the field to apply generic efficiency techniques to immediately get significant improvements, and also equip them with ideas for further research and experimentation to achieve additional gains.
Genome Compression Against a Reference
Being able to store and transmit human genome sequences is an important part in genomic research and industrial applications.
Learning from a Teacher using Unlabeled Data
Knowledge distillation is a widely used technique for model compression.
