Search Results for author:
Found 4 papers, 0 papers with code
LayerNAS: Neural Architecture Search in Polynomial Complexity
For a model architecture with $L$ layers, we perform layerwise-search for each layer, selecting from a set of search options $\mathbb{S}$.
SLaM: Student-Label Mixing for Distillation with Unlabeled Examples
Knowledge distillation with unlabeled examples is a powerful training paradigm for generating compact and lightweight student models in applications where the amount of labeled data is limited but one has access to a large pool of unlabeled data.
Weighted Distillation with Unlabeled Examples
Our method is hyper-parameter free, data-agnostic, and simple to implement.
Robust Active Distillation
Distilling knowledge from a large teacher model to a lightweight one is a widely successful approach for generating compact, powerful models in the semi-supervised learning setting where a limited amount of labeled data is available.
