Search Results for author:
Found 7 papers, 4 papers with code
Transformers Can Achieve Length Generalization But Not Robustly
We show that the success of length generalization is intricately linked to the data format and the type of position encoding.
DistillSpec: Improving Speculative Decoding via Knowledge Distillation
Finally, in practical scenarios with models of varying sizes, first using distillation to boost the performance of the target model and then applying DistillSpec to train a well-aligned draft model can reduce decoding latency by 6-10x with minimal performance drop, compared to standard decoding without distillation.
Identifying the Risks of LM Agents with an LM-Emulated Sandbox
Alongside the emulator, we develop an LM-based automatic safety evaluator that examines agent failures and quantifies associated risks.
On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
Instead of solely relying on a fixed set of output sequences, GKD trains the student on its self-generated output sequences by leveraging feedback from the teacher on such sequences.
Training on Thin Air: Improve Image Classification with Generated Data
In this paper, we present Diffusion Inversion, a simple yet effective method that leverages the pre-trained generative model, Stable Diffusion, to generate diverse, high-quality training data for image classification.
Large Language Models Are Human-Level Prompt Engineers
By conditioning on natural language instructions, large language models (LLMs) have displayed impressive capabilities as general-purpose computers.
Dataset Distillation using Neural Feature Regression
Dataset distillation can be formulated as a bi-level meta-learning problem where the outer loop optimizes the meta-dataset and the inner loop trains a model on the distilled data.
