Search Results for author:
Found 5 papers, 3 papers with code
Orca 2: Teaching Small Language Models How to Reason
Research on training small LMs has often relied on imitation learning to replicate the output of more capable models.
Ranked #1 on
Crass AI
on BIG-bench
SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inference
While existing token-level early exit methods show promising results for online inference, they cannot be readily applied for batch inferencing and Key-Value caching.
Orca: Progressive Learning from Complex Explanation Traces of GPT-4
To address these challenges, we develop Orca (We are working with our legal team to publicly release a diff of the model weights in accordance with LLaMA's release policy to be published at https://aka. ms/orca-lm), a 13-billion parameter model that learns to imitate the reasoning process of LFMs.
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning
Standard fine-tuning of large pre-trained language models (PLMs) for downstream tasks requires updating hundreds of millions to billions of parameters, and storing a large copy of the PLM weights for every task resulting in increased cost for storing, sharing and serving the models.
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning
Standard fine-tuning of large pre-trained language models (PLMs) for downstream tasks requires updating hundreds of millions to billions of parameters, and storing a large copy of the PLM weights for every task resulting in increased cost for storing, sharing and serving the models.
