Search Results for author:
Found 3 papers, 2 papers with code
FlexLLM: A System for Co-Serving Large Language Model Inference and Parameter-Efficient Finetuning
This is because existing systems cannot handle workloads that include a mix of inference and PEFT finetuning requests.
Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems
In the rapidly evolving landscape of artificial intelligence (AI), generative large language models (LLMs) stand at the forefront, revolutionizing how we interact with our data.
SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
Our evaluation shows that SpecInfer outperforms existing LLM serving systems by 1. 5-2. 8x for distributed LLM inference and by 2. 6-3. 5x for offloading-based LLM inference, while preserving the same generative performance.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
