A Multi-Level Superoptimizer for Tensor Programs

1 code implementation • 9 May 2024 • Mengdi Wu, Xinhao Cheng, Oded Padon, Zhihao Jia

We introduce Mirage, the first multi-level superoptimizer for tensor programs.

FlexLLM: A System for Co-Serving Large Language Model Inference and Parameter-Efficient Finetuning

1 code implementation • 29 Feb 2024 • Xupeng Miao, Gabriele Oliaro, Xinhao Cheng, Mengdi Wu, Colin Unger, Zhihao Jia

This is because existing systems cannot handle workloads that include a mix of inference and PEFT finetuning requests.

Language Modelling Large Language Model

Finding the Task-Optimal Low-Bit Sub-Distribution in Deep Neural Networks

1 code implementation • 30 Dec 2021 • Runpei Dong, Zhanhong Tan, Mengdi Wu, Linfeng Zhang, Kaisheng Ma

Besides, an efficient deployment flow for the mobile CPU is developed, achieving up to 7. 46$\times$ inference acceleration on an octa-core ARM CPU.

Image Classification Model Compression +3