Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs

1 code implementation • 16 Feb 2024 • Yeonhong Park, Jake Hyun, SangLyul Cho, Bonggeun Sim, Jae W. Lee

Recently, considerable efforts have been directed towards compressing Large Language Models (LLMs), which showcase groundbreaking capabilities across diverse applications but entail significant deployment costs due to their large sizes.

Quantization