Search Results for author: Chengbo Liu

Found 1 papers, 1 papers with code

SDSAT: Accelerating LLM Inference through Speculative Decoding with Semantic Adaptive Tokens

1 code implementation27 Mar 2024 Chengbo Liu, Yong Zhu

The core strategies involve: 1) Fine-tune the model by incorporating semantic adaptive tokens that possess flexible decoding capabilities without changing its structure, allowing them to generate high-quality draft tokens.

Cannot find the paper you are looking for? You can Submit a new open access paper.