Search Results for author:
Found 5 papers, 1 papers with code
QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference
We introduce QUICK, a group of novel optimized CUDA kernels for the efficient inference of quantized Large Language Models (LLMs).
Squeezing Large-Scale Diffusion Models for Mobile
The emergence of diffusion models has greatly broadened the scope of high-fidelity image synthesis, resulting in notable advancements in both practical implementation and academic research.
Temporal Dynamic Quantization for Diffusion Models
The diffusion model has gained popularity in vision applications due to its remarkable generative performance and versatility.
Double Viterbi: Weight Encoding for High Compression Ratio and Fast On-Chip Reconstruction for Deep Neural Network
In this paper, we propose a new sparse matrix format in order to enable a highly parallel decoding process of the entire sparse matrix.
Viterbi-based Pruning for Sparse Matrix with Fixed and High Index Compression Ratio
Hence, pruning is usually restricted to inference with a batch size of one, for which an efficient parallel matrix-vector multiplication method exists.
