Search Results for author:
Found 11 papers, 10 papers with code
Effective Quantization for Diffusion Models on CPUs
Diffusion models have gained popularity for generating images from textual descriptions.
Efficient LLM Inference on CPUs
Large language models (LLMs) have demonstrated remarkable performance and tremendous potential across a wide range of tasks.
TEQ: Trainable Equivalent Transformation for Quantization of LLMs
As large language models (LLMs) become more prevalent, there is a growing need for new and improved quantization methods that can meet the computationalast layer demands of these modern architectures while maintaining the accuracy.
Efficient Post-training Quantization with FP8 Formats
Recent advances in deep learning methods such as LLMs and Diffusion models have created a need for improved quantization methods that can meet the computational demands of these modern architectures while maintaining accuracy.
Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs
As the number of bits decreases, the quantization grid broadens, thus emphasizing the importance of up and down rounding.
An Efficient Sparse Inference Software Accelerator for Transformer-based Language Models on CPUs
We apply our sparse accelerator on widely-used Transformer-based language models including Bert-Mini, DistilBERT, Bert-Base, and BERT-Large.
QuaLA-MiniLM: a Quantized Length Adaptive MiniLM
Our quantized length-adaptive MiniLM model (QuaLA-MiniLM) is trained only once, dynamically fits any inference scenario, and achieves an accuracy-efficiency trade-off superior to any other efficient approaches per any computational budget on the SQuAD1. 1 dataset (up to x8. 8 speedup with <1% accuracy loss).
Fast DistilBERT on CPUs
In this work, we propose a new pipeline for creating and running Fast Transformer models on CPUs, utilizing hardware-aware pruning, knowledge distillation, quantization, and our own Transformer inference runtime engine with optimized kernels for sparse and quantized operators.
Prune Once for All: Sparse Pre-Trained Language Models
We show how the compressed sparse pre-trained models we trained transfer their knowledge to five different downstream natural language tasks with minimal accuracy loss.
Ranked #2 on
Natural Language Inference
on MultiNLI Dev
HIGHLY EFFICIENT 8-BIT LOW PRECISION INFERENCE OF CONVOLUTIONAL NEURAL NETWORKS
High throughput and low latency inference of deep neural networks are critical for the deployment of deep learning applications.
Highly Efficient 8-bit Low Precision Inference of Convolutional Neural Networks with IntelCaffe
High throughput and low latency inference of deep neural networks are critical for the deployment of deep learning applications.
