Search Results for author:
Found 6 papers, 4 papers with code
TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding
However, key-value (KV) cache, which is stored to avoid re-computation, has emerged as a critical bottleneck by growing linearly in size with the sequence length.
Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding
This paper introduces Sequoia, a scalable, robust, and hardware-aware algorithm for speculative decoding.
GNNPipe: Scaling Deep GNN Training with Pipelined Model Parallelism
Communication is a key bottleneck for distributed graph neural network (GNN) training.
SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
Our evaluation shows that SpecInfer outperforms existing LLM serving systems by 1. 5-2. 8x for distributed LLM inference and by 2. 6-3. 5x for offloading-based LLM inference, while preserving the same generative performance.
Quark: A Gradient-Free Quantum Learning Framework for Classification Tasks
To address the limitations of existing quantum ML methods, we introduce Quark, a gradient-free quantum learning framework that optimizes quantum ML models using quantum optimization.
Quantized Training of Gradient Boosting Decision Trees
Recent years have witnessed significant success in Gradient Boosting Decision Trees (GBDT) for a wide range of machine learning applications.
