Search Results for author:
Found 25 papers, 11 papers with code
FlexLLM: A System for Co-Serving Large Language Model Inference and Parameter-Efficient Finetuning
This is because existing systems cannot handle workloads that include a mix of inference and PEFT finetuning requests.
Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding
This paper introduces Sequoia, a scalable, robust, and hardware-aware algorithm for speculative decoding.
Accelerating Retrieval-Augmented Language Model Serving with Speculation
Retrieval-augmented language models (RaLM) have demonstrated the potential to solve knowledge-intensive natural language processing (NLP) tasks by combining a non-parametric knowledge base with a parametric language model.
Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models
Experiments show that QST can reduce the total memory footprint by up to 2. 3 $\times$ and speed up the finetuning process by up to 3 $\times$ while achieving competent performance compared with the state-of-the-art.
Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems
In the rapidly evolving landscape of artificial intelligence (AI), generative large language models (LLMs) stand at the forefront, revolutionizing how we interact with our data.
SpotServe: Serving Generative Large Language Models on Preemptible Instances
This paper aims to reduce the monetary cost for serving LLMs by leveraging preemptible GPU instances on modern clouds, which offer accesses to spare GPUs at a much cheaper price than regular instances but may be preempted by the cloud at any time.
Drone-NeRF: Efficient NeRF Based 3D Scene Reconstruction for Large-Scale Drone Survey
In this work, we propose the Drone-NeRF framework to enhance the efficient reconstruction of unbounded large-scale scenes suited for drone oblique photography using Neural Radiance Fields (NeRF).
Quarl: A Learning-Based Quantum Circuit Optimizer
Applying reinforcement learning (RL) to quantum circuit optimization raises two main challenges: the large and varying action space and the non-uniform state representation.
SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
Our evaluation shows that SpecInfer outperforms existing LLM serving systems by 1. 5-2. 8x for distributed LLM inference and by 2. 6-3. 5x for offloading-based LLM inference, while preserving the same generative performance.
Quark: A Gradient-Free Quantum Learning Framework for Classification Tasks
To address the limitations of existing quantum ML methods, we introduce Quark, a gradient-free quantum learning framework that optimizes quantum ML models using quantum optimization.
OLLIE: Derivation-based Tensor Program Optimizer
Boosting the runtime performance of deep neural networks (DNNs) is critical due to their wide adoption in real-world tasks.
BOND: Benchmarking Unsupervised Outlier Node Detection on Static Attributed Graphs
To bridge this gap, we present--to the best of our knowledge--the first comprehensive benchmark for unsupervised outlier node detection on static attributed graphs called BOND, with the following highlights.
Optimizing Mixture of Experts using Dynamic Recompilations
The Mixture of Experts architecture allows for outrageously large neural networks by scaling model parameter size independently from computational demand (FLOPs).
Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNs
DNN models across many domains continue to grow in size, resulting in high resource requirements for effective training, and unpalatable (and often unaffordable) costs for organizations and research labs across scales.
Collage: Seamless Integration of Deep Learning Backends with Automatic Placement
Given the fast-evolving nature of the DL ecosystem, this manual approach often slows down continuous innovations across different layers; it prevents hardware vendors from the fast deployment of their cutting-edge libraries, DL framework developers must repeatedly adjust their hand-coded rules to accommodate new versions of libraries, and machine learning practitioners need to wait for the integration of new technologies and often encounter unsatisfactory performance.
TOD: GPU-accelerated Outlier Detection via Tensor Operations
Outlier detection (OD) is a key learning task for finding rare and deviant data samples, with many time-critical applications such as fraud detection and intrusion detection.
GradSign: Model Performance Inference with Theoretical Insights
In addition, we design GradSign, an accurate and simple approximation of {\Psi} using the gradients of a network evaluated at a random initialization state.
Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads
Computation separation makes it possible to construct a deep, bounded-asynchronous pipeline where graph and tensor parallel tasks can fully overlap, effectively hiding the network latency incurred by Lambdas.
Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models
Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers.
IOS: Inter-Operator Scheduler for CNN Acceleration
To accelerate CNN inference, existing deep learning frameworks focus on optimizing intra-operator parallelization.
Redundancy-Free Computation Graphs for Graph Neural Networks
Graph Neural Networks (GNNs) are based on repeated aggregations of information across nodes' neighbors in a graph.
Beyond Data and Model Parallelism for Deep Neural Networks
We also propose FlexFlow, a deep learning framework that uses guided randomized search of the SOAP space to find a fast parallelization strategy for a specific parallel machine.
Distributed, Parallel, and Cluster Computing
Exploring Hidden Dimensions in Accelerating Convolutional Neural Networks
The past few years have witnessed growth in the computational requirements for training deep convolutional neural networks.
Exploring Hidden Dimensions in Parallelizing Convolutional Neural Networks
The past few years have witnessed growth in the computational requirements for training deep convolutional neural networks.
Exploring the Hidden Dimension in Accelerating Convolutional Neural Networks
DeePa is a deep learning framework that explores parallelism in all parallelizable dimensions to accelerate the training process of convolutional neural networks.
