Search Results for author:
Found 11 papers, 7 papers with code
GliDe with a CaPE: A Low-Hassle Method to Accelerate Speculative Decoding
Speculative decoding is a relatively new decoding framework that leverages small and efficient draft models to reduce the latency of LLMs.
Colossal-Auto: Unified Automation of Parallelization and Activation Checkpoint for Large-scale Models
To address these challenges, we introduce a system that can jointly optimize distributed execution and gradient checkpointing plans.
Elixir: Train a Large Language Model on a Small GPU Cluster
To reduce GPU memory usage, memory partitioning, and memory offloading have been proposed.
EnergonAI: An Inference System for 10-100 Billion Parameter Transformer Models
Although the AI community has expanded the model scale to the trillion parameter level, the practical deployment of 10-100 billion parameter models is still uncertain due to the latency, throughput, and memory constraints.
A Frequency-aware Software Cache for Large Recommendation System Embeddings
Deep learning recommendation models (DLRMs) have been widely applied in Internet companies.
Sky Computing: Accelerating Geo-distributed Computing in Federated Learning
In this paper, we proposed Sky Computing, a load-balanced model parallelism framework to adaptively allocate the weights to devices.
Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Training
The success of Transformer models has pushed the deep learning model scale to billions of parameters.
PatrickStar: Parallel Training of Pre-trained Models via Chunk-based Memory Management
PatrickStar uses the CPU-GPU heterogeneous memory space to store the model data.
Online Evolutionary Batch Size Orchestration for Scheduling Deep Learning Workloads in GPU Clusters
ONES automatically manages the elasticity of each job based on the training batch size, so as to maximize GPU utilization and improve scheduling efficiency.
Sequence Parallelism: Long Sequence Training from System Perspective
That is, with sparse attention, our sequence parallelism enables us to train transformer with infinite long sequence.
An Efficient 2D Method for Training Super-Large Deep Learning Models
However, due to memory constraints, model parallelism must be utilized to host large models that would otherwise not fit into the memory of a single device.
