Search Results for author:
Found 3 papers, 1 papers with code
Lancet: Accelerating Mixture-of-Experts Training via Whole Graph Computation-Communication Overlapping
The Mixture-of-Expert (MoE) technique plays a crucial role in expanding the size of DNN model parameters.
DynaPipe: Optimizing Multi-task Training through Dynamic Pipelines
This paper proposes a dynamic micro-batching approach to tackle sequence length variation and enable efficient multi-task model training.
dPRO: A Generic Profiling and Optimization System for Expediting Distributed DNN Training
Distributed training using multiple devices (e. g., GPUs) has been widely adopted for learning DNN models over large datasets.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
