Search Results for author:
Found 9 papers, 2 papers with code
Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
However, batching multiple requests leads to an interleaving of prefill and decode iterations which makes it challenging to achieve both high throughput and low latency.
SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
SARATHI employs chunked-prefills, which splits a prefill request into equal sized chunks, and decode-maximal batching, which constructs a batch using a single prefill chunk and populates the remaining slots with decodes.
NGAME: Negative Mining-aware Mini-batching for Extreme Classification
This paper identifies that memory overheads of popular negative mining techniques often force mini-batch sizes to remain small and slow training down.
Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads
At the heart of Singularity is a novel, workload-aware scheduler that can transparently preempt and elastically scale deep learning workloads to drive high utilization without impacting their correctness or performance, across a global fleet of AI accelerators (e. g., GPUs, FPGAs).
LRTuner: A Learning Rate Tuner for Deep Neural Networks
For example on ImageNet with Resnet-50, LRTuner shows up to 0. 2% absolute gains in test accuracy compared to the hand-tuned baseline schedule.
Unsupervised Clustering using Pseudo-semi-supervised Learning
In this paper, we propose a framework that leverages semi-supervised models to improve unsupervised clustering performance.
Wide-minima Density Hypothesis and the Explore-Exploit Learning Rate Schedule
Several papers argue that wide minima generalize better than narrow minima.
Ranked #6 on
Machine Translation
on WMT2014 German-English
AutoLR: A Method for Automatic Tuning of Learning Rate
We compare favorably against state of the art learning rate schedules for the given dataset and models, including for ImageNet on Resnet-50, Cifar-10 on Resnet-18, and SQuAD fine-tuning on BERT.
Privado: Practical and Secure DNN Inference with Enclaves
In this paper, we ask a timely question: "Can third-party cloud services use Intel SGX enclaves to provide practical, yet secure DNN Inference-as-a-service?"
