Search Results for author:
Found 13 papers, 2 papers with code
Large Language Model Adaptation for Networking
In this paper, we present NetLLM, the first LLM adaptation framework that efficiently adapts LLMs to solve networking problems.
Chatterbox: Robust Transport for LLM Token Streaming under Unstable Network
To render each generated token in real time, the LLM server generates response tokens one by one and streams each generated token (or group of a few tokens) through the network to the user right after it is generated, which we refer to as LLM token streaming.
CacheGen: KV Cache Compression and Streaming for Fast Language Model Serving
Compared to the recent systems that reuse the KV cache, CacheGen reduces the KV cache size by 3. 7-4. 3x and the total delay in fetching and processing contexts by 2. 7-3. 2x while having negligible impact on the LLM response quality in accuracy or perplexity.
Automatic and Efficient Customization of Neural Networks for ML Applications
ML APIs have greatly relieved application developers of the burden to design and train their own neural network models -- classifying objects in an image can now be as simple as one line of Python code to call an API.
OneAdapt: Fast Adaptation for Deep Learning Applications via Backpropagation
While the high demand for network bandwidth and GPU resources could be substantially reduced by optimally adapting the configuration knobs, such as video resolution and frame rate, current adaptation techniques fail to meet three requirements simultaneously: adapt configurations (i) with minimum extra GPU or bandwidth overhead; (ii) to reach near-optimal decisions based on how the data affects the final DNN's accuracy, and (iii) do so for a range of configuration knobs.
GRACE: Loss-Resilient Real-Time Video through Neural Codecs
In real-time video communication, retransmitting lost packets over high-latency networks is not viable due to strict latency requirements.
AccMPEG: Optimizing Video Encoding for Video Analytics
This paper presents AccMPEG, a new video encoding and streaming system that meets all the three requirements.
Sayer: Using Implicit Feedback to Optimize System Policies
We develop a methodology, called Sayer, that leverages implicit feedback to evaluate and train new system policies.
Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers
Compressed models that are deployed on the edge servers for inference suffer from data drift, where the live video data diverges from the training data.
Domain-specific Communication Optimization for Distributed DNN Training
Communication overhead poses an important obstacle to distributed DNN training and draws increasing attention in recent years.
ReXCam: Resource-Efficient, Cross-Camera Video Analytics at Scale
Enterprises are increasingly deploying large camera networks for video analytics.
Addressing Training Bias via Automated Image Annotation
Build accurate DNN models requires training on large labeled, context specific datasets, especially those matching the target scenario.
Scaling Video Analytics Systems to Large Camera Deployments
Driven by advances in computer vision and the falling costs of camera hardware, organizations are deploying video cameras en masse for the spatial monitoring of their physical premises.
