Search Results for author:
Found 25 papers, 12 papers with code
LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens
This is achieved by three key innovations: (i) we identify and exploit two forms of non-uniformities in positional interpolation through an efficient search, providing a better initialization for fine-tuning and enabling an 8x extension in non-fine-tuning scenarios; (ii) we introduce a progressive extension strategy that first fine-tunes a 256k length LLM and then conducts a second positional interpolation on the fine-tuned extended LLM to achieve a 2048k context window; (iii) we readjust LongRoPE on 8k length to recover the short context window performance.
Fewer is More: Boosting LLM Reasoning with Reinforced Context Pruning
In this work, we propose CoT-Influx, a novel approach that pushes the boundary of few-shot Chain-of-Thoughts (CoT) learning to improve LLM mathematical reasoning.
Ranked #105 on
Arithmetic Reasoning
on GSM8K
Compresso: Structured Pruning with Collaborative Prompting Learns Compact Large Language Models
To this end, Compresso prunes LLaMA-7B to 5. 4B, maintaining original performance and even surpassing LLaMA-7B in reading comprehension by 2. 62%.
Model-enhanced Vector Index
We empirically show that our model achieves better performance on the commonly used academic benchmarks MSMARCO Passage and Natural Questions, with comparable serving latency to dense retrieval solutions.
Accelerating In-Browser Deep Learning Inference on Diverse Edge Clients through Just-in-Time Kernel Optimizations
Web applications are increasingly becoming the primary platform for AI service delivery, making in-browser deep learning (DL) inference more prominent.
Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference
To tackle the high compute requirements of LLMs, the Mixture-of-Experts (MoE) architecture was introduced which is able to scale its model size without proportionally scaling up its computational requirements.
Constraint-aware and Ranking-distilled Token Pruning for Efficient Transformer Inference
Deploying pre-trained transformer models like BERT on downstream tasks in resource-constrained scenarios is challenging due to their high inference cost, which grows rapidly with input sequence length.
Accurate and Structured Pruning for Efficient Automatic Speech Recognition
In this paper, we propose a novel compression strategy that leverages structured pruning and knowledge distillation to reduce the model size and inference cost of the Conformer model while preserving high recognition performance.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Integer or Floating Point? New Outlooks for Low-Bit Quantization on Large Language Models
In this study, we conduct a comparative analysis of INT and FP quantization with the same bit-width, revealing that the optimal quantization format varies across different layers due to the complexity and diversity of tensor distribution.
IRGen: Generative Modeling for Image Retrieval
While generative modeling has been ubiquitous in natural language processing and computer vision, its application to image retrieval remains unexplored.
ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices
However, prior supernet training methods that rely on uniform sampling suffer from the gradient conflict issue: the sampled subnets can have vastly different model sizes (e. g., 50M vs. 2G FLOPs), leading to different optimization directions and inferior performance.
SpaceEvo: Hardware-Friendly Search Space Design for Efficient INT8 Inference
The combination of Neural Architecture Search (NAS) and quantization has proven successful in automatically designing low-FLOPs INT8 quantized neural networks (QNN).
LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookup
On-device Deep Neural Network (DNN) inference consumes significant computing resources and development efforts.
PIT: Optimization of Dynamic Sparse Deep Learning Models via Permutation Invariant Transformation
Dynamic sparsity, where the sparsity patterns are unknown until runtime, poses a significant challenge to deep learning.
SuperScaler: Supporting Flexible DNN Parallelization via a Unified Abstraction
SuperScaler is a system that facilitates the design and generation of highly flexible parallelization plans.
SwiftPruner: Reinforced Evolutionary Pruning for Efficient Ad Relevance
Remarkably, under our latency requirement of 1900us on CPU, SwiftPruner achieves a 0. 86% higher AUC than the state-of-the-art uniform sparse baseline for BERT-Mini on a large scale real-world dataset.
LordNet: Learning to Solve Parametric Partial Differential Equations without Simulated Data
Neural operators, as a powerful approximation to the non-linear operators between infinite-dimensional function spaces, have proved to be promising in accelerating the solution of partial differential equations (PDE).
Tutel: Adaptive Mixture-of-Experts at Scale
On efficiency, Flex accelerates SwinV2-MoE, achieving up to 1. 55x and 2. 11x speedup in training and inference over Fairseq, respectively.
A Neural Corpus Indexer for Document Retrieval
To this end, we propose Neural Corpus Indexer (NCI), a sequence-to-sequence network that generates relevant document identifiers directly for a designated query.
SPANN: Highly-efficient Billion-scale Approximate Nearest Neighbor Search
It stores the centroid points of the posting lists in the memory and the large posting lists in the disk.
WRENCH: A Comprehensive Benchmark for Weak Supervision
To address these problems, we introduce a benchmark platform, WRENCH, for thorough and standardized evaluation of WS approaches.
SPANN: Highly-efficient Billion-scale Approximate Nearest Neighborhood Search
It stores the centroid points of the posting lists in the memory and the large posting lists in the disk.
OpEvo: An Evolutionary Method for Tensor Operator Optimization
Training and inference efficiency of deep neural networks highly rely on the performance of tensor operators on hardware platforms.
TextNAS: A Neural Architecture Search Space tailored for Text Representation
Learning text representation is crucial for text classification and other language related tasks.
Time-Series Anomaly Detection Service at Microsoft
At Microsoft, we develop a time-series anomaly detection service which helps customers to monitor the time-series continuously and alert for potential incidents on time.
