Search Results for author:
Found 24 papers, 3 papers with code
Metric-aware LLM inference for regression and scoring
Large language models (LLMs) have demonstrated strong results on a range of NLP tasks.
ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agent
Answering complex natural language questions often necessitates multi-step reasoning and integrating external information.
Ranked #1 on
Question Answering
on Bamboogle
SpecTr: Fast Speculative Decoding via Optimal Transport
We show that the optimal draft selection algorithm (transport plan) can be computed via linear programming, whose best-known runtime is exponential in $k$.
Large Language Models with Controllable Working Memory
By contrast, when the context is irrelevant to the task, the model should ignore it and fall back on its internal knowledge.
Two-stage LLM Fine-tuning with Less Specialization and More Generalization
Pretrained large language models (LLMs) are general purpose problem solvers applicable to a diverse set of tasks with prompts.
The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers
This paper studies the curious phenomenon for machine learning models with Transformer architectures that their activation maps are sparse.
FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning
Federated learning~(FL) has recently attracted increasing attention from academia and industry, with the ultimate goal of achieving collaborative training under privacy and communication constraints.
Correlated quantization for distributed mean estimation and optimization
We show that applying the proposed protocol as sub-routine in distributed optimization algorithms leads to better convergence rates.
HD-cos Networks: Efficient Neural Architectures for Secure Multi-Party Computation
In this paper, we study training and inference of neural networks under the MPC setup.
HD-cos Networks: Efficient Neural Architechtures for Secure Multi-Party Computation
In this paper, we study training and inference of neural networks under the MPC setup.
InFillmore: Frame-Guided Language Generation with Bidirectional Context
We propose a structured extension to bidirectional-context conditional language generation, or "infilling," inspired by Frame Semantic theory (Fillmore, 1976).
Modifying Memories in Transformer Models
In this paper, we propose a new task of \emph{explicitly modifying specific factual knowledge in Transformer models while ensuring the model performance does not degrade on the unmodified facts}.
Semantic Label Smoothing for Sequence to Sequence Problems
Label smoothing has been shown to be an effective regularization strategy in classification, that prevents overfitting and helps in label de-noising.
Learning discrete distributions: user vs item-level privacy
If each user has $m$ samples, we show that straightforward applications of Laplace or Gaussian mechanisms require the number of users to be $\mathcal{O}(k/(m\alpha^2) + k/\epsilon\alpha)$ to achieve an $\ell_1$ distance of $\alpha$ between the true and estimated distributions, with the privacy-induced penalty $k/\epsilon\alpha$ independent of the number of samples per user $m$.
Self-supervised Learning for Large-scale Item Recommendations
Our online results also verify our hypothesis that our framework indeed improves model performance even more on slices that lack supervision.
Doubly-stochastic mining for heterogeneous retrieval
Modern retrieval problems are characterised by training sets with potentially billions of labels, and heterogeneous data distributions across subpopulations (e. g., users of a retrieval system may be from different countries), each of which poses a challenge.
Take the Scenic Route: Improving Generalization in Vision-and-Language Navigation
In the Vision-and-Language Navigation (VLN) task, an agent with egocentric vision navigates to a destination given natural language instructions.
Sampled Softmax with Random Fourier Features
For the settings where a large number of classes are involved, a common method to speed up training is to sample a subset of classes and utilize an estimate of the loss gradient based on these classes, known as the sampled softmax method.
Hyperparameter Optimization in Black-box Image Processing using Differentiable Proxies
We present a fully automatic system to optimize the parameters of black-box hardware and software image processing pipelines according to any arbitrary (i. e., application-specific) metric.
Stochastic Negative Mining for Learning with Large Output Spaces
Furthermore, we identify a particularly intuitive class of loss functions in the aforementioned family and show that they are amenable to practical implementation in the large output space setting (i. e. computation is possible without evaluating scores of all labels) by developing a technique called Stochastic Negative Mining.
Loss Decomposition for Fast Learning in Large Output Spaces
For problems with large output spaces, evaluation of the loss function and its gradient are expensive, typically taking linear time in the size of the output space.
cpSGD: Communication-efficient and differentially-private distributed SGD
Distributed stochastic gradient descent is an important subroutine in distributed learning.
Multiscale Quantization for Fast Similarity Search
We propose a multiscale quantization approach for fast similarity search on large, high-dimensional datasets.
Lattice Rescoring Strategies for Long Short Term Memory Language Models in Speech Recognition
Recurrent neural network (RNN) language models (LMs) and Long Short Term Memory (LSTM) LMs, a variant of RNN LMs, have been shown to outperform traditional N-gram LMs on speech recognition tasks.
