Search Results for author:
Found 15 papers, 11 papers with code
Reliable, Adaptable, and Attributable Language Models with Retrieval
Parametric language models (LMs), which are trained on vast amounts of web data, exhibit remarkable flexibility and capability.
REST: Retrieval-Based Speculative Decoding
We introduce Retrieval-Based Speculative Decoding (REST), a novel algorithm designed to speed up language model generation.
Poisoning Retrieval Corpora by Injecting Adversarial Passages
Dense retrievers have achieved state-of-the-art performance in various information retrieval tasks, but to what extent can they be safely deployed in real-world applications?
Privacy Implications of Retrieval-Based Language Models
Crucially, we find that $k$NN-LMs are more susceptible to leaking private information from their private datastore than parametric models.
MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions
The information stored in large language models (LLMs) falls out of date quickly, and retraining from scratch is often not an option.
$k$NN-Adapter: Efficient Domain Adaptation for Black-Box Language Models
Fine-tuning a language model on a new domain is standard practice for domain adaptation.
Training Language Models with Memory Augmentation
Recent work has improved language models (LMs) remarkably by equipping them with a non-parametric memory component.
Recovering Private Text in Federated Learning of Language Models
For the first time, we show the feasibility of recovering text from large batch sizes of up to 128 sentences.
Structured Pruning Learns Compact and Accurate Models
The growing size of neural language models has led to increased attention in model compression.
Should You Mask 15% in Masked Language Modeling?
In this work, we revisit this important choice of MLM pre-training.
Simple Entity-Centric Questions Challenge Dense Retrievers
Open-domain question answering has exploded in popularity recently due to the success of dense retrieval models, which have surpassed sparse models using only a few supervised training examples.
Ranked #3 on
Passage Retrieval
on EntityQuestions
Factual Probing Is [MASK]: Learning vs. Learning to Recall
Petroni et al. (2019) demonstrated that it is possible to retrieve world facts from a pre-trained language model by expressing them as cloze-style prompts and interpret the model's prediction accuracy as a lower bound on the amount of factual information it encodes.
A Frustratingly Easy Approach for Entity and Relation Extraction
Our approach essentially builds on two independent encoders and merely uses the entity model to construct the input for the relation model.
Ranked #1 on
Named Entity Recognition (NER)
on ACE 2005
SemRegex: A Semantics-Based Approach for Generating Regular Expressions from Natural Language Specifications
Recent research proposes syntax-based approaches to address the problem of generating programs from natural language specifications.
MULDEF: Multi-model-based Defense Against Adversarial Examples for Neural Networks
In particular, given a target model, our framework includes multiple models (constructed from the target model) to form a model family.
