Search Results for author:
Found 9 papers, 8 papers with code
QuRating: Selecting High-Quality Data for Training Language Models
Selecting high-quality pre-training data is important for creating capable language models, but existing methods rely on simple heuristics.
Poisoning Retrieval Corpora by Injecting Adversarial Passages
Dense retrievers have achieved state-of-the-art performance in various information retrieval tasks, but to what extent can they be safely deployed in real-world applications?
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
We find real-world software engineering to be a rich, sustainable, and challenging testbed for evaluating the next generation of language models.
Ranked #2 on
Bug fixing
on SWE-bench
Adapting Language Models to Compress Contexts
Transformer-based language models (LMs) are powerful and widely-applicable tools, but their usefulness is constrained by a finite context window and the expensive computational cost of processing long text documents.
Finding Dataset Shortcuts with Grammar Induction
Many NLP datasets have been found to contain shortcuts: simple decision rules that achieve surprisingly high accuracy.
A Kernel-Based View of Language Model Fine-Tuning
It has become standard to solve NLP tasks by fine-tuning pre-trained language models (LMs), especially in low-data settings.
Should You Mask 15% in Masked Language Modeling?
In this work, we revisit this important choice of MLM pre-training.
Phrase Retrieval Learns Passage Retrieval, Too
Dense retrieval methods have shown great promise over sparse retrieval methods in a range of NLP problems.
