Search Results for author:
Found 13 papers, 4 papers with code
Jamba: A Hybrid Transformer-Mamba Language Model
We present Jamba, a new base large language model based on a novel hybrid Transformer-Mamba mixture-of-experts (MoE) architecture.
Generating Benchmarks for Factuality Evaluation of Language Models
FACTOR automatically transforms a factual corpus of interest into a benchmark evaluating an LM's propensity to generate true facts from the corpus vs. similar but incorrect statements.
Human or Not? A Gamified Approach to the Turing Test
Over the course of a month, the game was played by over 1. 5 million users who engaged in anonymous two-minute chat sessions with either another human or an AI language model which was prompted to behave like humans.
In-Context Retrieval-Augmented Language Models
Retrieval-Augmented Language Modeling (RALM) methods, which condition a language model (LM) on relevant documents from a grounding corpus during generation, were shown to significantly improve language modeling performance.
Parallel Context Windows for Large Language Models
We present Parallel Context Windows (PCW), a method that alleviates the context window restriction for any off-the-shelf LLM without further training.
MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning
Huge language models (LMs) have ushered in a new era for AI, serving as a gateway to natural-language-based knowledge tasks.
Standing on the Shoulders of Giant Frozen Language Models
To demonstrate this, we introduce three novel methods for leveraging frozen models: input-dependent prompt tuning, frozen readers, and recursive LMs, each of which vastly improves on current frozen-model approaches.
PMI-Masking: Principled masking of correlated spans
Specifically, we show experimentally that PMI-Masking reaches the performance of prior masking approaches in half the training time, and consistently improves performance at the end of training.
The Cost of Training NLP Models: A Concise Overview
We review the cost of training large-scale language models, and the drivers of these costs.
SenseBERT: Driving Some Sense into BERT
The ability to learn from large unlabeled corpora has allowed neural language models to advance the frontier in natural language understanding.
Ranked #11 on
Word Sense Disambiguation
on Words in Context
