Search Results for author:
Found 6 papers, 5 papers with code
GEE! Grammar Error Explanation with Large Language Models
To address this gap, we propose the task of grammar error explanation, where a system needs to provide one-sentence explanations for each grammatical error in a pair of erroneous and corrected sentences.
A Critical Evaluation of Evaluations for Long-form Question Answering
We present a careful analysis of experts' evaluation, which focuses on new aspects such as the comprehensiveness of the answer.
KNN-LM Does Not Improve Open-ended Text Generation
Digging deeper, we find that interpolating with a retrieval distribution actually increases perplexity compared to a baseline Transformer LM for the majority of tokens in the WikiText-103 test set, even though the overall perplexity is lower due to a smaller number of tokens for which perplexity dramatically decreases after interpolation.
Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
To increase the robustness of AI-generated text detection to paraphrase attacks, we introduce a simple defense that relies on retrieving semantically-similar generations and must be maintained by a language model API provider.
DEMETR: Diagnosing Evaluation Metrics for Translation
While machine translation evaluation metrics based on string overlap (e. g., BLEU) have their limitations, their computations are transparent: the BLEU score assigned to a particular candidate translation can be traced back to the presence or absence of certain words.
SLING: Sino Linguistic Evaluation of Large Language Models
To understand what kinds of linguistic knowledge are encoded by pretrained Chinese language models (LMs), we introduce the benchmark of Sino LINGuistics (SLING), which consists of 38K minimal sentence pairs in Mandarin Chinese grouped into 9 high-level linguistic phenomena.
