Search Results for author:
Found 11 papers, 5 papers with code
Introducing v0.5 of the AI Safety Benchmark from MLCommons
We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0. 5 benchmark.
Consistent Explanations in the Face of Model Indeterminacy via Ensembling
This work addresses the challenge of providing consistent explanations for predictive models in the presence of model indeterminacy, which arises due to the existence of multiple (nearly) equally well-performing models for a given dataset and task.
Baselines for Identifying Watermarked Large Language Models
We consider the emerging problem of identifying the presence and use of watermarking schemes in widely used, publicly hosted, closed source large language models (LLMs).
Learning the Wrong Lessons: Inserting Trojans During Knowledge Distillation
In recent years, knowledge distillation has become a cornerstone of efficiently deployed machine learning, with labs and industries using knowledge distillation to train models that are inexpensive and resource-optimized.
MAUD: An Expert-Annotated Legal NLP Dataset for Merger Agreement Understanding
Reading comprehension of legal text can be a particularly challenging task due to the length and complexity of legal clauses and a shortage of expert-annotated datasets.
The Naughtyformer: A Transformer Understands Offensive Humor
Jokes are intentionally written to be funny, but not all jokes are created the same.
Lila: A Unified Benchmark for Mathematical Reasoning
Mathematical reasoning skills are essential for general-purpose intelligent systems to perform tasks from grocery shopping to climate modeling.
Ranked #1 on
Mathematical Reasoning
on Lila (OOD)
From Human Days to Machine Seconds: Automatically Answering and Generating Machine Learning Final Exams
We curate a dataset and benchmark of questions from machine learning final exams available online and code for answering these questions and generating new questions.
A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Level
We automatically synthesize programs using few-shot learning and OpenAI's Codex transformer and execute them to solve course problems at 81% automatic accuracy.
PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures
In real-world applications of machine learning, reliable and safe systems must consider measures of performance beyond standard test set accuracy.
Solving Probability and Statistics Problems by Program Synthesis
Our work is the first to introduce a new dataset of university-level probability and statistics problems and solve these problems in a scalable fashion using the program synthesis capabilities of large language models.
