Search Results for author: Max Marion

Found 1 papers, 0 papers with code

When Less is More: Investigating Data Pruning for Pretraining LLMs at Scale

no code implementations8 Sep 2023 Max Marion, Ahmet Üstün, Luiza Pozzobon, Alex Wang, Marzieh Fadaee, Sara Hooker

In this work, we take a wider view and explore scalable estimates of data quality that can be used to systematically measure the quality of pretraining data.

Memorization

Cannot find the paper you are looking for? You can Submit a new open access paper.