Search Results for author:
Found 4 papers, 3 papers with code
The Fine Line: Navigating Large Language Model Pretraining with Down-streaming Capability Analysis
To address the aforementioned limitations, this paper undertakes a comprehensive comparison of model capabilities at various pretraining intermediate checkpoints.
Machine Unlearning of Pre-trained Large Language Models
This study investigates the concept of the `right to be forgotten' within the context of large language models (LLMs).
Data Engineering for Scaling Language Models to 128K Context
We demonstrate that continual pretraining of the full model on 1B-5B tokens of such data is an effective and affordable strategy for scaling the context length of language models to 128K.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
