Search Results for author:
Found 4 papers, 2 papers with code
Checkpoint Merging via Bayesian Optimization in LLM Pretraining
The rapid proliferation of large language models (LLMs) such as GPT-4 and Gemini underscores the intense demand for resources during their training processes, posing significant challenges due to substantial computational and environmental costs.
Pre-training with Synthetic Data Helps Offline Reinforcement Learning
Recently, it has been shown that for offline deep reinforcement learning (DRL), pre-training Decision Transformer with a large language corpus can improve downstream performance (Reid et al., 2022).
Robust Unstructured Knowledge Access in Conversational Dialogue with ASR Errors
Knowledge cluster classification is boosted from 0. 7924 to 0. 9333 in Recall@1.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Suffix Retrieval-Augmented Language Modeling
SUREALM employs an embedding retriever to search for training sentences in a data store that share similar word history during sequence generation.
