Search Results for author: Sizhe Dang

Found 2 papers, 0 papers with code

Second-Order Fine-Tuning without Pain for LLMs:A Hessian Informed Zeroth-Order Optimizer

no code implementations23 Feb 2024 Yanjun Zhao, Sizhe Dang, Haishan Ye, Guang Dai, Yi Qian, Ivor W. Tsang

Fine-tuning large language models (LLMs) with classic first-order optimizers entails prohibitive GPU memory due to the backpropagation process.

Cannot find the paper you are looking for? You can Submit a new open access paper.