1 code implementation • 26 Aug 2023 • Mengwei Xu, Dongqi Cai, Yaozong Wu, Xiang Li, Shangguang Wang
Federated Learning (FL), a method to preserve user data privacy, is often employed in fine-tuning LLMs to downstream mobile tasks, an approach known as FedLLM.
1 code implementation • 12 Dec 2022 • Dongqi Cai, Shangguang Wang, Yaozong Wu, Felix Xiaozhu Lin, Mengwei Xu
Such an inadequacy of data labels is known as a few-shot scenario; it becomes the key blocker for mobile NLP applications.
no code implementations • 1 Dec 2022 • Dongqi Cai, Yaozong Wu, Haitao Yuan, Shangguang Wang, Felix Xiaozhu Lin, Mengwei Xu
To address these challenges, we first introduce a data generator for federated few-shot learning tasks, which encompasses the quantity and skewness of scarce labeled data in a realistic setting.
1 code implementation • 20 May 2022 • Dongqi Cai, Yaozong Wu, Shangguang Wang, Felix Xiaozhu Lin, Mengwei Xu
A key challenge is to properly configure the depth and width of adapters, to which the training speed and efficiency is highly sensitive.