Prompt Public Large Language Models to Synthesize Data for Private On-device Applications

no code implementations • 5 Apr 2024 • Shanshan Wu, Zheng Xu, Yanxiang Zhang, Yuanbo Zhang, Daniel Ramage

Pre-training on public data is an effective method to improve the performance for federated learning (FL) with differential privacy (DP).

Federated Learning

Efficient Language Model Architectures for Differentially Private Federated Learning

no code implementations • 12 Mar 2024 • Jae Hun Ro, Srinadh Bhojanapalli, Zheng Xu, Yanxiang Zhang, Ananda Theertha Suresh

Cross-device federated learning (FL) is a technique that trains a model on data distributed across typically millions of edge devices without data leaving the devices.

Computational Efficiency Federated Learning +1

Federated Learning of Gboard Language Models with Differential Privacy

no code implementations • 29 May 2023 • Zheng Xu, Yanxiang Zhang, Galen Andrew, Christopher A. Choquette-Choo, Peter Kairouz, H. Brendan McMahan, Jesse Rosenstock, Yuanbo Zhang

We train language models (LMs) with federated learning (FL) and differential privacy (DP) in the Google Keyboard (Gboard).

Federated Learning

MT-SNN: Enhance Spiking Neural Network with Multiple Thresholds

no code implementations • 20 Mar 2023 • Xiaoting Wang, Yanxiang Zhang, Yongzhe Zhang

Spiking neural networks (SNNs), as a biology-inspired method mimicking the spiking nature of brain neurons, is a promising energy-efficient alternative to the traditional artificial neural networks (ANNs).