no code implementations • ICML 2020 • Jayadev Acharya, Kallista Bonawitz, Peter Kairouz, Daniel Ramage, Ziteng Sun
The original definition of LDP assumes that all the elements in the data domain are equally sensitive.
no code implementations • 8 May 2024 • Eugene Bagdasaryan, Ren Yi, Sahra Ghalebikesabi, Peter Kairouz, Marco Gruteser, Sewoong Oh, Borja Balle, Daniel Ramage
The growing use of large language model (LLM)-based conversational agents to manage sensitive user data raises significant privacy concerns.
no code implementations • 2 May 2024 • Wei-Ning Chen, Berivan Isik, Peter Kairouz, Albert No, Sewoong Oh, Zheng Xu
We study $L_2$ mean estimation under central differential privacy and communication constraints, and address two key challenges: firstly, existing mean estimation schemes that simultaneously handle both constraints are usually optimized for $L_\infty$ geometry and rely on random rotation or Kashin's representation to adapt to $L_2$ geometry, resulting in suboptimal leading constants in mean square errors (MSEs); secondly, schemes achieving order-optimal communication-privacy trade-offs do not extend seamlessly to streaming differential privacy (DP) settings (e. g., tree aggregation or matrix factorization), rendering them incompatible with DP-FTRL type optimizers.
3 code implementations • 16 Apr 2024 • Hubert Eichner, Daniel Ramage, Kallista Bonawitz, Dzmitry Huba, Tiziano Santoro, Brett McLarnon, Timon Van Overveldt, Nova Fallen, Peter Kairouz, Albert Cheu, Katharine Daly, Adria Gascon, Marco Gruteser, Brendan Mcmahan
Federated Learning and Analytics (FLA) have seen widespread adoption by technology platforms for processing sensitive on-device data.
no code implementations • 1 Apr 2024 • Florian Hartmann, Duc-Hieu Tran, Peter Kairouz, Victor Cărbune, Blaise Aguera y Arcas
In this work, we show the feasibility of applying cascade systems in such setups by equipping the local model with privacy-preserving techniques that reduce the risk of leaking private information when querying the remote model.
no code implementations • 21 Feb 2024 • Da Yu, Peter Kairouz, Sewoong Oh, Zheng Xu
Service providers of large language model (LLM) applications collect user instructions in the wild and use them in further aligning LLMs with users' intentions.
no code implementations • 13 Oct 2023 • Nikhil Kandpal, Krishna Pillutla, Alina Oprea, Peter Kairouz, Christopher A. Choquette-Choo, Zheng Xu
Fine-tuning is a common and effective method for tailoring large language models (LLMs) to specialized tasks and applications.
1 code implementation • 20 Jul 2023 • Enayat Ullah, Christopher A. Choquette-Choo, Peter Kairouz, Sewoong Oh
We propose new techniques for reducing communication in private federated learning without the need for setting or tuning compression rates.
no code implementations • NeurIPS 2023 • Jingfeng Wu, Wennan Zhu, Peter Kairouz, Vladimir Braverman
For single-round FFE, it is known that count sketching is nearly information-theoretically optimal for achieving the fundamental accuracy-communication trade-offs [Chen et al., 2022].
no code implementations • 29 May 2023 • Zheng Xu, Yanxiang Zhang, Galen Andrew, Christopher A. Choquette-Choo, Peter Kairouz, H. Brendan McMahan, Jesse Rosenstock, Yuanbo Zhang
We train language models (LMs) with federated learning (FL) and differential privacy (DP) in the Google Keyboard (Gboard).
1 code implementation • 6 Feb 2023 • Galen Andrew, Peter Kairouz, Sewoong Oh, Alina Oprea, H. Brendan McMahan, Vinith M. Suriyakumar
Privacy estimation techniques for differentially private (DP) algorithms are useful for comparing against analytical bounds, or to empirically measure privacy loss in settings where known analytical bounds are not tight.
no code implementations • 9 Jul 2022 • Wei-Ning Chen, Ayfer Özgür, Peter Kairouz
Unlike previous discrete DP schemes based on additive noise, our mechanism encodes local information into a parameter of the binomial distribution, and hence the output distribution is discrete with bounded support.
no code implementations • 7 Jun 2022 • YuHan Liu, Ananda Theertha Suresh, Wennan Zhu, Peter Kairouz, Marco Gruteser
In this scenario, the amount of noise injected into the histogram to obtain differential privacy is proportional to the maximum user contribution, which can be amplified by few outliers.
no code implementations • 7 Mar 2022 • Wei-Ning Chen, Christopher A. Choquette-Choo, Peter Kairouz, Ananda Theertha Suresh
We consider the problem of training a $d$ dimensional model with distributed differential privacy (DP) where secure aggregation (SecAgg) is used to ensure that the server only sees the noisy sum of $n$ model updates in every training round.
no code implementations • 13 Jan 2022 • Jiang Zhang, Lillian Clark, Matthew Clark, Konstantinos Psounis, Peter Kairouz
Cellular providers and data aggregating companies crowdsource celluar signal strength measurements from user devices to generate signal maps, which can be used to improve network performance.
1 code implementation • 3 Nov 2021 • Eugene Bagdasaryan, Peter Kairouz, Stefan Mellem, Adrià Gascón, Kallista Bonawitz, Deborah Estrin, Marco Gruteser
We design a scalable algorithm to privately generate location heatmaps over decentralized data from millions of user devices.
no code implementations • 29 Oct 2021 • Abhin Shah, Wei-Ning Chen, Johannes Balle, Peter Kairouz, Lucas Theis
Compressing the output of \epsilon-locally differentially private (LDP) randomizers naively leads to suboptimal utility.
1 code implementation • NeurIPS 2021 • Naman Agarwal, Peter Kairouz, Ziyu Liu
We introduce the multi-dimensional Skellam mechanism, a discrete differential privacy mechanism based on the difference of two independent Poisson random variables.
1 code implementation • 23 Aug 2021 • Virat Shejwalkar, Amir Houmansadr, Peter Kairouz, Daniel Ramage
While recent works have indicated that federated learning (FL) may be vulnerable to poisoning attacks by compromised clients, their real impact on production FL systems is not fully understood.
2 code implementations • 14 Jul 2021 • Jianyu Wang, Zachary Charles, Zheng Xu, Gauri Joshi, H. Brendan McMahan, Blaise Aguera y Arcas, Maruan Al-Shedivat, Galen Andrew, Salman Avestimehr, Katharine Daly, Deepesh Data, Suhas Diggavi, Hubert Eichner, Advait Gadhikar, Zachary Garrett, Antonious M. Girgis, Filip Hanzely, Andrew Hard, Chaoyang He, Samuel Horvath, Zhouyuan Huo, Alex Ingerman, Martin Jaggi, Tara Javidi, Peter Kairouz, Satyen Kale, Sai Praneeth Karimireddy, Jakub Konecny, Sanmi Koyejo, Tian Li, Luyang Liu, Mehryar Mohri, Hang Qi, Sashank J. Reddi, Peter Richtarik, Karan Singhal, Virginia Smith, Mahdi Soltanolkotabi, Weikang Song, Ananda Theertha Suresh, Sebastian U. Stich, Ameet Talwalkar, Hongyi Wang, Blake Woodworth, Shanshan Wu, Felix X. Yu, Honglin Yuan, Manzil Zaheer, Mi Zhang, Tong Zhang, Chunxiang Zheng, Chen Zhu, Wennan Zhu
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy protection.
no code implementations • 16 Jun 2021 • Wei-Ning Chen, Peter Kairouz, Ayfer Özgür
For the interactive setting, we propose a novel tree-based estimation scheme and show that the minimum sample-size needed to achieve dimension-free convergence can be further reduced to $n^*(s, d, b) = \tilde{O}\left( {s^2\log^2 d}/{2^b} \right)$.
no code implementations • 11 May 2021 • Antonious M. Girgis, Deepesh Data, Suhas Diggavi, Ananda Theertha Suresh, Peter Kairouz
The central question studied in this paper is Renyi Differential Privacy (RDP) guarantees for general discrete local mechanisms in the shuffle privacy model.
2 code implementations • 26 Feb 2021 • Peter Kairouz, Brendan Mcmahan, Shuang Song, Om Thakkar, Abhradeep Thakurta, Zheng Xu
We consider training models with differential privacy (DP) using mini-batch gradients.
1 code implementation • 12 Feb 2021 • Peter Kairouz, Ziyu Liu, Thomas Steinke
To ensure privacy, we add on-device noise and use secure aggregation so that only the noisy sum is revealed to the server.
no code implementations • 30 Oct 2020 • Jayadev Acharya, Peter Kairouz, YuHan Liu, Ziteng Sun
We consider the problem of estimating sparse discrete distributions under local differential privacy (LDP) and communication constraints.
no code implementations • 17 Aug 2020 • Antonious M. Girgis, Deepesh Data, Suhas Diggavi, Peter Kairouz, Ananda Theertha Suresh
We consider a distributed empirical risk minimization (ERM) optimization problem with communication efficiency and privacy requirements, motivated by the federated learning (FL) framework.
no code implementations • 14 Aug 2020 • Peter Kairouz, Mónica Ribero, Keith Rush, Abhradeep Thakurta
In particular, we show that if the gradients lie in a known constant rank subspace, and assuming algorithmic access to an envelope which bounds decaying sensitivity, one can achieve faster convergence to an excess empirical risk of $\tilde O(1/\epsilon n)$, where $\epsilon$ is the privacy budget and $n$ the number of samples.
no code implementations • NeurIPS 2020 • Wei-Ning Chen, Peter Kairouz, Ayfer Özgür
In particular, we consider the problems of mean estimation and frequency estimation under $\varepsilon$-local differential privacy and $b$-bit communication constraints.
no code implementations • NeurIPS 2020 • Borja Balle, Peter Kairouz, H. Brendan McMahan, Om Thakkar, Abhradeep Thakurta
It has privacy/accuracy trade-offs similar to privacy amplification by subsampling/shuffling.
1 code implementation • 27 Jan 2020 • Reihaneh Torkzadehmahani, Peter Kairouz, Benedict Paten
Generative Adversarial Networks (GANs) are one of the well-known models to generate synthetic data including images, especially for research communities that cannot use original sensitive datasets because they are not publicly accessible.
8 code implementations • 10 Dec 2019 • Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D'Oliveira, Hubert Eichner, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaid Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konečný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, Sen Zhao
FL embodies the principles of focused data collection and minimization, and can mitigate many of the systemic privacy risks and costs resulting from traditional, centralized machine learning and data science approaches.
no code implementations • 18 Nov 2019 • Ziteng Sun, Peter Kairouz, Ananda Theertha Suresh, H. Brendan McMahan
This paper focuses on backdoor attacks in the federated learning setting, where the goal of the adversary is to reduce the performance of the model on targeted tasks while maintaining good performance on the main task.
3 code implementations • ICLR 2020 • Sean Augenstein, H. Brendan McMahan, Daniel Ramage, Swaroop Ramaswamy, Peter Kairouz, Mingqing Chen, Rajiv Mathews, Blaise Aguera y Arcas
To improve real-world applications of machine learning, experienced modelers develop intuition about their datasets, their models, and how the two interact.
no code implementations • 8 Nov 2019 • Mario Diaz, Peter Kairouz, Jiachun Liao, Lalitha Sankar
Privacy concerns have led to the development of privacy-preserving approaches for learning models from sensitive data.
no code implementations • 31 Oct 2019 • Jayadev Acharya, Keith Bonawitz, Peter Kairouz, Daniel Ramage, Ziteng Sun
Local differential privacy (LDP) is a strong notion of privacy for individual users that often comes at the expense of a significant drop in utility.
no code implementations • 27 Sep 2019 • Peter Kairouz, Jiachun Liao, Chong Huang, Maunil Vyas, Monica Welfert, Lalitha Sankar
We present a data-driven framework for learning fair universal representations (FUR) that guarantee statistical fairness for any learning task that may not be known a priori.
1 code implementation • 5 Jun 2019 • Tyler Sypherd, Mario Diaz, John Kevin Cava, Gautam Dasarathy, Peter Kairouz, Lalitha Sankar
We introduce a tunable loss function called $\alpha$-loss, parameterized by $\alpha \in (0,\infty]$, which interpolates between the exponential loss ($\alpha = 1/2$), the log-loss ($\alpha = 1$), and the 0-1 loss ($\alpha = \infty$), for the machine learning setting of classification.
no code implementations • ICLR 2019 • Chong Huang, Xiao Chen, Peter Kairouz, Lalitha Sankar, Ram Rajagopal
We present Generative Adversarial Privacy and Fairness (GAPF), a data-driven framework for learning private and fair representations of the data.
no code implementations • 12 Feb 2019 • Tyler Sypherd, Mario Diaz, Lalitha Sankar, Peter Kairouz
We present $\alpha$-loss, $\alpha \in [1,\infty]$, a tunable loss function for binary classification that bridges log-loss ($\alpha=1$) and $0$-$1$ loss ($\alpha = \infty$).
4 code implementations • 15 Dec 2018 • H. Brendan McMahan, Galen Andrew, Ulfar Erlingsson, Steve Chien, Ilya Mironov, Nicolas Papernot, Peter Kairouz
In this work we address the practical challenges of training machine learning models on privacy-sensitive datasets by introducing a modular approach that minimizes changes to training algorithms, provides a variety of configuration strategies for the privacy mechanism, and then isolates and simplifies the critical logic that computes the final privacy guarantees.
no code implementations • 21 Sep 2018 • Xiao Chen, Peter Kairouz, Ram Rajagopal
Designing a data sharing mechanism without sacrificing too much privacy can be considered as a game between data holders and malicious attackers.
no code implementations • ICLR 2019 • Chong Huang, Peter Kairouz, Xiao Chen, Lalitha Sankar, Ram Rajagopal
We present a data-driven framework called generative adversarial privacy (GAP).
no code implementations • 23 Apr 2018 • Witold Oleszkiewicz, Peter Kairouz, Karol Piczak, Ram Rajagopal, Tomasz Trzcinski
Extensive evaluation on a biometric dataset of fingerprints and cartoon faces confirms usefulness of our simple yet effective method.
no code implementations • 26 Oct 2017 • Chong Huang, Peter Kairouz, Xiao Chen, Lalitha Sankar, Ram Rajagopal
On the one hand, context-free privacy solutions, such as differential privacy, provide strong privacy guarantees, but often lead to a significant reduction in utility.
no code implementations • 24 Feb 2016 • Peter Kairouz, Keith Bonawitz, Daniel Ramage
The collection and analysis of user data drives improvements in the app and web ecosystems, but comes with risks to privacy.
no code implementations • NeurIPS 2015 • Peter Kairouz, Sewoong Oh, Pramod Viswanath
In this setting, each party is interested in computing a function on its private bit and all the other parties' bits.
no code implementations • 29 Dec 2014 • Giulia Fanti, Peter Kairouz, Sewoong Oh, Pramod Viswanath
Whether for fear of judgment or personal endangerment, it is crucial to keep anonymous the identity of the user who initially posted a sensitive message.
no code implementations • 4 Nov 2013 • Peter Kairouz, Sewoong Oh, Pramod Viswanath
Sequential querying of differentially private mechanisms degrades the overall privacy level.
Data Structures and Algorithms Cryptography and Security Information Theory Information Theory