Search Results for author:
Found 28 papers, 1 papers with code
Public-data Assisted Private Stochastic Optimization: Power and Limitations
We also study PA-DP supervised learning with \textit{unlabeled} public samples.
Differentially Private Worst-group Risk Minimization
We first present a new algorithm that achieves excess worst-group population risk of $\tilde{O}(\frac{p\sqrt{d}}{K\epsilon} + \sqrt{\frac{p}{K}})$, where $K$ is the total number of samples drawn from all groups and $d$ is the problem dimension.
Differentially Private Non-Convex Optimization under the KL Condition with Optimal Rates
We further show that, without assuming the KL condition, the same gradient descent algorithm can achieve fast convergence to a stationary point when the gradient stays sufficiently large during the run of the algorithm.
Differentially Private Domain Adaptation with Theoretical Guarantees
This is the modern problem of supervised domain adaptation from a public source to a private target domain.
Differentially Private Algorithms for the Stochastic Saddle Point Problem with Optimal Rates for the Strong Gap
We show that convex-concave Lipschitz stochastic saddle point problems (also known as stochastic minimax optimization) can be solved under the constraint of $(\epsilon,\delta)$-differential privacy with \emph{strong (primal-dual) gap} rate of $\tilde O\big(\frac{1}{\sqrt{n}} + \frac{\sqrt{d}}{n\epsilon}\big)$, where $n$ is the dataset size and $d$ is the dimension of the problem.
Private Domain Adaptation from a Public Source
A key problem in a variety of applications is that of domain adaptation from a public source domain, for which a relatively large amount of labeled data with no privacy constraints is at one's disposal, to a private target domain, for which a private sample is available with very few or no labeled data.
Faster Rates of Convergence to Stationary Points in Differentially Private Optimization
We provide a new efficient algorithm that finds an $\tilde{O}\big(\big[\frac{\sqrt{d}}{n\varepsilon}\big]^{2/3}\big)$-stationary point in the finite-sum setting, where $n$ is the number of samples.
Differentially Private Generalized Linear Models Revisited
For this case, we close the gap in the existing work and show that the optimal rate is (up to log factors) $\Theta\left(\frac{\Vert w^*\Vert}{\sqrt{n}} + \min\left\{\frac{\Vert w^*\Vert}{\sqrt{n\epsilon}},\frac{\sqrt{\text{rank}}\Vert w^*\Vert}{n\epsilon}\right\}\right)$, where $\text{rank}$ is the rank of the design matrix.
Differentially Private Learning with Margin Guarantees
For the family of linear hypotheses, we give a pure DP learning algorithm that benefits from relative deviation margin guarantees, as well as an efficient DP learning algorithm with margin guarantees.
Differentially Private Stochastic Optimization: New Results in Convex and Non-Convex Settings
For the $\ell_1$-case with smooth losses and polyhedral constraint, we provide the first nearly dimension independent rate, $\tilde O\big(\frac{\log^{2/3}{d}}{{(n\varepsilon)^{1/3}}}\big)$ in linear time.
Non-Euclidean Differentially Private Stochastic Convex Optimization: Optimal Rates in Linear Time
For $2 < p \leq \infty$, we show that existing linear-time constructions for the Euclidean setup attain a nearly optimal excess risk in the low-dimensional regime.
Learning from Mixtures of Private and Public Populations
Inspired by the above example, we consider a model in which the population $\mathcal{D}$ is a mixture of two sub-populations: a private sub-population $\mathcal{D}_{\sf priv}$ of private and sensitive data, and a public sub-population $\mathcal{D}_{\sf pub}$ of data with no privacy concerns.
Stability of Stochastic Gradient Descent on Nonsmooth Convex Losses
Our work is the first to address uniform stability of SGD on {\em nonsmooth} convex losses.
Private Query Release Assisted by Public Data
In comparison, with only private samples, this problem cannot be solved even for simple query classes with VC-dimension one, and without any private samples, a larger public sample of size $d/\alpha^2$ is needed.
Limits of Private Learning with Access to Public Data
We consider learning problems where the training set consists of two types of examples: private and public.
Private Stochastic Convex Optimization with Optimal Rates
A long line of existing work on private convex optimization focuses on the empirical loss and derives asymptotically tight bounds on the excess empirical loss.
Privately Answering Classification Queries in the Agnostic PAC Model
We formally study this problem in the agnostic PAC model and derive a new upper bound on the private sample complexity.
Model-Agnostic Private Learning
In the PAC model, we analyze our construction and prove upper bounds on the sample complexity for both the realizable and the non-realizable cases.
On exponential convergence of SGD in non-convex over-parametrized learning
Large over-parametrized models learned via stochastic gradient descent (SGD) methods have become a key element in modern machine learning.
Linear Queries Estimation with Local Differential Privacy
We study the problem of estimating a set of $d$ linear queries with respect to some unknown distribution $\mathbf{p}$ over a domain $\mathcal{J}=[J]$ based on a sensitive data set of $n$ individuals under the constraint of local differential privacy.
Model-Agnostic Private Learning via Stability
We provide a new technique to boost the average-case stability properties of learning algorithms to strong (worst-case) stability properties, and then exploit them to obtain private classification algorithms.
The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learning
We show that there is a critical batch size $m^*$ such that: (a) SGD iteration with mini-batch size $m\leq m^*$ is nearly equivalent to $m$ iterations of mini-batch size $1$ (\emph{linear scaling regime}).
Learners that Use Little Information
We discuss an approach that allows us to prove upper bounds on the amount of information that algorithms reveal about their inputs, and also provide a lower bound by showing a simple concept class for which every (possibly randomized) empirical risk minimizer must reveal a lot of information.
Typical Stability
We show that typical stability can control generalization error in adaptive data analysis even when the samples in the dataset are not necessarily independent and when queries to be computed are not necessarily of bounded-sensitivity as long as the results of the queries over the dataset (i. e., the computed statistics) follow a distribution with a "light" tail.
Algorithmic Stability for Adaptive Data Analysis
Specifically, suppose there is an unknown distribution $\mathbf{P}$ and a set of $n$ independent samples $\mathbf{x}$ is drawn from $\mathbf{P}$.
Local, Private, Efficient Protocols for Succinct Histograms
Moreover, we show that this much error is necessary, regardless of computational efficiency, and even for the simple setting where only one item appears with significant frequency in the data set.
More General Queries and Less Generalization Error in Adaptive Data Analysis
However, generalization error is typically bounded in a non-adaptive model, where all questions are specified before the dataset is drawn.
Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds
We provide a separate set of algorithms and matching lower bounds for the setting in which the loss functions are known to also be strongly convex.
