Protecting Sensitive Data through Federated Co-Training
In many applications, sensitive data is inherently distributed and may not be pooled due to privacy concerns. Federated learning allows us to collaboratively train a model without pooling the data by iteratively aggregating the parameters of local models. It is possible, though, to infer upon the sensitive data from the shared model parameters. We propose to use a federated co-training approach where clients share hard labels on a public unlabeled dataset instead of model parameters. A consensus on the shared labels forms a pseudo labeling for the unlabeled dataset that clients use in combination with their private data to train local models. We show that sharing hard labels substantially improves privacy over sharing model parameters. At the same time, federated co-training achieves a model quality comparable to federated learning. Moreover, it allows us to use local models such as (gradient boosted) decision trees, rule ensembles, and random forests that do not lend themselves to the parameter aggregation used in federated learning.
PDF Abstract
SVHN
Fashion-MNIST
