no code implementations • 2 May 2024 • Wei-Ning Chen, Berivan Isik, Peter Kairouz, Albert No, Sewoong Oh, Zheng Xu
We study $L_2$ mean estimation under central differential privacy and communication constraints, and address two key challenges: firstly, existing mean estimation schemes that simultaneously handle both constraints are usually optimized for $L_\infty$ geometry and rely on random rotation or Kashin's representation to adapt to $L_2$ geometry, resulting in suboptimal leading constants in mean square errors (MSEs); secondly, schemes achieving order-optimal communication-privacy trade-offs do not extend seamlessly to streaming differential privacy (DP) settings (e. g., tree aggregation or matrix factorization), rendering them incompatible with DP-FTRL type optimizers.
1 code implementation • 8 Feb 2024 • Onur G. Guleryuz, Philip A. Chou, Berivan Isik, Hugues Hoppe, Danhang Tang, Ruofei Du, Jonathan Taylor, Philip Davidson, Sean Fanello
Through a variety of examples, we apply the sandwich architecture to sources with different numbers of channels, higher resolution, higher dynamic range, and perceptual distortion measures.
no code implementations • 6 Feb 2024 • Berivan Isik, Natalia Ponomareva, Hussein Hazimeh, Dimitris Paparas, Sergei Vassilvitskii, Sanmi Koyejo
With sufficient alignment, both downstream cross-entropy and BLEU score improve monotonically with more pretraining data.
1 code implementation • 22 Jun 2023 • Berivan Isik, Francesco Pase, Deniz Gunduz, Sanmi Koyejo, Tsachy Weissman, Michele Zorzi
The high communication cost of sending model updates from the clients to the server is a significant bottleneck for scalable federated learning (FL).
no code implementations • 20 Mar 2023 • Berivan Isik, Onur G. Guleryuz, Danhang Tang, Jonathan Taylor, Philip A. Chou
We propose differentiable approximations to key video codec components and demonstrate that, in addition to providing meaningful compression improvements over the standard codec, the neural codes of the sandwich lead to significantly better rate-distortion performance in two important scenarios. When transporting high-resolution video via low-resolution HEVC, the sandwich system obtains 6. 5 dB improvements over standard HEVC.
1 code implementation • 30 Sep 2022 • Berivan Isik, Francesco Pase, Deniz Gunduz, Tsachy Weissman, Michele Zorzi
At the end of the training, the final model is a sparse network with random weights -- or a subnetwork inside the dense random network.
no code implementations • 7 Feb 2022 • Berivan Isik, Tsachy Weissman
In this sense, the utility of the data for learning is essentially maintained, while reducing storage and privacy leakage by quantifiable amounts.
1 code implementation • 17 Nov 2021 • Berivan Isik, Philip A. Chou, Sung Jin Hwang, Nick Johnston, George Toderici
We consider the attributes of a point cloud as samples of a vector-valued volumetric function at discrete positions.
no code implementations • 7 May 2021 • Berivan Isik
In this work, we take a different approach and compress a functional representation of 3D scenes.
1 code implementation • 16 Feb 2021 • Berivan Isik, Tsachy Weissman, Albert No
We study the neural network (NN) compression problem, viewing the tension between the compression ratio and NN performance through the lens of rate-distortion theory.
no code implementations • 15 Feb 2021 • Berivan Isik, Kristy Choi, Xin Zheng, Tsachy Weissman, Stefano Ermon, H. -S. Philip Wong, Armin Alaghi
Compression and efficient storage of neural network (NN) parameters is critical for applications that run on resource-constrained devices.
no code implementations • NeurIPS Workshop DL-IG 2020 • Berivan Isik, Kristy Choi, Xin Zheng, H.-S. Philip Wong, Stefano Ermon, Tsachy Weissman, Armin Alaghi
Efficient compression and storage of neural network (NN) parameters is critical for resource-constrained, downstream machine learning applications.
no code implementations • 21 May 2020 • Leighton Pate Barnes, Huseyin A. Inan, Berivan Isik, Ayfer Ozgur
The statistically optimal communication scheme arising from the analysis of this model leads to a new sparsification technique for SGD, which concatenates random-k and top-k, considered separately in the prior literature.