Search Results for author:
Found 28 papers, 7 papers with code
AdaGossip: Adaptive Consensus Step-size for Decentralized Deep Learning with Communication Compression
Decentralized learning is crucial in supporting on-device learning over large distributed datasets, eliminating the need for a central server.
Asynchronous Federated Reinforcement Learning with Policy Gradient Updates: Algorithm Design and Convergence Analysis
Moreover, compared to synchronous FedPG, AFedPG improves the time complexity from $\mathcal{O}(\frac{t_{\max}}{N})$ to $\mathcal{O}(\frac{1}{\sum_{i=1}^{N} \frac{1}{t_{i}}})$, where $t_{i}$ denotes the time consumption in each iteration at the agent $i$, and $t_{\max}$ is the largest one.
Localized Distributional Robustness in Submodular Multi-Task Subset Selection
This approach bridges the existing gap in the optimization of performance-robustness trade-offs in multi-task subset selection.
FedNMUT -- Federated Noisy Model Update Tracking Convergence Analysis
A novel Decentralized Noisy Model Update Tracking Federated Learning algorithm (FedNMUT) is proposed that is tailored to function efficiently in the presence of noisy communication channels that reflect imperfect information exchange.
Unveiling Privacy, Memorization, and Input Curvature Links
Second, we present a novel insight showing that input loss curvature is upper-bounded by the differential privacy parameter.
Improved Convergence Analysis and SNR Control Strategies for Federated Learning in the Presence of Noise
We propose an improved convergence analysis technique that characterizes the distributed learning paradigm of federated learning (FL) with imperfect/noisy uplink and downlink communications.
Communication-Efficient Zeroth-Order Distributed Online Optimization: Algorithm, Theory, and Applications
This paper focuses on a multi-agent zeroth-order online optimization problem in a federated learning setting for target tracking.
Global Update Tracking: A Decentralized Learning Algorithm for Heterogeneous Data
Decentralized learning enables the training of deep learning models over large distributed datasets generated at different locations, without the need for a central server.
On the Convergence of Decentralized Federated Learning Under Imperfect Information Sharing
The first algorithm, Federated Noisy Decentralized Learning (FedNDL1), comes from the literature, where the noise is added to their parameters to simulate the scenario of the presence of noisy communication channels.
No-Regret Learning in Dynamic Stackelberg Games
The regret of the proposed learning algorithm is independent of the size of the state space and polynomial in the rest of the parameters of the game.
On the Benefits of Inducing Local Lipschitzness for Robust Generative Adversarial Imitation Learning
We explore methodologies to improve the robustness of generative adversarial imitation learning (GAIL) algorithms to observation noise.
Robust Training in High Dimensions via Block Coordinate Geometric Median Descent
Geometric median (\textsc{Gm}) is a classical method in statistics for achieving a robust estimation of the uncorrupted data; under gross corruption, it achieves the optimal breakdown point of 0. 5.
Ranked #19 on
Image Classification
on MNIST
(Accuracy metric)
On the Convergence of Differentially Private Federated Learning on Non-Lipschitz Objectives, and with Normalized Client Updates
The primary reason for this is that the clipping operation (i. e., projection onto an $\ell_2$ ball of a fixed radius called the clipping threshold) for bounding the sensitivity of the average update to each client's update introduces bias depending on the clipping threshold and the number of local steps in FL, and analyzing this is not easy.
Generalization Bounds for Sparse Random Feature Expansions
In particular, we provide generalization bounds for functions in a certain class (that is dense in a reproducing kernel Hilbert space) depending on the number of samples and the distribution of features.
Physical-Layer Security via Distributed Beamforming in the Presence of Adversaries with Unknown Locations
In such a setting, we develop a periodic transmission strategy, i. e., a sequence of joint beamforming gain and artificial noise pairs, that prevents the adversaries from decreasing their uncertainty on the information sequence by eavesdropping on the transmission.
Communication-Efficient Variance-Reduced Decentralized Stochastic Optimization over Time-Varying Directed Graphs
To our knowledge, this is the first decentralized optimization framework for time-varying directed networks that achieves such a convergence rate and applies to settings requiring sparsified communication.
Faster Non-Convex Federated Learning via Global and Local Momentum
We propose \texttt{FedGLOMO}, a novel federated learning (FL) algorithm with an iteration complexity of $\mathcal{O}(\epsilon^{-1. 5})$ to converge to an $\epsilon$-stationary point (i. e., $\mathbb{E}[\|\nabla f(\bm{x})\|^2] \leq \epsilon$) for smooth non-convex functions -- under arbitrary client heterogeneity and compressed communication -- compared to the $\mathcal{O}(\epsilon^{-2})$ complexity of most prior works.
On the Benefits of Multiple Gossip Steps in Communication-Constrained Decentralized Optimization
In this paper, we show that, in such compressed decentralized optimization settings, there are benefits to having {\em multiple} gossip steps between subsequent gradient iterations, even when the cost of doing so is appropriately accounted for e. g. by means of reducing the precision of compressed information.
Decentralized Optimization On Time-Varying Directed Graphs Under Communication Constraints
We propose a communication-efficient algorithm for decentralized convex optimization that rely on sparsification of local updates exchanged between neighboring agents in the network.
Identifying Sparse Low-Dimensional Structures in Markov Chains: A Nonnegative Matrix Factorization Approach
We formulate the task of representation learning as that of mapping the state space of the model to a low-dimensional state space, called the kernel space.
Performance-Complexity Tradeoffs in Greedy Weak Submodular Maximization with Random Sampling
The latter implies that uniform sampling strategies with a fixed sampling size achieve a non-trivial approximation factor; however, we show that with overwhelming probability, these methods fail to find the optimal subset.
Evolutionary Self-Expressive Models for Subspace Clustering
The problem of organizing data that evolves over time into clusters is encountered in a number of practical settings.
Towards Accelerated Greedy Sampling and Reconstruction of Bandlimited Graph Signals
Then, we consider the Bayesian scenario where we formulate the sampling task as the problem of maximizing a monotone weak submodular function, and propose a randomized-greedy algorithm to find a sub-optimal subset of informative nodes.
Accelerated Sparse Subspace Clustering
State-of-the-art algorithms for sparse subspace clustering perform spectral clustering on a similarity matrix typically obtained by representing each data point as a sparse combination of other points using either basis pursuit (BP) or orthogonal matching pursuit (OMP).
Sampling and Reconstruction of Graph Signals via Weak Submodularity and Semidefinite Relaxation
We study the problem of sampling a bandlimited graph signal in the presence of noise, where the objective is to select a node subset of prescribed cardinality that minimizes the signal reconstruction mean squared error (MSE).
Sparse recovery via Orthogonal Least-Squares under presence of Noise
We consider the Orthogonal Least-Squares (OLS) algorithm for the recovery of a $m$-dimensional $k$-sparse signal from a low number of noisy linear measurements.
Sampling Requirements and Accelerated Schemes for Sparse Linear Regression with Orthogonal Least-Squares
We analyze the performance of AOLS and establish lower bounds on the probability of exact recovery for both noiseless and noisy random linear measurements.
Sparse Linear Regression via Generalized Orthogonal Least-Squares
Sparse linear regression, which entails finding a sparse solution to an underdetermined system of linear equations, can formally be expressed as an $l_0$-constrained least-squares problem.
