Search Results for author:
Found 10 papers, 3 papers with code
The Relative Gaussian Mechanism and its Application to Private Gradient Descent
Leveraging this assumption, we introduce the Relative Gaussian Mechanism (RGM), in which the variance of the noise depends on the norm of the output.
Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees
In particular, we show that (i) for deterministic gradient descent, the clipping threshold only affects the higher-order terms of convergence, (ii) in the stochastic setting convergence to the true optimum cannot be guaranteed under the standard noise assumption, even under arbitrary small step-sizes.
Beyond spectral gap (extended): The role of the topology in decentralized learning
This paper aims to paint an accurate picture of sparsely-connected distributed optimization.
Beyond spectral gap: The role of the topology in decentralized learning
In data-parallel optimization of machine learning models, workers collaborate to improve their estimates of the model: more accurate gradients allow them to use larger learning rates and optimize faster.
Continuized Accelerations of Deterministic and Stochastic Gradient Descents, and of Gossip Algorithms
We introduce the ``continuized'' Nesterov acceleration, a close variant of Nesterov acceleration whose variables are indexed by a continuous time parameter.
A Continuized View on Nesterov Acceleration for Stochastic Gradient Descent and Randomized Gossip
We introduce the continuized Nesterov acceleration, a close variant of Nesterov acceleration whose variables are indexed by a continuous time parameter.
Asynchronous speedup in decentralized optimization
Our approach yields a precise characterization of convergence time and of its dependency on heterogeneous delays in the network.
Asynchronous Accelerated Proximal Stochastic Gradient for Strongly Convex Distributed Finite Sums
In this work, we study the problem of minimizing the sum of strongly convex functions split over a network of $n$ nodes.
Optimization and Control Distributed, Parallel, and Cluster Computing
Accelerated Decentralized Optimization with Local Updates for Smooth and Strongly Convex Objectives
Applying $ESDACD$ to quadratic local functions leads to an accelerated randomized gossip algorithm of rate $O( \sqrt{\theta_{\rm gossip}/n})$ where $\theta_{\rm gossip}$ is the rate of the standard randomized gossip.
Dynamic Safe Interruptibility for Decentralized Multi-Agent Reinforcement Learning
We give realistic sufficient conditions on the learning algorithm to enable dynamic safe interruptibility in the case of joint action learners, yet show that these conditions are not sufficient for independent learners.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
