Search Results for author:
Found 27 papers, 3 papers with code
A Symplectic Analysis of Alternating Mirror Descent
We derive new error bounds on the MH when truncated at orders in the stepsize in terms of the number of iterations, $K$, and utilize this bound to show an improved $\mathcal{O}(K^{1/5})$ total regret bound and an $\mathcal{O}(K^{-4/5})$ duality gap of the average iterates for AMD.
On Independent Samples Along the Langevin Diffusion and the Unadjusted Langevin Algorithm
We study the rate at which the initial and current random variables become independent along a Markov chain, focusing on the Langevin diffusion in continuous time and the Unadjusted Langevin Algorithm (ULA) in discrete time.
Optimal score estimation via empirical Bayes smoothing
We study the problem of estimating the score function of an unknown probability distribution $\rho^*$ from $n$ independent and identically distributed observations in $d$ dimensions.
Fast sampling from constrained spaces using the Metropolis-adjusted Mirror Langevin algorithm
This algorithm adds an accept-reject filter to the Markov chain induced by a single step of the Mirror Langevin algorithm (Zhang et al., 2020), which is a basic discretisation of the Mirror Langevin dynamics.
Extragradient Type Methods for Riemannian Variational Inequality Problems
In the context of Euclidean space, it is established that the last-iterates of both the extragradient (EG) and past extragradient (PEG) methods converge to the solution of monotone variational inequality problems at a rate of $O\left(\frac{1}{\sqrt{T}}\right)$ (Cai et al., 2022).
Mitigating Catastrophic Forgetting in Long Short-Term Memory Networks
Compared to state-of-the-art weight regularization methods to mitigate catastrophic forgetting, our approach is simple, effective, and enables faster learning.
Continuized Acceleration for Quasar Convex Functions in Non-Convex Optimization
Quasar convexity is a condition that allows some first-order methods to efficiently minimize a function even when the optimization landscape is non-convex.
Convergence of the Inexact Langevin Algorithm and Score-based Generative Models in KL Divergence
We study the Inexact Langevin Dynamics (ILD), Inexact Langevin Algorithm (ILA), and Score-based Generative Modeling (SGM) when utilizing estimated score functions for sampling.
Aggregation in the Mirror Space (AIMS): Fast, Accurate Distributed Machine Learning in Military Settings
Distributed machine learning (DML) can be an important capability for modern military to take advantage of data and devices distributed at multiple vantage points to adapt and learn.
Towards Understanding GD with Hard and Conjugate Pseudo-labels for Test-Time Adaptation
We consider a setting that a model needs to adapt to a new domain under distribution shifts, given that only unlabeled test samples from the new domain are accessible at test time.
Accelerating Hamiltonian Monte Carlo via Chebyshev Integration Time
When the potential $f$ is $L$-smooth and $m$-strongly convex, i. e.\ for sampling from a log-smooth and strongly log-concave target distribution $\pi$, it is known that under a constant integration time, the number of iterations that ideal HMC takes to get an $\epsilon$ Wasserstein-2 distance to the target $\pi$ is $O( \kappa \log \frac{1}{\epsilon} )$, where $\kappa := \frac{L}{m}$ is the condition number.
Provable Acceleration of Heavy Ball beyond Quadratics for a Class of Polyak-Łojasiewicz Functions when the Non-Convexity is Averaged-Out
An additional condition needs to be satisfied for the acceleration result of HB beyond quadratics in this work, which naturally holds when the dimension is one or, more broadly, when the Hessian is diagonal.
Alternating Mirror Descent for Constrained Min-Max Games
In this paper we study two-player bilinear zero-sum games with constrained strategy spaces.
Improved analysis for a proximal algorithm for sampling
We study the proximal sampler of Lee, Shen, and Tian (2021) and obtain new convergence guarantees under weaker assumptions than strong log-concavity: namely, our results hold for (1) weakly log-concave targets, and (2) targets satisfying isoperimetric assumptions which allow for non-log-concavity.
Achieving Efficient Distributed Machine Learning Using a Novel Non-Linear Class of Aggregation Functions
In this paper, we analyze properties of the WPM and rigorously prove convergence properties of our aggregation mechanism.
The Mirror Langevin Algorithm Converges with Vanishing Bias
The Mirror Langevin Diffusion (MLD) is a sampling analogue of mirror flow in continuous time, and it has nice convergence properties under log-Sobolev or Poincare inequalities relative to the Hessian metric, as shown by Chewi et al. (2020).
Proximal Langevin Algorithm: Rapid Convergence Under Isoperimetry
We study the Proximal Langevin Algorithm (PLA) for sampling from a probability distribution $\nu = e^{-f}$ on $\mathbb{R}^n$ under isoperimetry.
Last-iterate convergence rates for min-max optimization
While classic work in convex-concave min-max optimization relies on average-iterate convergence results, the emergence of nonconvex applications such as training Generative Adversarial Networks has led to renewed interest in last-iterate convergence guarantees.
Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Suffices
We also prove convergence guarantees in R\'enyi divergence of order $q > 1$ assuming the limit of ULA satisfies either the log-Sobolev or Poincar\'e inequality.
Accelerating Rescaled Gradient Descent: Fast Optimization of Smooth Functions
We also introduce a new first-order algorithm, called rescaled gradient descent (RGD), and show that RGD achieves a faster convergence rate than gradient descent provided the function is strongly smooth -- a natural generalization of the standard smoothness assumption on the objective function.
Optimization and Control
Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problem
We show SLA is in fact consistent for Gaussian target measure, whereas ULA is not.
A Variational Perspective on Accelerated Methods in Optimization
We show that there is a Lagrangian functional that we call the \emph{Bregman Lagrangian} which generates a large class of accelerated methods in continuous time, including (but not limited to) accelerated gradient descent, its non-Euclidean extension, and accelerated higher-order gradient methods.
Concavity of reweighted Kikuchi approximation
We establish sufficient conditions for the concavity of our reweighted objective function in terms of weight assignments in the Kikuchi expansion, and show that a reweighted version of the sum product algorithm applied to the Kikuchi region graph will produce global optima of the Kikuchi approximation whenever the algorithm converges.
Optimal rates for zero-order convex optimization: the power of two function evaluations
We consider derivative-free algorithms for stochastic and non-stochastic convex optimization problems that use only function values rather than gradients.
How to Hedge an Option Against an Adversary: Black-Scholes Pricing is Minimax Optimal
We consider a popular problem in finance, option pricing, through the lens of an online learning game between Nature and an Investor.
Streaming Variational Bayes
We present SDA-Bayes, a framework for (S)treaming, (D)istributed, (A)synchronous computation of a Bayesian posterior.
Finite Sample Convergence Rates of Zero-Order Stochastic Optimization Methods
We consider derivative-free algorithms for stochastic optimization problems that use only noisy function values rather than gradients, analyzing their finite-sample convergence rates.
