Search Results for author:
Found 45 papers, 11 papers with code
A Nearly-Linear Time Algorithm for Exact Community Recovery in Stochastic Block Model
In this paper, we focus on the problem of exactly recovering the communities in a binary symmetric SBM, where a graph of $n$ vertices is partitioned into two equal-sized communities and the vertices are connected with probability $p = \alpha\log(n)/n$ within communities and $q = \beta\log(n)/n$ across communities for some $\alpha>\beta>0$.
Spurious Stationarity and Hardness Results for Mirror Descent
Despite the considerable success of Bregman proximal-type algorithms, such as mirror descent, in machine learning, a critical question remains: Can existing stationarity measures, often based on Bregman divergence, reliably distinguish between stationary and non-stationary points?
Extreme Point Pursuit -- Part I: A Framework for Constant Modulus Optimization
This study develops a framework for a class of constant modulus (CM) optimization problems, which covers binary constraints, discrete phase constraints, semi-orthogonal matrix constraints, non-negative semi-orthogonal matrix constraints, and several types of binary assignment constraints.
Extreme Point Pursuit -- Part II: Further Error Bound Analysis and Applications
In the first part of this study, a convex-constrained penalized formulation was studied for a class of constant modulus (CM) problems.
On the Estimation Performance of Generalized Power Method for Heteroscedastic Probabilistic PCA
The heteroscedastic probabilistic principal component analysis (PCA) technique, a variant of the classic PCA that considers data heterogeneity, is receiving more and more attention in the data science and signal processing communities.
ReSync: Riemannian Subgradient-based Robust Rotation Synchronization
This work presents ReSync, a Riemannian subgradient-based algorithm for solving the robust rotation synchronization problem, which arises in various engineering applications.
Revisiting Subgradient Method: Complexity and Convergence Beyond Lipschitz Continuity
In particular, when $f$ is convex, we show $\mathcal{O}(\log(k)/\sqrt{k})$ rate of convergence in terms of the suboptimality gap.
Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder
Grounded on our analysis, we propose a novel partial attention language model to solve the attention degeneration problem.
Decentralized Weakly Convex Optimization Over the Stiefel Manifold
We focus on a class of non-smooth optimization problems over the Stiefel manifold in the decentralized setting, where a connected network of $n$ agents cooperatively minimize a finite-sum objective function with each component being weakly convex in the ambient Euclidean space.
A Convergent Single-Loop Algorithm for Relaxation of Gromov-Wasserstein in Graph Data
This observation allows us to provide an approximation bound for the distance between the fixed-point set of BAPG and the critical point set of GW.
Testing Stationarity Concepts for ReLU Networks: Hardness, Regularity, and Robust Algorithms
This implies that testing a certain stationarity concept for a modern nonsmooth neural network is in general computationally intractable.
Outlier-Robust Gromov-Wasserstein for Graph Data
Gromov-Wasserstein (GW) distance is a powerful tool for comparing and aligning probability distributions supported on different metric spaces.
A Stability Analysis of Fine-Tuning a Pre-Trained Model
The theoretical bounds explain why and how several existing methods can stabilize the fine-tuning procedure.
Nonsmooth Nonconvex-Nonconcave Minimax Optimization: Primal-Dual Balancing and Iteration Complexity Analysis
Specifically, we consider the setting where the primal function has a nonsmooth composite structure and the dual function possesses the Kurdyka-Lojasiewicz (KL) property with exponent $\theta \in [0, 1)$.
Riemannian Natural Gradient Methods
Moreover, we show that if the loss function satisfies certain convexity and smoothness conditions and the input-output map satisfies a Riemannian Jacobian stability condition, then our proposed method enjoys a local linear -- or, under the Lipschitz continuity of the Riemannian Jacobian of the input-output map, even quadratic -- rate of convergence.
Convergence and Recovery Guarantees of the K-Subspaces Method for Subspace Clustering
The K-subspaces (KSS) method is a generalization of the K-means method for subspace clustering.
Fast and Provably Convergent Algorithms for Gromov-Wasserstein in Graph Data
In this paper, we study the design and analysis of a class of efficient algorithms for computing the Gromov-Wasserstein (GW) distance tailored to large-scale graph learning tasks.
Exact Community Recovery over Signed Graphs
Signed graphs encode similarity and dissimilarity relationships among different entities with positive and negative edges.
Non-Convex Joint Community Detection and Group Synchronization via Generalized Power Method
This paper proposes a Generalized Power Method (GPM) to tackle the problem of community detection and group synchronization simultaneously in a direct non-convex manner.
Orthogonal Group Synchronization with Incomplete Measurements: Error Bounds and Linear Convergence of the Generalized Power Method
In this paper, we focus on the orthogonal group synchronization problem with general additive noise models under incomplete measurements, which is much more general than the commonly considered setting of complete measurements.
Accelerating Perturbed Stochastic Iterates in Asynchronous Lock-Free Optimization
We show that stochastic acceleration can be achieved under the perturbed iterate framework (Mania et al., 2017) in asynchronous lock-free optimization, which leads to the optimal incremental gradient complexity for finite-sum objectives.
SISAL Revisited
By establishing associations between the SISAL algorithm and a line-search-based proximal gradient method, we confirm that SISAL can indeed guarantee convergence to a stationary point.
Practical Schemes for Finding Near-Stationary Points of Convex Finite-Sums
In convex optimization, the problem of finding near-stationary points has not been adequately studied yet, unlike other optimality measures such as the function value.
Distributionally Robust Graph Learning from Smooth Signals under Moment Uncertainty
To address this issue, we propose a novel graph learning model based on the distributionally robust optimization methodology, which aims to identify a graph that not only provides a smooth representation of but is also robust against uncertainties in the observed signals.
Probabilistic Simplex Component Analysis
PRISM uses a simple probabilistic model, namely, uniform simplex data distribution and additive Gaussian noise, and it carries out inference by maximum likelihood.
A Theoretical Analysis of the Repetition Problem in Text Generation
The experimental results show that our theoretical framework is applicable in general generation models and our proposed rebalanced encoding approach alleviates the repetition problem significantly.
Fast Epigraphical Projection-based Incremental Algorithms for Wasserstein Distributionally Robust Support Vector Machine
In this paper, we focus on a family of Wasserstein distributionally robust support vector machine (DRSVM) problems and propose two novel epigraphical projection-based incremental algorithms to solve them.
A Unified Approach to Synchronization Problems over Subgroups of the Orthogonal Group
In this paper, we consider the class of synchronization problems in which the group is a closed subgroup of the orthogonal group.
Sparse High-Order Portfolios via Proximal DCA and SCA
Optimization for the MVSKC model is of great difficulty in two parts.
Non-Convex Exact Community Recovery in Stochastic Block Model
Community detection in graphs that are generated according to stochastic block models (SBMs) has received much attention lately.
Understanding Notions of Stationarity in Non-Smooth Optimization
Many contemporary applications in signal processing and machine learning give rise to structured non-convex non-smooth optimization problems that can often be tackled by simple iterative methods quite effectively.
Boosting First-Order Methods by Shifting Objective: New Schemes with Faster Worst-Case Rates
Specifically, instead of tackling the original objective directly, we construct a shifted objective function that has the same minimizer as the original objective and encodes both the smoothness and strong convexity of the original objective in an interpolation condition.
Manifold Proximal Point Algorithms for Dual Principal Component Pursuit and Orthogonal Dictionary Learning
Second, we propose a stochastic variant of ManPPA called StManPPA, which is well suited for large-scale computation, and establish its sublinear convergence rate.
A First-Order Algorithmic Framework for Distributionally Robust Logistic Regression
In this paper, we take a first step towards resolving the above difficulty by developing a first-order algorithmic framework for tackling a class of Wasserstein distance-based distributionally robust logistic regression (DRLR) problem.
A First-Order Algorithmic Framework for Wasserstein Distributionally Robust Logistic Regression
In this paper, we take a first step towards resolving the above difficulty by developing a first-order algorithmic framework for tackling a class of Wasserstein distance-based distributionally robust logistic regression (DRLR) problem.
Voting-Based Multi-Agent Reinforcement Learning for Intelligent IoT
The recent success of single-agent reinforcement learning (RL) in Internet of things (IoT) systems motivates the study of multi-agent reinforcement learning (MARL), which is more challenging but more useful in large-scale IoT.
An Efficient Augmented Lagrangian Based Method for Constrained Lasso
Variable selection is one of the most important tasks in statistics and machine learning.
Proximal Gradient Method for Nonsmooth Optimization over the Stiefel Manifold
We prove that the proposed method globally converges to a stationary point.
Nonconvex Robust Low-rank Matrix Recovery
In this paper we study the problem of recovering a low-rank matrix from a number of random linear measurements that are corrupted by outliers taking arbitrary values.
Information Theory Information Theory
Scalable and Flexible Multiview MAX-VAR Canonical Correlation Analysis
Generalized canonical correlation analysis (GCCA) aims at finding latent low-dimensional common structure from multiple views (feature vectors in different domains) of the same entities.
A Unified Approach to Error Bounds for Structured Convex Optimization Problems
In this paper, we present a new framework for establishing error bounds for a class of structured convex optimization problems, in which the objective function is the sum of a smooth convex function and a general closed proper convex function.
Quadratic Optimization with Orthogonality Constraints: Explicit Lojasiewicz Exponent and Linear Convergence of Line-Search Methods
To determine the convergence rate of these methods, we give an explicit estimate of the exponent in a Lojasiewicz inequality for the (non-convex) set of critical points of the aforementioned class of problems.
On the Linear Convergence of the Proximal Gradient Method for Trace Norm Regularization
Motivated by various applications in machine learning, the problem of minimizing a convex smooth loss function with trace norm regularization has received much attention lately.
Non-Asymptotic Convergence Analysis of Inexact Gradient Methods for Machine Learning Without Strong Convexity
In this paper, we combine the best of these two types of results and establish---under the standard assumption that the gradient approximation errors decrease linearly to zero---the non-asymptotic linear convergence of IGMs when applied to a class of structured convex optimization problems.
