Search Results for author:
Found 12 papers, 4 papers with code
ALEXR: An Optimal Single-Loop Algorithm for Convex Finite-Sum Coupled Compositional Stochastic Optimization
This paper revisits a class of convex Finite-Sum Coupled Compositional Stochastic Optimization (cFCCO) problems with many applications, including group distributionally robust optimization (GDRO), learning with imbalanced data, reinforcement learning, and learning to rank.
Everything Perturbed All at Once: Enabling Differentiable Graph Attacks
As powerful tools for representation learning on graphs, graph neural networks (GNNs) have played an important role in applications including social networks, recommendation systems, and online web services.
Provable Multi-instance Deep AUC Maximization with Stochastic Pooling
This paper considers a novel application of deep AUC maximization (DAM) for multi-instance learning (MIL), in which a single class label is assigned to a bag of instances (e. g., multiple 2D slices of a CT scan for a patient).
GraphFM: Improving Large-Scale GNN Training via Feature Momentum
GraphFM-IB applies FM to in-batch sampled data, while GraphFM-OB applies FM to out-of-batch data that are 1-hop neighborhood of in-batch data.
When AUC meets DRO: Optimizing Partial AUC for Deep Learning with Non-Convex Convergence Guarantee
In this paper, we propose systematic and efficient gradient-based methods for both one-way and two-way partial AUC (pAUC) maximization that are applicable to deep learning.
Finite-Sum Coupled Compositional Stochastic Optimization: Theory and Applications
This paper studies stochastic optimization for a sum of compositional functions, where the inner-level function of each summand is coupled with the corresponding summation index.
Optimal Algorithms for Stochastic Multi-Level Compositional Optimization
To address these limitations, we propose a Stochastic Multi-level Variance Reduction method (SMVR), which achieves the optimal sample complexity of $\mathcal{O}\left(1 / \epsilon^{3}\right)$ to find an $\epsilon$-stationary point for non-convex objectives.
Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning
The proposed algorithms require sampling a constant number of tasks and data samples per iteration, making them suitable for the continual learning scenario.
Theoretically Better and Numerically Faster Distributed Optimization with Smoothness-Aware Quantization Techniques
To address the high communication costs of distributed machine learning, a large body of work has been devoted in recent years to designing various compression strategies, such as sparsification and quantization, and optimization algorithms capable of using them.
IntSGD: Adaptive Floatless Compression of Stochastic Gradients
We propose a family of adaptive integer compression operators for distributed Stochastic Gradient Descent (SGD) that do not communicate a single float.
Riemannian Stochastic Proximal Gradient Methods for Nonsmooth Optimization over the Stiefel Manifold
However, most of the existing Riemannian stochastic algorithms require the objective function to be differentiable, and they do not apply to the case where the objective function is nonsmooth.
Towards Fair Deep Clustering With Multi-State Protected Variables
Fair clustering under the disparate impact doctrine requires that population of each protected group should be approximately equal in every cluster.
