Search Results for author:
Found 48 papers, 9 papers with code
Bridging Domains with Approximately Shared Features
Under our framework, we design and analyze a learning procedure consisting of learning approximately shared feature representation from source tasks and fine-tuning it on the target task.
Controllable Prompt Tuning For Balancing Group Distributional Robustness
Models trained on data composed of different groups or domains can suffer from severe performance degradation under distribution shifts.
Data Reconstruction Attacks and Defenses: A Systematic Evaluation
In this work, we propose a strong reconstruction attack in the setting of federated learning.
An Information-Theoretic Analysis of In-Context Learning
Previous theoretical results pertaining to meta-learning on sequences build on contrived assumptions and are somewhat convoluted.
Beyond Gradient and Priors in Privacy Attacks: Leveraging Pooler Layer Inputs of Language Models in Federated Learning
Language models trained via federated learning (FL) demonstrate impressive capabilities in handling complex tasks while protecting user privacy.
Towards Robust Pruning: An Adaptive Knowledge-Retention Pruning Strategy for Language Models
The pruning objective has recently extended beyond accuracy and sparsity to robustness in language models.
Breaking through Deterministic Barriers: Randomized Pruning Mask Generation and Selection
It is widely acknowledged that large and sparse models have higher accuracy than small and dense models under the same model size constraints.
Sample Complexity for Quadratic Bandits: Hessian Dependent Bounds and Optimal Algorithms
We consider a fundamental setting in which the objective function is quadratic, and provide the first tight characterization of the optimal Hessian-dependent sample complexity.
Reconstructing Training Data from Model Gradient, Provably
Understanding when and how much a model gradient leaks information about the training sample is an important question in privacy.
Optimization for Amortized Inverse Problems
Incorporating a deep generative model as the prior distribution in inverse problems has established substantial success in reconstructing images from corrupted observations.
Efficient Medical Image Assessment via Self-supervised Learning
Existing data assessment methods commonly require knowing the labels in advance, which are not feasible to achieve our goal of 'knowing which data to label.'
Nearly Minimax Algorithms for Linear Bandits with Shared Representation
We give novel algorithms for multi-task and lifelong linear bandits with shared representation.
Sample Efficiency of Data Augmentation Consistency Regularization
Data augmentation is popular in the training of large neural networks; currently, however, there is no clear theoretical comparison between different algorithmic choices on how to use augmented data.
Bi-CLKT: Bi-Graph Contrastive Learning based Knowledge Tracing
The goal of Knowledge Tracing (KT) is to estimate how well students have mastered a concept based on their historical learning of related exercises.
PANOM: Automatic Hyper-parameter Tuning for Inverse Problems
Automated hyper-parameter tuning for unsupervised learning, including inverse problems, remains a long-standing open problem due to the lack of validation data.
Provable Hierarchy-Based Meta-Reinforcement Learning
To address this gap, we analyze HRL in the meta-RL setting, where a learner learns latent hierarchical structure during meta-training for use in a downstream task.
Theoretical Analysis of Consistency Regularization with Limited Augmented Data
Data augmentation is popular in the training of large neural networks; currently, however, there is no clear theoretical comparison between different algorithmic choices on how to use augmented data.
Going Beyond Linear RL: Sample Efficient Neural Function Approximation
While the theory of RL has traditionally focused on linear function approximation (or eluder dimension) approaches, little is known about nonlinear RL with neural net approximations of the Q functions.
Optimal Gradient-based Algorithms for Non-concave Bandit Optimization
This work considers a large family of bandit problems where the unknown underlying reward function is non-concave, including the low-rank generalized linear bandit problems and two-layer neural network with polynomial activation bandit problem.
A Short Note on the Relationship of Information Gain and Eluder Dimension
Eluder dimension and information gain are two widely used methods of complexity measures in bandit and reinforcement learning.
Near-Optimal Linear Regression under Distribution Shift
Transfer learning is essential when sufficient data comes from the source domain, with scarce labeled data from the target domain.
How Fine-Tuning Allows for Effective Meta-Learning
Representation learning has been widely studied in the context of meta-learning, enabling rapid learning of new tasks through shared representations.
A Theory of Label Propagation for Subpopulation Shift
In this work, we propose a provably effective framework for domain adaptation based on label propagation.
Compressed Sensing with Invertible Generative Models and Dependent Noise
We study image inverse problems with invertible generative priors, specifically normalizing flow models.
Fast Convergence of Langevin Dynamics on Manifold: Geodesics meet Log-Sobolev
Sampling is a fundamental and arguably very important task with numerous applications in Machine Learning.
Predicting What You Already Know Helps: Provable Self-Supervised Learning
Self-supervised representation learning solves auxiliary prediction tasks (known as pretext tasks) without requiring labeled data to learn useful semantic representations.
Steepest Descent Neural Architecture Optimization: Escaping Local Optimum with Signed Neural Splitting
Recently, Liu et al.[19] proposed a splitting steepest descent (S2D) method that jointly optimizes the neural parameters and architectures based on progressively growing network structures by splitting neurons into multiple copies in a steepest descent fashion.
Solving Inverse Problems with a Flow-based Noise Model
We study image inverse problems with a normalizing flow prior.
Few-Shot Learning via Learning the Representation, Provably
First, we study the setting where this common representation is low-dimensional and provide a fast rate of $O\left(\frac{\mathcal{C}\left(\Phi\right)}{n_1T} + \frac{k}{n_2}\right)$; here, $\Phi$ is the representation function class, $\mathcal{C}\left(\Phi\right)$ is its complexity measure, and $k$ is the dimension of the representation.
CAT: Customized Adversarial Training for Improved Robustness
Adversarial training has become one of the most effective methods for improving robustness of neural networks.
Last iterate convergence in no-regret learning: constrained min-max optimization for convex-concave landscapes
In a recent series of papers it has been established that variants of Gradient Descent/Ascent and Mirror Descent exhibit last iterate convergence in convex-concave zero-sum games.
Primal-Dual Block Generalized Frank-Wolfe
We propose a generalized variant of Frank-Wolfe algorithm for solving a class of sparse/low-rank optimization problems.
Communication-Efficient Asynchronous Stochastic Frank-Wolfe over Nuclear-norm Balls
Large-scale machine learning training suffers from two prior challenges, specifically for nuclear-norm constrained problems with distributed systems: the synchronization slowdown due to the straggling workers, and high communication costs.
SGD Learns One-Layer Networks in WGANs
Generative adversarial networks (GANs) are a widely used framework for learning generative models.
Inverting Deep Generative models, One layer at a time
For generative models of arbitrary depth, we show that exact recovery is possible in polynomial time with high probability, if the layers are expanding and the weights are randomly selected.
Primal-Dual Block Frank-Wolfe
We propose a variant of the Frank-Wolfe algorithm for solving a class of sparse/low-rank optimization problems.
Discrete Adversarial Attacks and Submodular Optimization with Applications to Text Classification
In this paper we formulate the attacks with discrete input on a set function as an optimization task.
Random Warping Series: A Random Features Method for Time-Series Embedding
The proposed kernel does not suffer from the issue of diagonal dominance while naturally enjoys a \emph{Random Features} (RF) approximation, which reduces the computational complexity of existing DTW-based techniques from quadratic to linear in terms of both the number and the length of time-series.
Stabilizing Gradients for Deep Neural Networks via Efficient SVD Parameterization
Theoretically, we demonstrate that our parameterization does not lose any expressive power, and show how it controls generalization of RNN for the classification task.
Hessian-based Analysis of Large Batch Training and Robustness to Adversaries
Extensive experiments on multiple networks show that saddle-points are not the cause for generalization gap of large batch size training, and the results consistently show that large batch converges to points with noticeably higher Hessian spectrum.
Gradient Coding: Avoiding Stragglers in Distributed Learning
We propose a novel coding theoretic framework for mitigating stragglers in distributed learning.
Doubly Greedy Primal-Dual Coordinate Descent for Sparse Empirical Risk Minimization
We consider the popular problem of sparse empirical risk minimization with linear predictors and a large number of both features and observations.
Negative-Unlabeled Tensor Factorization for Location Category Inference from Highly Inaccurate Mobility Data
In order to efficiently solve the proposed framework, we propose a parameter-free and scalable optimization algorithm by effectively exploring the sparse and low-rank structure of the tensor.
Similarity Preserving Representation Learning for Time Series Clustering
A considerable amount of clustering algorithms take instance-feature matrices as their inputs.
Gradient Coding
We propose a novel coding theoretic framework for mitigating stragglers in distributed learning.
Coordinate-wise Power Method
The vanilla power method simultaneously updates all the coordinates of the iterate, which is essential for its convergence analysis.
A Greedy Approach for Budgeted Maximum Inner Product Search
Maximum Inner Product Search (MIPS) is an important task in many machine learning applications such as the prediction phase of a low-rank matrix factorization model for a recommender system.
Coordinate Descent Methods for Symmetric Nonnegative Matrix Factorization
Given a symmetric nonnegative matrix $A$, symmetric nonnegative matrix factorization (symNMF) is the problem of finding a nonnegative matrix $H$, usually with much fewer columns than $A$, such that $A \approx HH^T$.
