Search Results for author:
Found 23 papers, 2 papers with code
Just Wing It: Optimal Estimation of Missing Mass in a Markovian Sequence
Operating in the general setting in which the size of the state space may be much larger than the length $n$ of the trajectory, we develop a linear-runtime estimator called \emph{Windowed Good--Turing} (\textsc{WingIt}) and show that its risk decays as $\widetilde{\mathcal{O}}(\mathsf{T_{mix}}/n)$, where $\mathsf{T_{mix}}$ denotes the mixing time of the chain in total variation distance.
Balanced Data, Imbalanced Spectra: Unveiling Class Disparities with Spectral Imbalance
Classification models are expected to perform equally well for different classes, yet in practice, there are often large gaps in their performance.
Faster Margin Maximization Rates for Generic and Adversarially Robust Optimization Methods
To address this limitation, we present a series of state-of-the-art implicit bias rates for mirror descent and steepest descent algorithms.
New Equivalences Between Interpolation and SVMs: Kernels and Structured Features
The support vector machine (SVM) is a supervised learning algorithm that finds a maximum-margin linear classifier, often after mapping the data to a high-dimensional feature space via the kernel trick.
General Loss Functions Lead to (Approximate) Interpolation in High Dimensions
We provide a unified framework, applicable to a general family of convex losses and across binary and multiclass settings in the overparameterized regime, to approximately characterize the implicit bias of gradient descent in closed form.
Estimating Optimal Policy Value in General Linear Contextual Bandits
Whether this is possible for more realistic context distributions has remained an open and important question for tasks such as model selection.
Adaptive Oracle-Efficient Online Learning
The classical algorithms for online learning and decision-making have the benefit of achieving the optimal performance guarantees, but suffer from computational complexity limitations when implemented at scale.
The good, the bad and the ugly sides of data augmentation: An implicit spectral regularization perspective
Data augmentation (DA) is a powerful workhorse for bolstering performance in modern machine learning.
Harmless interpolation in regression and classification with structured features
Our results recover prior independent-features results (with a much simpler analysis), but they furthermore show that harmless interpolation can occur in more general settings such as features that are a bounded orthonormal system.
Universal and data-adaptive algorithms for model selection in linear contextual bandits
In this paper, we introduce new algorithms that a) explore in a data-adaptive manner, and b) provide model selection guarantees of the form $\mathcal{O}(d^{\alpha} T^{1- \alpha})$ with no feature diversity conditions whatsoever, where $d$ denotes the dimension of the linear model and $T$ denotes the total number of rounds.
Classification and Adversarial examples in an Overparameterized Linear Model: A Signal Processing Perspective
We find that the learned model is susceptible to adversaries in an intermediate regime where classification generalizes but regression does not.
A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning
The rapid recent progress in machine learning (ML) has raised a number of scientific questions that challenge the longstanding dogma of the field.
Learning from an Exploring Demonstrator: Optimal Reward Estimation for Bandits
We introduce the "inverse bandit" problem of estimating the rewards of a multi-armed bandit instance from observing the learning process of a low-regret demonstrator.
Benign Overfitting in Multiclass Classification: All Roads Lead to Interpolation
The literature on "benign overfitting" in overparameterized models has been mostly restricted to regression or binary classification; however, modern machine learning operates in the multiclass setting.
On the Impossibility of Convergence of Mixed Strategies with No Regret Learning
We study the limiting behavior of the mixed strategies that result from optimal no-regret learning strategies in a repeated game setting where the stage game is any 2 by 2 competitive game.
Online Model Selection for Reinforcement Learning with Function Approximation
Towards this end, we consider the problem of model selection in RL with function approximation, given a set of candidate RL algorithms with known regret guarantees.
On the proliferation of support vectors in high dimensions
The support vector machine (SVM) is a well-established classification method whose name refers to the particular training examples, called support vectors, that determine the maximum margin separating hyperplane.
Classification vs regression in overparameterized regimes: Does the loss function matter?
We compare classification and regression tasks in an overparameterized linear model with Gaussian features.
OSOM: A simultaneously optimal algorithm for multi-armed and linear contextual bandits
We consider the stochastic linear (multi-armed) contextual bandit problem with the possibility of hidden simple multi-armed bandit structure in which the rewards are independent of the contextual information.
Harmless interpolation of noisy data in regression
A continuing mystery in understanding the empirical success of deep neural networks is their ability to achieve zero training error and generalize well, even when the training data is noisy and there are more parameters than data points.
Understanding Unequal Gender Classification Accuracy from Face Images
Recent work shows unequal performance of commercial face classification services in the gender classification task across intersectional groups defined by skin type and gender.
Best of many worlds: Robust model selection for online supervised learning
We introduce algorithms for online, full-information prediction that are competitive with contextual tree experts of unknown complexity, in both probabilistic and adversarial settings.
Worst-case vs Average-case Design for Estimation from Fixed Pairwise Comparisons
We show that when the assignment of items to the topology is arbitrary, these permutation-based models, unlike their parametric counterparts, do not admit consistent estimation for most comparison topologies used in practice.
