Search Results for author:
Found 15 papers, 2 papers with code
Noise misleads rotation invariant algorithms on sparse targets
It is well known that the class of rotation invariant algorithms are suboptimal even for learning sparse linear problems when the number of examples is below the "dimension" of the problem.
Consistent algorithms for multi-label classification with macro-at-$k$ metrics
We consider the optimization of complex performance metrics in multi-label classification under the population utility framework.
Generalized test utilities for long-tail performance in extreme multi-label classification
As such, it is characterized by long-tail labels, i. e., most labels have very few positive instances.
Learning from Randomly Initialized Neural Network Features
We present the surprising result that randomly initialized neural networks are good feature extractors in expectation.
Robust Online Convex Optimization in the Presence of Outliers
If the outliers are chosen adversarially, we show that a simple filtering strategy on extreme gradients incurs O(k) additive overhead compared to the usual regret bounds, and that this is unimprovable, which means that k needs to be sublinear in the number of rounds.
A case where a spindly two-layer linear network whips any neural network with a fully connected input layer
It was conjectured that any neural network of any structure and arbitrary differentiable transfer functions at the nodes cannot learn the following problem sample efficiently when trained with gradient descent: The instances are the rows of a $d$-dimensional Hadamard matrix and the target is one of the features, i. e. very sparse.
Adaptive scale-invariant online algorithms for learning linear models
We consider online learning with linear models, where the algorithm predicts on sequentially revealed instances (feature vectors), and is compared against the best linear function (comparator) in hindsight.
Bandit Principal Component Analysis
We consider a partial-feedback variant of the well-studied online PCA problem where a learner attempts to predict a sequence of $d$-dimensional vectors in terms of a quadratic loss, while only having limited feedback about the environment's choices.
The Many Faces of Exponential Weights in Online Learning
A standard introduction to online learning might place Online Gradient Descent at its center and then proceed to develop generalizations and extensions like Online Mirror Descent and second-order methods.
Scale-invariant unconstrained online learning
We first give a negative result, showing that no algorithm can achieve a meaningful bound in terms of scale-invariant norm of the comparator in the worst case.
Consistency Analysis for Binary Classification Revisited
Statistical learning theory is at an inflection point enabled by recent advances in understanding and optimizing a wide range of metrics.
Online Isotonic Regression
We then prove that the Exponential Weights algorithm played over a covering net of isotonic functions has a regret bounded by $O\big(T^{1/3} \log^{2/3}(T)\big)$ and present a matching $\Omega(T^{1/3})$ lower bound on regret.
PCA with Gaussian perturbations
We develop a simple algorithm that needs $O(kn^2)$ per trial whose regret is off by a small factor of $O(n^{1/4})$.
Surrogate regret bounds for generalized classification performance metrics
We show that the regret of the resulting classifier (obtained from thresholding $f$ on $\widehat{\theta}$) measured with respect to the target metric is upperbounded by the regret of $f$ measured with respect to the surrogate loss.
Consistent optimization of AMS by logistic loss minimization
First, a real-valued function is learned by minimizing a surrogate loss for binary classification, such as logistic loss, on the training sample.
