Search Results for author:
Found 17 papers, 3 papers with code
Error Exponent in Agnostic PAC Learning
In this paper, we consider PAC learning using a somewhat different tradeoff, the error exponent - a well established analysis method in Information Theory - which describes the exponential behavior of the probability that the risk will exceed a certain threshold as function of the sample size.
Batches Stabilize the Minimum Norm Risk in High Dimensional Overparameterized Linear Regression
Learning algorithms that divide the data into batches are prevalent in many machine-learning applications, typically offering useful trade-offs between computational efficiency and performance.
Beyond Ridge Regression for Distribution-Free Data
In the context of online prediction where the min-max solution is the Normalized Maximum Likelihood (NML), it has been suggested to use NML with ``luckiness'': A prior-like function is applied to the hypothesis class, which reduces its effective size.
Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detection
Furthermore, we describe how to efficiently apply the derived pNML regret to any pretrained deep NN, by employing the explicit pNML for the last layer, followed by the softmax function.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Utilizing Adversarial Targeted Attacks to Boost Adversarial Robustness
Specifically, our defense performs adversarial targeted attacks according to different hypotheses, where each hypothesis assumes a specific label for the test sample.
Distribution Free Uncertainty for the Minimum Norm Solution of Over-parameterized Linear Regression
Modern machine learning models do not obey this paradigm: They produce an accurate prediction even with a perfect fit to the training set.
Sequential prediction under log-loss and misspecification
The well-specified case corresponds to an additional assumption that the data-generating distribution belongs to the hypothesis class as well.
Efficient Data-Dependent Learnability
This approach has yields a learnability measure that can also be interpreted as a stability measure.
Can Implicit Bias Explain Generalization? Stochastic Convex Optimization as a Case Study
The notion of implicit bias, or implicit regularization, has been suggested as a means to explain the surprising generalization ability of modern-days overparameterized learning algorithms.
Universal Learning Approach for Adversarial Defense
In particular, we follow the recently suggested Predictive Normalized Maximum Likelihood (pNML) scheme for universal learning, whose goal is to optimally compete with a reference learner that knows the true label of the test sample but is restricted to use a learner from a given hypothesis class.
A New Look at an Old Problem: A Universal Learning Approach to Linear Regression
Linear regression is a classical paradigm in statistics.
Deep pNML: Predictive Normalized Maximum Likelihood for Deep Neural Networks
Finally, we extend the pNML to a ``twice universal'' solution, that provides universality for model class selection and generates a learner competing with the best one from all model classes.
Universal Supervised Learning for Individual Data
Universal supervised learning is considered from an information theoretic point of view following the universal prediction approach, see Merhav and Feder (1998).
Non-linear Canonical Correlation Analysis: A Compressed Representation Approach
In this work we introduce an information-theoretic compressed representation framework for the non-linear CCA problem (CRCCA), which extends the classical ACE approach.
Linear Independent Component Analysis over Finite Fields: Algorithms and Bounds
Importantly, we show that the overhead of our suggested algorithm (compared with the lower bound) typically decreases, as the scale of the problem grows.
Outperforming Good-Turing: Preliminary Report
Estimating a large alphabet probability distribution from a limited number of samples is a fundamental problem in machine learning and statistics.
Accumulation of nonlinear interference noise in fiber-optic systems
Through a series of extensive system simulations we show that all of the previously not understood discrepancies between the Gaussian noise (GN) model and simulations can be attributed to the omission of an important, recently reported, fourth-order noise (FON) term, that accounts for the statistical dependencies within the spectrum of the interfering channel.
Optics
