Search Results for author:
Found 74 papers, 6 papers with code
The Power of Resets in Online Reinforcement Learning
We use local simulator access to unlock new statistical guarantees that were previously out of reach: - We show that MDPs with low coverability Xie et al. 2023 -- a general structural condition that subsumes Block MDPs and Low-Rank MDPs -- can be learned in a sample-efficient fashion with only $Q^{\star}$-realizability (realizability of the optimal state-value function); existing online RL algorithms require significantly stronger representation conditions.
Online Estimation via Offline Estimation: An Information-Theoretic Framework
Our main results settle the statistical and computational complexity of online estimation in this framework.
Offline Reinforcement Learning: Role of State Aggregation and Trajectory Data
We revisit the problem of offline reinforcement learning with value function realizability but without Bellman completeness.
On the Performance of Empirical Risk Minimization with Smoothed Data
In order to circumvent statistical and computational hardness results in sequential decision-making, recent work has considered smoothed online learning, where the distribution of data at each time is assumed to have bounded likeliehood ratio with respect to a base measure when conditioned on the history.
Foundations of Reinforcement Learning and Interactive Decision Making
These lecture notes give a statistical perspective on the foundations of reinforcement learning and interactive decision making.
Efficient Model-Free Exploration in Low-Rank MDPs
A major challenge in reinforcement learning is to develop practical, sample-efficient algorithms for exploration in high-dimensional domains where generalization and function approximation is required.
On the Complexity of Multi-Agent Decision Making: From Learning in Games to Partial Monitoring
Compared to the best results for the single-agent setting, our bounds have additional gaps.
Representation Learning with Multi-Step Inverse Kinematics: An Efficient and Optimal Approach to Rich-Observation RL
We address these issues by providing the first computationally efficient algorithm that attains rate-optimal sample complexity with respect to the desired accuracy level, with minimal statistical assumptions.
Tight Bounds for $γ$-Regret via the Decision-Estimation Coefficient
Our lower bound shows that the $\gamma$-DEC is a fundamental limit for any model class $\mathcal{F}$: for any algorithm, there exists some $f \in \mathcal{F}$ for which the $\gamma$-regret of that algorithm scales (nearly) with the $\gamma$-DEC of $\mathcal{F}$.
Oracle-Efficient Smoothed Online Learning for Piecewise Continuous Decision Making
Smoothed online learning has emerged as a popular framework to mitigate the substantial loss in statistical and computational complexity that arises when one moves from classical to adversarial learning.
On the Complexity of Adversarial Decision Making
A central problem in online learning and decision making -- from bandits to reinforcement learning -- is to understand what modeling assumptions lead to sample-efficient learning guarantees.
Damped Online Newton Step for Portfolio Selection
In this paper, we build on the recent work by Haipeng et al. 2018 and present the first practical online portfolio selection algorithm with a logarithmic regret and whose per-round time and space complexities depend only logarithmically on the horizon.
Smoothed Online Learning is as Easy as Statistical Learning
We then prove a lower bound on the oracle complexity of any proper learning algorithm, which matches the oracle-efficient upper bounds up to a polynomial factor, thus demonstrating the existence of a statistical-computational gap in smooth online learning.
The Statistical Complexity of Interactive Decision Making
The main result of this work provides a complexity measure, the Decision-Estimation Coefficient, that is proven to be both necessary and sufficient for sample-efficient interactive learning.
On Submodular Contextual Bandits
We consider the problem of contextual bandits where actions are subsets of a ground set and mean rewards are modeled by an unknown monotone submodular function that belongs to a class $\mathcal{F}$.
Intrinsic Dimension Estimation Using Wasserstein Distances
It has long been thought that high-dimensional data encountered in many practical machine learning tasks have low-dimensional structure, i. e., the manifold hypothesis holds.
Deep learning: a statistical viewpoint
We conjecture that specific principles underlie these phenomena: that overparametrization allows gradient methods to find interpolating solutions, that these methods implicitly impose regularization, and that overparametrization leads to benign overfitting.
On the Minimal Error of Empirical Risk Minimization
We study the minimal error of the Empirical Risk Minimization (ERM) procedure in the task of regression, both in the random and the fixed design settings.
Top-$k$ eXtreme Contextual Bandits with Arm Hierarchy
We show that our algorithm has a regret guarantee of $O(k\sqrt{(A-k+1)T \log (|\mathcal{F}|T)})$, where $A$ is the total number of arms and $\mathcal{F}$ is the class containing the regression function, while only requiring $\tilde{O}(A)$ computation per time step.
Computational Efficiency
Extreme Multi-Label Classification
+2
Learning the Linear Quadratic Regulator from Nonlinear Observations
We introduce a new algorithm, RichID, which learns a near-optimal policy for the RichLQR with sample complexity scaling only with the dimension of the latent state space and the capacity of the decoder function class.
Instance-Dependent Complexity of Contextual Bandits and Reinforcement Learning: A Disagreement-Based Perspective
In the classical multi-armed bandit problem, instance-dependent algorithms attain improved performance on "easy" problems with a gap between the best and second-best arm.
Fast Mixing of Multi-Scale Langevin Dynamics under the Manifold Hypothesis
Recently, the task of image generation has attracted much attention.
On Suboptimality of Least Squares with Application to Estimation of Convex Bodies
We develop a technique for establishing lower bounds on the sample complexity of Least Squares (or, Empirical Risk Minimization) for large classes of functions.
Learning nonlinear dynamical systems from a single trajectory
We introduce algorithms for learning nonlinear dynamical systems of the form $x_{t+1}=\sigma(\Theta^{\star}x_t)+\varepsilon_t$, where $\Theta^{\star}$ is a weight matrix, $\sigma$ is a nonlinear link function, and $\varepsilon_t$ is a mean-zero noise process.
Beyond UCB: Optimal and Efficient Contextual Bandits with Regression Oracles
We characterize the minimax rates for contextual bandits with general, potentially nonparametric function classes, and show that our algorithm is minimax optimal whenever the oracle obtains the optimal rate for regression.
Generative Modeling with Denoising Auto-Encoders and Langevin Sampling
We show that both DAE and DSM provide estimates of the score of the Gaussian smoothed population density, allowing us to apply the machinery of Empirical Processes.
$\ell_{\infty}$ Vector Contraction for Rademacher Complexity
We show that the Rademacher complexity of any $\mathbb{R}^{K}$-valued function class composed with an $\ell_{\infty}$-Lipschitz function is bounded by the maximum Rademacher complexity of the restriction of the function class along each coordinate, times a factor of $\tilde{O}(\sqrt{K})$.
On the Multiple Descent of Minimum-Norm Interpolants and Restricted Lower Isometry of Kernels
We study the risk of minimum-norm interpolants of data in Reproducing Kernel Hilbert Spaces.
Using effective dimension to analyze feature transformations in deep neural networks
In a typical deep learning approach to a computer vision task, Convolutional Neural Networks (CNNs) are used to extract features at varying levels of abstraction from an image and compress a high dimensional input into a lower dimensional decision space through a series of transformations.
Breast Tumor Cellularity Assessment using Deep Neural Networks
Breast cancer is one of the main causes of death worldwide.
Optimality of Maximum Likelihood for Log-Concave Density Estimation and Bounded Convex Regression
In this paper, we study two problems: (1) estimation of a $d$-dimensional log-concave distribution and (2) bounded multivariate convex regression with random design with an underlying log-concave density or a compactly supported distribution with a continuous density.
Consistency of Interpolation with Laplace Kernels is a High-Dimensional Phenomenon
We show that minimum-norm interpolation in the Reproducing Kernel Hilbert Space corresponding to the Laplace kernel is not consistent if input dimension is constant.
Just Interpolate: Kernel "Ridgeless" Regression Can Generalize
In the absence of explicit regularization, Kernel "Ridgeless" Regression with nonlinear kernels has the potential to fit the training data perfectly.
Does data interpolation contradict statistical optimality?
We show that learning methods interpolating the training data can achieve optimal rates for the problems of nonparametric regression and prediction with square loss.
Angiodysplasia Detection and Localization Using Deep Convolutional Neural Networks
Accurate detection and localization for angiodysplasia lesions is an important problem in early stage diagnostics of gastrointestinal bleeding and anemia.
Online Learning: Sufficient Statistics and the Burkholder Method
We uncover a fairly general principle in online learning: If regret can be (approximately) expressed as a function of certain "sufficient statistics" for the data sequence, then there exists a special Burkholder function that 1) can be used algorithmically to achieve the regret bound and 2) only depends on these sufficient statistics, not the entire data sequence, so that the online strategy is only required to keep the sufficient statistics in memory.
Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learning
Semantic segmentation of robotic instruments is an important problem for the robot-assisted surgery.
Deep Convolutional Neural Networks for Breast Cancer Histology Image Analysis
In this work, we develop the computational approach based on deep convolution neural networks for breast cancer histology image classification.
Theory of Deep Learning IIb: Optimization Properties of SGD
In Theory IIb we characterize with a mix of theory and experiments the optimization of deep convolutional networks by Stochastic Gradient Descent.
Size-Independent Sample Complexity of Neural Networks
We study the sample complexity of learning neural networks, by providing new bounds on their Rademacher complexity assuming norm constraints on the parameter matrix of each layer.
Pediatric Bone Age Assessment Using Deep Convolutional Neural Networks
Skeletal bone age assessment is a common clinical practice to diagnose endocrine and metabolic disorders in child development.
Fisher-Rao Metric, Geometry, and Complexity of Neural Networks
We study the relationship between geometry and capacity measures for deep neural networks from an invariance viewpoint.
Weighted Message Passing and Minimum Energy Flow for Heterogeneous Stochastic Block Models with Side Information
We develop an optimally weighted message passing algorithm to reconstruct labels for SBM based on the minimum energy flow and the eigenvectors of a certain Markov transition matrix.
ZigZag: A new approach to adaptive online learning
To develop a general theory of when this type of adaptive regret bound is achievable we establish a connection to the theory of decoupling inequalities for martingales in Banach spaces.
Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis
Stochastic Gradient Langevin Dynamics (SGLD) is a popular variant of Stochastic Gradient Descent, where properly scaled isotropic Gaussian noise is added to an unbiased estimate of the gradient at each iteration.
A Tutorial on Online Supervised Learning with Applications to Node Classification in Social Networks
We revisit the elegant observation of T. Cover '65 which, perhaps, is not as well-known to the broader community as it should be.
On Detection and Structural Reconstruction of Small-World Random Networks
In this paper, we study detection and fast reconstruction of the celebrated Watts-Strogatz (WS) small-world random graph model \citep{watts1998collective} which aims to describe real-world complex networks that exhibit both high clustering and short average length properties.
Inference via Message Passing on Partially Labeled Stochastic Block Models
We study the community detection and recovery problem in partially-labeled stochastic block models (SBM).
Distributed Estimation of Dynamic Parameters : Regret Analysis
To this end, we use a notion of dynamic regret which suits the online, non-stationary nature of the problem.
BISTRO: An Efficient Relaxation-Based Method for Contextual Bandits
We present efficient algorithms for the problem of contextual bandits with i. i. d.
On Equivalence of Martingale Tail Bounds and Deterministic Regret Inequalities
We study an equivalence of (i) deterministic pathwise statements appearing in the online learning literature (termed \emph{regret bounds}), (ii) high-probability tail bounds for the supremum of a collection of martingales (of a specific form arising from uniform laws of large numbers for martingales), and (iii) in-expectation bounds for the supremum.
Adaptive Online Learning
We propose a general framework for studying adaptive regret bounds in the online learning framework, including model selection bounds and data-dependent bounds.
Hierarchies of Relaxations for Online Prediction Problems with Evolving Constraints
We study online prediction where regret of the algorithm is measured against a benchmark defined via evolving constraints.
Learning with Square Loss: Localization through Offset Rademacher Complexity
We consider regression with square loss and general classes of functions without the boundedness assumption.
Computational and Statistical Boundaries for Submatrix Localization in a Large Noisy Matrix
The second threshold, $\sf SNR_s$, captures the statistical boundary, below which no method can succeed with probability going to one in the minimax sense.
Sequential Probability Assignment with Binary Alphabets and Large Classes of Experts
We analyze the problem of sequential probability assignment for binary outcomes with side information and logarithmic loss, where regret---or, redundancy---is measured with respect to a (possibly infinite) class of experts.
Escaping the Local Minima via Simulated Annealing: Optimization of Approximately Convex Functions
We consider the problem of optimizing an approximately convex function over a bounded convex set in $\mathbb{R}^n$ using only function evaluations.
Online Optimization : Competing with Dynamic Comparators
Recent literature on online learning has focused on developing adaptive algorithms that take advantage of a regularity of the sequence of observations, yet retain worst-case performance guarantees.
Online Nonparametric Regression with General Loss Functions
This paper establishes minimax rates for online regression with arbitrary classes of functions and general losses.
Distributed Detection : Finite-time Analysis and Impact of Network Topology
In contrast to the existing literature which focuses on asymptotic learning, we provide a finite-time analysis.
Geometric Inference for General High-Dimensional Linear Inverse Problems
This paper presents a unified geometric framework for the statistical analysis of a general ill-posed linear inverse model which includes as special cases noisy compressed sensing, sign vector recovery, trace regression, orthogonal matrix estimation, and noisy matrix completion.
On Zeroth-Order Stochastic Convex Optimization via Random Walks
The method is based on a random walk (the \emph{Ball Walk}) on the epigraph of the function.
Online Nonparametric Regression
The optimal rates are shown to exhibit a phase transition analogous to the i. i. d./statistical learning case, studied in (Rakhlin, Sridharan, Tsybakov 2013).
Optimization, Learning, and Games with Predictable Sequences
We provide several applications of Optimistic Mirror Descent, an online learning algorithm based on the idea of predictable sequences.
Online Learning of Dynamic Parameters in Social Networks
Based on the decomposition of the global loss function, we introduce two update mechanisms, each of which generates an estimate of the true state.
Efficient Sampling from Time-Varying Log-Concave Distributions
Within the context of exponential families, the proposed method produces samples from a posterior distribution which is updated as data arrive in a streaming fashion.
Empirical entropy, minimax regret and minimax risk
Furthermore, for $p\in(0, 2)$, the excess risk rate matches the behavior of the minimax risk of function estimation in regression problems under the well-specified model.
Online Learning with Predictable Sequences
Variance and path-length bounds can be seen as particular examples of online learning with simple predictable sequences.
Lower Bounds for Passive and Active Learning
For passive learning, our lower bounds match the upper bounds of Gine and Koltchinskii up to constants and generalize analogous results of Massart and Nedelec.
Online Learning: Stochastic, Constrained, and Smoothed Adversaries
We define the minimax value of a game where the adversary is restricted in his moves, capturing stochastic and non-stochastic assumptions on data.
Stochastic convex optimization with bandit feedback
This paper addresses the problem of minimizing a convex, Lipschitz function $f$ over a convex, compact set $X$ under a stochastic bandit feedback model.
Random Walk Approach to Regret Minimization
We propose a computationally efficient random walk on a convex body which rapidly mixes to a time-varying Gibbs distribution.
Online Learning: Random Averages, Combinatorial Parameters, and Learnability
We develop a theory of online learning by defining several complexity measures.
Online Learning via Sequential Complexities
We consider the problem of sequential prediction and provide tools to study the minimax value of the associated game.
