Search Results for author:
Found 25 papers, 0 papers with code
Challenges in Variable Importance Ranking Under Correlation
Recently, several adjustments to marginal permutation utilizing feature knockoffs were proposed to address this issue, such as the variable importance measure known as conditional predictive impact (CPI).
Stochastic Gradient Descent for Additive Nonparametric Regression
This paper introduces an iterative algorithm for training additive models that enjoys favorable memory storage and computational requirements.
Inference with Mondrian Random Forests
Random forests are popular methods for classification and regression, and many different variants have been proposed in recent years.
Robust Transfer Learning with Unreliable Source Data
This paper addresses challenges in robust transfer learning stemming from ambiguity in Bayes classifiers and weak transferable signals between the target and source distribution.
Error Reduction from Stacked Regressions
In this paper, we learn these weights analogously by minimizing an estimate of the population risk subject to a nonnegativity constraint.
On the Implicit Bias of Adam
In previous literature, backward error analysis was used to find ordinary differential equations (ODEs) approximating the gradient descent trajectory.
Sharp Convergence Rates for Matching Pursuit
We study the fundamental limits of matching pursuit, or the pure greedy algorithm, for approximating a target function by a sparse linear combination of elements from a dictionary.
On the Pointwise Behavior of Recursive Partitioning and Its Implications for Heterogeneous Causal Effect Estimation
Decision tree learning is increasingly being used for pointwise inference.
Large Scale Prediction with Decision Trees
This paper shows that decision trees constructed with Classification and Regression Trees (CART) and C4. 5 methodology are consistent for regression and classification tasks, even when the number of predictor variables grows sub-exponentially with the sample size, under natural 0-norm and 1-norm sparsity constraints.
Nonparametric Variable Screening with Optimal Decision Stumps
Decision trees and their ensembles are endowed with a rich set of diagnostic tools for ranking and screening variables in a predictive model.
Good Classifiers are Abundant in the Interpolating Regime
Inspired by the statistical mechanics approach to learning, we formally define and develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers from several model classes.
Sparse learning with CART
In doing so, we find that the training error is governed by the Pearson correlation between the optimal decision stump and response data in each node, which we bound by constructing a prior distribution on the split points and solving a nonlinear optimization problem.
Global Capacity Measures for Deep ReLU Networks via Path Sampling
Classical results on the statistical complexity of linear models have commonly identified the norm of the weights $\|w\|$ as a fundamental capacity measure.
Analyzing CART
For binary classification and regression models, this approach recursively divides the data into two near-homogenous daughter nodes according to a split point that maximizes the reduction in sum of squares error (the impurity) along a particular variable.
Complexity, Statistical Risk, and Metric Entropy of Deep Nets Using Total Path Variation
For any ReLU network there is a representation in which the sum of the absolute values of the weights into each node is exactly $1$, and the input layer variables are multiplied by a value $V$ coinciding with the total variation of the path weights.
Approximation and Estimation for High-Dimensional Deep Learning Networks
It has been experimentally observed in recent years that multi-layer artificial neural networks have a surprising ability to generalize, even when trained with far more parameters than observations.
Sharp Analysis of a Simple Model for Random Forests
Random forests have become an important tool for improving accuracy in regression and classification problems since their inception by Leo Breiman in 2001.
Counting Motifs with Graph Sampling
Applied researchers often construct a network from a random sample of nodes in order to infer properties of the parent network.
Estimating the Number of Connected Components in a Graph via Subgraph Sampling
Learning properties of large graphs from samples has been an important problem in statistical network analysis since the early work of Goodman \cite{Goodman1949} and Frank \cite{Frank1978}.
Finite-sample risk bounds for maximum likelihood estimation with arbitrary penalties
The MDL two-part coding $ \textit{index of resolvability} $ provides a finite-sample upper bound on the statistical risk of penalized likelihood estimators over countable models.
Estimating the Coefficients of a Mixture of Two Linear Regressions by Expectation Maximization
We also show that the population EM operator for mixtures of two regressions is anti-contractive from the target parameter vector if the cosine angle between the input vector and the target parameter vector is too small, thereby establishing the necessity of our conic condition.
Minimax Lower Bounds for Ridge Combinations Including Neural Nets
Estimation of functions of $ d $ variables is considered using ridge combinations of the form $ \textstyle\sum_{k=1}^m c_{1, k} \phi(\textstyle\sum_{j=1}^d c_{0, j, k}x_j-b_k) $ where the activation function $ \phi $ is a function with bounded value and derivative.
Statistical Guarantees for Estimating the Centers of a Two-component Gaussian Mixture by EM
In that method, the basin of attraction for valid initialization is required to be a ball around the truth.
Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with $ \ell^1 $ and $ \ell^0 $ Controls
We establish $ L^{\infty} $ and $ L^2 $ error bounds for functions of many variables that are approximated by linear combinations of ReLU (rectified linear unit) and squared ReLU ridge functions with $ \ell^1 $ and $ \ell^0 $ controls on their inner and outer parameters.
Risk Bounds for High-dimensional Ridge Function Combinations Including Neural Networks
On the other hand, if the candidate fits are chosen from a discretization, we show that $ \mathbb{E}\|\hat{f} - f^{\star} \|^2 \leq \left(v^3_{f^{\star}}\frac{\log d}{n}\right)^{2/5} $.
