Search Results for author:
Found 7 papers, 1 papers with code
MetaOptimize: A Framework for Optimizing Step Sizes and Other Meta-parameters
This paper addresses the challenge of optimizing meta-parameters (i. e., hyperparameters) in machine learning algorithms, a critical factor influencing training efficiency and model performance.
Step-size Optimization for Continual Learning
We conclude by suggesting that combining both approaches could be a promising future direction to improve the performance of neural networks in continual learning.
Toward Efficient Gradient-Based Value Estimation
Gradient-based methods for value estimation in reinforcement learning have favorable stability properties, but they are typically much slower than Temporal Difference (TD) learning methods.
Jumping Fluid Models and Delay Stability of Max-Weight Dynamics under Heavy-Tailed Traffic
We say that a random variable is $light$-$tailed$ if moments of order $2+\epsilon$ are finite for some $\epsilon>0$; otherwise, we say that it is $heavy$-$tailed$.
Order Optimal Bounds for One-Shot Federated Learning over non-Convex Loss Functions
We then prove that this lower bound is order optimal in $m$ and $n$ by presenting a distributed learning algorithm, called Multi-Resolution Estimator for Non-Convex loss function (MRE-NC), whose expected loss matches the lower bound for large $mn$ up to polylogarithmic factors.
Bounds on Over-Parameterization for Guaranteed Existence of Descent Paths in Shallow ReLU Networks
We show that there exist poor local minima with positive curvature for some training sets of size $n\geq m+2d-2$.
Order Optimal One-Shot Distributed Learning
We propose an algorithm called Multi-Resolution Estimator (MRE) whose expected error is no larger than $\tilde{O}\big(m^{-{1}/{\max(d, 2)}} n^{-1/2}\big)$, where $d$ is the dimension of the parameter space.
