Search Results for author:
Found 5 papers, 0 papers with code
A Gentle Introduction to Gradient-Based Optimization and Variational Inequalities for Machine Learning
The rapid progress in machine learning in recent years has been based on a highly productive connection to gradient-based optimization.
Whitening and second order optimization both destroy information about the dataset, and can make generalization impossible
We show that both data whitening and second order optimization can harm or entirely prevent generalization.
Whitening and second order optimization both make information in the dataset unusable during training, and can reduce or prevent generalization
We show that both data whitening and second order optimization can harm or entirely prevent generalization.
Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network Losses
Despite the fact that the loss functions of deep neural networks are highly non-convex, gradient-based optimization algorithms converge to approximately the same performance from many random initial points.
Numerically Recovering the Critical Points of a Deep Linear Autoencoder
Numerically locating the critical points of non-convex surfaces is a long-standing problem central to many fields.
