Search Results for author:
Found 3 papers, 0 papers with code
Learn2Hop: Learned Optimization on Rough Landscapes
Optimization of non-convex loss surfaces containing many local minima remains a critical problem in a variety of domains, including operations research, informatics, and material design.
Temperature check: theory and practice for training models with softmax-cross-entropy losses
In this work we develop a theory of early learning for models trained with softmax-cross-entropy loss and show that the learning dynamics depend crucially on the inverse-temperature $\beta$ as well as the magnitude of the logits at initialization, $||\beta{\bf z}||_{2}$.
Disentangling Trainability and Generalization in Deep Learning
In this paper, we discuss these challenging issues in the context of wide neural networks at large depths where we will see that the situation simplifies considerably.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
