no code implementations • 5 Apr 2024 • Lucas Böttcher, Gregory Wheeler
Artificial neural networks can be seen as high-dimensional mathematical functions, and understanding the geometric properties of their loss landscapes (i. e., the high-dimensional space on which one wishes to find extrema or saddles) can provide valuable insights into their optimization behavior, generalization abilities, and overall performance.
1 code implementation • 28 Aug 2022 • Lucas Böttcher, Gregory Wheeler
We show that saddle points in the original space are rarely correctly identified as such in expected lower-dimensional representations if random projections are used.