no code implementations • 14 Feb 2024 • Loek van Rossem, Andrew M. Saxe
We show through experiments that the effective theory describes aspects of representation learning dynamics across a range of deep networks with different activation functions and architectures, and exhibits phenomena similar to the "rich" and "lazy" regime.