Emergent mechanisms for long timescales depend on training curriculum and affect performance in memory tasks

no code implementations • 22 Sep 2023 • Sina Khajehabdollahi, Roxana Zeraati, Emmanouil Giannakakis, Tim Jakob Schäfer, Georg Martius, Anna Levina

We find that for both tasks RNNs develop longer timescales with increasing $N$, but depending on the learning objective, they use different mechanisms.