Dynamics of specialization in neural modules under resource constraints

1 code implementation • 4 Jun 2021 • Gabriel Béna, Dan F. M. Goodman

We further find that in this setup (1) specialization can only emerge in environments where features of that environment are meaningfully separable, (2) specialization preferentially emerges when the network is strongly resource-constrained, and (3) these findings are qualitatively similar across the different variations of network architectures that we tested, but that the quantitative relationships depend on the precise architecture.

BIG-bench Machine Learning Open-Ended Question Answering