no code implementations • 2 Apr 2024 • Mattia Opper, N. Siddharth
This paper presents two simple improvements to the Self-Structuring AutoEncoder (Self-StrAE).
no code implementations • 31 Oct 2023 • Mattia Opper, J. Morrison, N. Siddharth
Using BabyBERTa as a probe, we find that grammar acquisition is largely driven by exposure to speech data, and in particular through exposure to two of the BabyLM training corpora: AO-Childes and Open Subtitles.
no code implementations • 9 May 2023 • Mattia Opper, Victor Prokhorov, N. Siddharth
This work presents StrAE: a Structured Autoencoder framework that through strict adherence to explicit structure, and use of a novel contrastive objective over tree-structured representations, enables effective learning of multi-level representations.