Search Results for author:
Found 9 papers, 4 papers with code
The Developmental Landscape of In-Context Learning
We show that in-context learning emerges in transformers in discrete developmental stages, when they are trained on either language modeling or linear regression tasks.
Dynamical versus Bayesian Phase Transitions in a Toy Model of Superposition
We investigate phase transitions in a Toy Model of Superposition (TMS) using Singular Learning Theory (SLT).
Quantifying degeneracy in singular models via the learning coefficient
Deep neural networks (DNN) are singular statistical models which exhibit complex degeneracies.
Geometry of Program Synthesis
We re-evaluate universal computation based on the synthesis of Turing machines.
Deep Learning is Singular, and That's Good
In singular models, the optimal set of parameters forms an analytic set with singularities and classical statistical inference cannot be applied to such models.
Logic and the 2-Simplicial Transformer
We introduce the 2-simplicial Transformer, an extension of the Transformer which includes a form of higher-dimensional attention generalising the dot-product attention, and uses this attention to update entity representations with tensor products of value vectors.
Logic and the $2$-Simplicial Transformer
We introduce the $2$-simplicial Transformer, an extension of the Transformer which includes a form of higher-dimensional attention generalising the dot-product attention, and uses this attention to update entity representations with tensor products of value vectors.
Constructing $A_\infty$-categories of matrix factorisations
We study constructive $A_\infty$-models of the DG-category of matrix factorisations of a potential over a commutative $\mathbb{Q}$-algebra $k$, consisting of a Hom-finite $A_\infty$-category equipped with an $A_\infty$-idempotent functor.
Algebraic Geometry Mathematical Physics Mathematical Physics
Logic and linear algebra: an introduction
We give an introduction to logic tailored for algebraists, explaining how proofs in linear logic can be viewed as algorithms for constructing morphisms in symmetric closed monoidal categories with additional structure.
Logic Logic in Computer Science Category Theory
