no code implementations • 2 Apr 2024 • Lena Strobl, Dana Angluin, David Chiang, Jonathan Rawski, Ashish Sabharwal
We study the sequence-to-sequence mapping capacity of transformers by relating them to finite transducers, and find that they can express surprisingly large classes of transductions.
no code implementations • 1 Nov 2023 • Lena Strobl, William Merrill, Gail Weiss, David Chiang, Dana Angluin
As transformers have gained prominence in natural language processing, some researchers have investigated theoretically what problems they can and cannot solve, by treating problems as formal languages.
no code implementations • 6 Aug 2023 • Lena Strobl
Transformers have emerged as a widely used neural network model for various natural language processing tasks.