no code implementations • 2 Sep 2023 • Shunjie Wang, Shane Steinert-Threlkeld
Despite the fact that Transformers perform well in NLP tasks, recent studies suggest that self-attention is theoretically limited in learning even some regular and context-free languages.
