1 code implementation • 5 Jul 2022 • Laida Kushnareva, Dmitri Piontkovski, Irina Piontkovskaya
We apply methods of topological analysis to the attention graphs, calculated on the attention heads of the BERT model ( arXiv:1810. 04805v2 ).
1 code implementation • 19 May 2022 • Daniil Cherniavskii, Eduard Tulchinskii, Vladislav Mikhailov, Irina Proskurina, Laida Kushnareva, Ekaterina Artemova, Serguei Barannikov, Irina Piontkovskaya, Dmitri Piontkovski, Evgeny Burnaev
The role of the attention mechanism in encoding linguistic knowledge has received special interest in NLP.
Ranked #1 on Linguistic Acceptability on ItaCoLA
2 code implementations • EMNLP 2021 • Laida Kushnareva, Daniil Cherniavskii, Vladislav Mikhailov, Ekaterina Artemova, Serguei Barannikov, Alexander Bernstein, Irina Piontkovskaya, Dmitri Piontkovski, Evgeny Burnaev
The impressive capabilities of recent generative models to create texts that are challenging to distinguish from the human-written ones can be misused for generating fake news, product reviews, and even abusive content.