Search Results for author:
Found 21 papers, 7 papers with code
InFoBERT: Zero-Shot Approach to Natural Language Understanding Using Contextualized Word Embedding
Natural language understanding is an important task in modern dialogue systems.
Multiple Teacher Distillation for Robust and Greener Models
The language models nowadays are in the center of natural language processing progress.
Ask Me Anything in Your Native Language
Cross-lingual question answering is a thriving field in the modern world, helping people to search information on the web more efficiently.
Single Example Can Improve Zero-Shot Data Generation
We explore two approaches to the generation of task-oriented utterances: in the zero-shot approach, the model is trained to generate utterances from seen intents and is further used to generate utterances for intents unseen during training.
Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule
Progress in neural grammatical error correction (GEC) is hindered by the lack of annotated training data.
Sinkhorn Transformations for Single-Query Postprocessing in Text-Video Retrieval
A recent trend in multimodal retrieval is related to postprocessing test set results via the dual-softmax loss (DSL).
GEC-DePenD: Non-Autoregressive Grammatical Error Correction with Decoupled Permutation and Decoding
Grammatical error correction (GEC) is an important NLP task that is currently usually solved with autoregressive sequence-to-sequence models.
AI-generated text boundary detection with RoFT
Due to the rapid development of large language models, people increasingly often encounter texts that may start as written by a human but continue as machine-generated.
Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts
Rapidly increasing quality of AI-generated content makes it difficult to distinguish between human and AI-generated texts, which may lead to undesirable consequences for society.
Can BERT eat RuCoLA? Topological Data Analysis to Explain
Our results contribute to understanding the behavior of monolingual LMs in the acceptability classification task, provide insights into the functional roles of attention heads, and highlight the advantages of TDA-based approaches for analyzing LMs.
Ranked #1 on
Linguistic Acceptability
on RuCoLA
PanGu-Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing
In this work, we develop a system that trained a trillion-parameter language model on a cluster of Ascend 910 AI processors and MindSpore framework, and present the language model with 1. 085T parameters named PanGu-{\Sigma}.
Topological Data Analysis for Speech Processing
We apply topological data analysis (TDA) to speech classification problems and to the introspection of a pretrained speech model, HuBERT.
Betti numbers of attention graphs is all you really need
We apply methods of topological analysis to the attention graphs, calculated on the attention heads of the BERT model ( arXiv:1810. 04805v2 ).
Template-based Approach to Zero-shot Intent Recognition
We outperform previous state-of-the-art f1-measure by up to 16\% for unseen intents, using intent labels and user utterances and without accessing external sources (such as knowledge bases).
Acceptability Judgements via Examining the Topology of Attention Maps
The role of the attention mechanism in encoding linguistic knowledge has received special interest in NLP.
Ranked #1 on
Linguistic Acceptability
on ItaCoLA
Artificial Text Detection via Examining the Topology of Attention Maps
The impressive capabilities of recent generative models to create texts that are challenging to distinguish from the human-written ones can be misused for generating fake news, product reviews, and even abusive content.
A Single Example Can Improve Zero-Shot Data Generation
In the zero-shot approach, the model is trained to generate utterances from seen intents and is further used to generate utterances for intents unseen during training.
Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detection
In turn, the Mahalanobis distance captures this disparity easily.
SumTitles: a Summarization Dataset with Low Extractiveness
The existing dialogue summarization corpora are significantly extractive.
Distributed Fine-tuning of Language Models on Private Data
In language modeling, users’ language (e. g. in private messaging) could change in a year and be completely different from what we observe in publicly available data.
Differentially Private Distributed Learning for Language Modeling Tasks
One of the big challenges in machine learning applications is that training data can be different from the real-world data faced by the algorithm.
