Search Results for author:
Found 11 papers, 6 papers with code
On Isotropy Calibration of Transformer Models
Different studies of the embedding space of transformer models suggest that the distribution of contextual representations is highly anisotropic - the embeddings are distributed in a narrow cone.
pNLP-Mixer: an Efficient all-MLP Architecture for Language
Large pre-trained language models based on transformer architecture have drastically changed the natural language processing (NLP) landscape.
On Isotropy Calibration of Transformers
Different studies of the embedding space of transformer models suggest that the distribution of contextual representations is highly anisotropic - the embeddings are distributed in a narrow cone.
A Plug-and-Play Method for Controlled Text Generation
Large pre-trained language models have repeatedly shown their ability to produce fluent text.
Towards BERT-based Automatic ICD Coding: Limitations and Opportunities
Unlike the general trend in language processing, no transformer model has been reported to reach high performance on this task.
Of Non-Linearity and Commutativity in BERT
In this work we provide new insights into the transformer architecture, and in particular, its best-known variant, BERT.
Directed Beam Search: Plug-and-Play Lexically Constrained Language Generation
Given that state-of-the-art language models are too large to be trained from scratch in a manageable time, it is desirable to control these models without re-training them.
Brain2Word: Decoding Brain Activity for Language Generation
In the case of language stimuli, recent studies have shown that it is possible to decode fMRI scans into an embedding of the word a subject is reading.
Medley2K: A Dataset of Medley Transitions
The automatic generation of medleys, i. e., musical pieces formed by different songs concatenated via smooth transitions, is not well studied in the current literature.
Telling BERT's full story: from Local Attention to Global Aggregation
This way, we propose a distinction between local patterns revealed by attention and global patterns that refer back to the input, and analyze BERT from both angles.
Synthetic Epileptic Brain Activities Using Generative Adversarial Networks
In this work, we generate synthetic seizure-like brain electrical activities, i. e., EEG signals, that can be used to train seizure detection algorithms, alleviating the need for recorded data.
