Search Results for author:
Found 7 papers, 3 papers with code
Structural Self-Supervised Objectives for Transformers
This thesis focuses on improving the pre-training of natural language models using unsupervised raw data to make them more efficient and aligned with downstream applications.
Ranked #1 on
Question Answering
on TrecQA
(using extra training data)
Context-Aware Transformer Pre-Training for Answer Sentence Selection
Answer Sentence Selection (AS2) is a core component for building an accurate Question Answering pipeline.
Ranked #4 on
Question Answering
on TrecQA
(using extra training data)
Effective Pre-Training Objectives for Transformer-based Autoencoders
In this paper, we study trade-offs between efficiency, cost and accuracy when pre-training Transformer encoders with different pre-training objectives.
Pre-training Transformer Models with Sentence-Level Objectives for Answer Sentence Selection
An important task for designing QA systems is answer sentence selection (AS2): selecting the sentence containing (or constituting) the answer to a question from a set of retrieved relevant documents.
Ranked #1 on
Answer Selection
on ASNQ
Paragraph-based Transformer Pre-training for Multi-Sentence Inference
Our evaluation on three AS2 and one fact verification datasets demonstrates the superiority of our pre-training technique over the traditional ones for transformers used as joint models for multi-candidate inference tasks, as well as when used as cross-encoders for sentence-pair formulations of these tasks.
Ranked #3 on
Fact Verification
on FEVER
Efficient pre-training objectives for Transformers
The Transformer architecture deeply changed the natural language processing, outperforming all previous state-of-the-art models.
Efficient Generation of Structured Objects with Constrained Adversarial Networks
Generative Adversarial Networks (GANs) struggle to generate structured objects like molecules and game maps.
