HuggingFace's Transformers: State-of-the-art Natural Language Processing

9 Oct 20191 code implementation

In this paper, we present HuggingFace's Transformers library, a library for state-of-the-art NLP, making these developments available to the community by gathering state-of-the-art general-purpose pretrained models under a unified API together with an ecosystem of libraries, examples, tutorials and scripts targeting many downstream NLP tasks.

TEXT GENERATION TRANSFER LEARNING

AllenNLP: A Deep Semantic Natural Language Processing Platform

WS 2018 1 code implementation

This paper describes AllenNLP, a platform for research on deep learning methods in natural language understanding.

READING COMPREHENSION SEMANTIC ROLE LABELING

GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing

9 Jul 20192 code implementations

We present GluonCV and GluonNLP, the deep learning toolkits for computer vision and natural language processing based on Apache MXNet (incubating).

Pre-trained Models for Natural Language Processing: A Survey

18 Mar 20202 code implementations

Recently, the emergence of pre-trained models (PTMs) has brought natural language processing (NLP) to a new era.

REPRESENTATION LEARNING

CamemBERT: a Tasty French Language Model

10 Nov 20193 code implementations

We measure the performance of CamemBERT compared to multilingual models in multiple downstream tasks, namely part-of-speech tagging, dependency parsing, named-entity recognition, and natural language inference.

DEPENDENCY PARSING LANGUAGE MODELLING NAMED ENTITY RECOGNITION NATURAL LANGUAGE INFERENCE PART-OF-SPEECH TAGGING

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

26 Sep 201915 code implementations

Increasing model size when pretraining natural language representations often results in improved performance on downstream tasks.

LINGUISTIC ACCEPTABILITY NATURAL LANGUAGE INFERENCE QUESTION ANSWERING SEMANTIC TEXTUAL SIMILARITY

Unsupervised Cross-lingual Representation Learning at Scale

5 Nov 20195 code implementations

We also present a detailed empirical evaluation of the key factors that are required to achieve these gains, including the trade-offs between (1) positive transfer and capacity dilution and (2) the performance of high and low resource languages at scale.

CROSS-LINGUAL TRANSFER LANGUAGE MODELLING REPRESENTATION LEARNING

RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 Jul 201910 code implementations

Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging.

 SOTA for Question Answering on SQuAD2.0 dev (using extra training data)

LANGUAGE MODELLING LEXICAL SIMPLIFICATION NATURAL LANGUAGE INFERENCE QUESTION ANSWERING READING COMPREHENSION SEMANTIC TEXTUAL SIMILARITY SENTIMENT ANALYSIS