HuggingFace's Transformers: State-of-the-art Natural Language Processing

9 Oct 20191 code implementation

In this paper, we present HuggingFace's Transformers library, a library for state-of-the-art NLP, making these developments available to the community by gathering state-of-the-art general-purpose pretrained models under a unified API together with an ecosystem of libraries, examples, tutorials and scripts targeting many downstream NLP tasks.

TEXT GENERATION TRANSFER LEARNING

AllenNLP: A Deep Semantic Natural Language Processing Platform

WS 2018 1 code implementation

This paper describes AllenNLP, a platform for research on deep learning methods in natural language understanding.

READING COMPREHENSION SEMANTIC ROLE LABELING

GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing

9 Jul 20192 code implementations

We present GluonCV and GluonNLP, the deep learning toolkits for computer vision and natural language processing based on Apache MXNet (incubating).

RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 Jul 20195 code implementations

Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging.

 SOTA for Question Answering on SQuAD2.0 dev (using extra training data)

LANGUAGE MODELLING LEXICAL SIMPLIFICATION NATURAL LANGUAGE INFERENCE QUESTION ANSWERING READING COMPREHENSION SEMANTIC TEXTUAL SIMILARITY SENTIMENT ANALYSIS

Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context

ACL 2019 9 code implementations

Transformers have a potential of learning longer-term dependency, but are limited by a fixed-length context in the setting of language modeling.

LANGUAGE MODELLING

XLNet: Generalized Autoregressive Pretraining for Language Understanding

NeurIPS 2019 11 code implementations

With the capability of modeling bidirectional contexts, denoising autoencoding based pretraining like BERT achieves better performance than pretraining approaches based on autoregressive language modeling.

DOCUMENT RANKING LANGUAGE MODELLING NATURAL LANGUAGE INFERENCE QUESTION ANSWERING READING COMPREHENSION SEMANTIC TEXTUAL SIMILARITY SENTIMENT ANALYSIS TEXT CLASSIFICATION

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

2 Oct 20194 code implementations

As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under constrained computational training or inference budgets remains challenging.

LANGUAGE MODELLING LINGUISTIC ACCEPTABILITY NATURAL LANGUAGE INFERENCE QUESTION ANSWERING SEMANTIC TEXTUAL SIMILARITY SENTIMENT ANALYSIS TRANSFER LEARNING

Phrase-Based & Neural Unsupervised Machine Translation

EMNLP 2018 8 code implementations

Machine translation systems achieve near human-level performance on some languages, yet their effectiveness strongly relies on the availability of large amounts of parallel sentences, which hinders their applicability to the majority of language pairs.

UNSUPERVISED MACHINE TRANSLATION