ACL 2020

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

ACL 2020 huggingface/transformers

We evaluate a number of noising approaches, finding the best performance by both randomly shuffling the order of the original sentences and using a novel in-filling scheme, where spans of text are replaced with a single mask token.

DENOISING MACHINE TRANSLATION NATURAL LANGUAGE INFERENCE QUESTION ANSWERING TEXT GENERATION

Unsupervised Cross-lingual Representation Learning at Scale

ACL 2020 huggingface/transformers

We also present a detailed empirical analysis of the key factors that are required to achieve these gains, including the trade-offs between (1) positive transfer and capacity dilution and (2) the performance of high and low resource languages at scale.

CROSS-LINGUAL TRANSFER LANGUAGE MODELLING REPRESENTATION LEARNING

Mapping Natural Language Instructions to Mobile UI Action Sequences

ACL 2020 google-research/google-research

We present a new problem: grounding natural language instructions to mobile user interface actions, and create three new datasets for it.

BPE-Dropout: Simple and Effective Subword Regularization

ACL 2020 google/sentencepiece

Subword segmentation is widely used to address the open vocabulary problem in machine translation.

MACHINE TRANSLATION

The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding

ACL 2020 namisan/mt-dnn

We present MT-DNN, an open-source natural language understanding (NLU) toolkit that makes it easy for researchers and developers to train customized deep learning models.

MULTI-TASK LEARNING NATURAL LANGUAGE UNDERSTANDING STRUCTURED PREDICTION

SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization

ACL 2020 namisan/mt-dnn

However, due to limited data resources from downstream tasks and the extremely large capacity of pre-trained models, aggressive fine-tuning often causes the adapted model to overfit the data of downstream tasks and forget the knowledge of the pre-trained model.

TRANSFER LEARNING

SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis

ACL 2020 baidu/Senta

In particular, the prediction of aspect-sentiment pairs is converted into multi-label classification, aiming to capture the dependency between words in a pair.

MULTI-LABEL CLASSIFICATION SENTIMENT ANALYSIS

jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models

ACL 2020 nyu-mll/jiant

We introduce jiant, an open source toolkit for conducting multitask and transfer learning experiments on English NLU tasks.

TRANSFER LEARNING