ACL 2019

Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context

ACL 2019 huggingface/pytorch-transformers

Transformers have a potential of learning longer-term dependency, but are limited by a fixed-length context in the setting of language modeling.

LANGUAGE MODELLING

Proactive Human-Machine Conversation with Explicit Conversation Goal

ACL 2019 PaddlePaddle/models

Konv enables a very challenging task as the model needs to both understand dialogue and plan over the given knowledge graph.

Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings

ACL 2019 facebookresearch/LASER

Machine translation is highly sensitive to the size and quality of the training data, which has led to an increasing interest in collecting and filtering large parallel corpora.

CROSS-LINGUAL BITEXT MINING MACHINE TRANSLATION PARALLEL CORPUS MINING SENTENCE EMBEDDINGS

Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation

ACL 2019 asyml/texar

The versatile toolkit also fosters technique sharing across different text generation tasks.

MACHINE TRANSLATION TEXT GENERATION

Energy and Policy Considerations for Deep Learning in NLP

ACL 2019 DerwenAI/pytextrank

Recent progress in hardware and methodology for training neural networks has ushered in a new generation of large networks trained on abundant data.

Multi-Task Deep Neural Networks for Natural Language Understanding

ACL 2019 namisan/mt-dnn

In this paper, we present a Multi-Task Deep Neural Network (MT-DNN) for learning representations across multiple natural language understanding (NLU) tasks.

DOMAIN ADAPTATION LANGUAGE MODELLING LINGUISTIC ACCEPTABILITY NATURAL LANGUAGE INFERENCE PARAPHRASE IDENTIFICATION SENTIMENT ANALYSIS

A Multiscale Visualization of Attention in the Transformer Model

ACL 2019 jessevig/bertviz

The Transformer is a sequence model that forgoes traditional recurrent architectures in favor of a fully attention-based approach.

ERNIE: Enhanced Language Representation with Informative Entities

ACL 2019 thunlp/ERNIE

Neural language representation models such as BERT pre-trained on large-scale corpora can well capture rich semantic patterns from plain text, and be fine-tuned to consistently improve the performance of various NLP tasks.

ENTITY TYPING KNOWLEDGE GRAPHS NATURAL LANGUAGE INFERENCE SENTIMENT ANALYSIS

BERT Rediscovers the Classical NLP Pipeline

ACL 2019 nyu-mll/jiant

Pre-trained text encoders have rapidly advanced the state of the art on many NLP tasks.

NAMED ENTITY RECOGNITION

Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling

ACL 2019 nyu-mll/jiant

Natural language understanding has recently seen a surge of progress with the use of sentence encoders like ELMo (Peters et al., 2018a) and BERT (Devlin et al., 2019) which are pretrained on variants of language modeling.

LANGUAGE MODELLING TRANSFER LEARNING