Text Generation

368 papers with code · Natural Language Processing

Text generation is the task of generating text with the goal of appearing indistinguishable to human-written text.

( Image credit: Adversarial Ranking for Language Generation )

Benchmarks

Greatest papers with code

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

NeurIPS 2020 huggingface/transformers

Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks.

TEXT GENERATION

Plug and Play Language Models: A Simple Approach to Controlled Text Generation

ICLR 2020 huggingface/transformers

Large transformer-based language models (LMs) trained on huge text corpora have shown unparalleled generation capabilities.

LANGUAGE MODELLING TEXT GENERATION

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

ACL 2020 huggingface/transformers

We evaluate a number of noising approaches, finding the best performance by both randomly shuffling the order of the original sentences and using a novel in-filling scheme, where spans of text are replaced with a single mask token.

Ranked #10 on Question Answering on SQuAD1.1 dev (F1 metric)

DENOISING MACHINE TRANSLATION NATURAL LANGUAGE INFERENCE QUESTION ANSWERING TEXT GENERATION

HuggingFace's Transformers: State-of-the-art Natural Language Processing

9 Oct 2019huggingface/transformers

Transformer architectures have facilitated building higher-capacity models and pretraining has made it possible to effectively utilize this capacity for a wide variety of tasks.

TEXT GENERATION TRANSFER LEARNING

Language Models are Unsupervised Multitask Learners

Preprint 2019 huggingface/transformers

Natural language processing tasks, such as question answering, machine translation, reading comprehension, and summarization, are typically approached with supervised learning on taskspecific datasets.

 Ranked #1 on Language Modelling on enwik8 (using extra training data)

COMMON SENSE REASONING DATA-TO-TEXT GENERATION DOCUMENT SUMMARIZATION LANGUAGE MODELLING MACHINE TRANSLATION QUESTION ANSWERING READING COMPREHENSION

Stepwise Extractive Summarization and Planning with Structured Transformers

6 Oct 2020google-research/google-research

We propose encoder-centric stepwise models for extractive summarization using structured transformers -- HiBERT and Extended Transformers.

TABLE-TO-TEXT GENERATION

Neural Assistant: Joint Action Prediction, Response Generation, and Latent Knowledge Reasoning

31 Oct 2019tensorflow/tensor2tensor

In this paper, we develop Neural Assistant: a single neural network model that takes conversation history and an external knowledge source as input and jointly produces both text response and action to be taken by the system as output.

TEXT GENERATION

Generating Sequences With Recurrent Neural Networks

4 Aug 2013karpathy/char-rnn

This paper shows how Long Short-term Memory recurrent neural networks can be used to generate complex sequences with long-range structure, simply by predicting one data point at a time.

LANGUAGE MODELLING TEXT GENERATION

fairseq: A Fast, Extensible Toolkit for Sequence Modeling

NAACL 2019 pytorch/fairseq

fairseq is an open-source sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language modeling, and other text generation tasks.

LANGUAGE MODELLING TEXT GENERATION