Search Results for author:
Found 16 papers, 9 papers with code
XGen-7B Technical Report
Most open-source LLMs, on the other hand, are limited in their ability to support longer sequence lengths, which is a key requirement for many tasks that require inference over an input context.
CodeGen2: Lessons for Training LLMs on Programming and Natural Languages
In this study, we attempt to render the training of LLMs for program synthesis more efficient by unifying four key components: (1) model architectures, (2) learning methods, (3) infill sampling, and, (4) data distributions.
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code
This problem is especially pertinent in natural language generation which requires ever-improving suites of datasets, metrics, and human evaluation to make definitive claims.
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
To democratize this, we train and release a family of large language models up to 16. 1B parameters, called CODEGEN, on natural language and programming language data, and open source the training library JAXFORMER.
Ranked #81 on
Code Generation
on HumanEval
DEEP: DEnoising Entity Pre-training for Neural Machine Translation
It has been shown that machine translation models usually generate poor translations for named entities that are infrequent in the training corpus.
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
This paper surveys and organizes research works in a new paradigm in natural language processing, which we dub "prompt-based learning".
WikiAsp: A Dataset for Multi-domain Aspect-based Summarization
In this paper, we propose WikiAsp, a large-scale dataset for multi-domain aspect-based summarization that attempts to spur research in the direction of open-domain aspect-based summarization.
What's New? Summarizing Contributions in Scientific Literature
To overcome this problem, we introduce a new task of disentangled paper summarization, which seeks to generate separate summaries for the paper contributions and the context of the work, making it easier to identify the key findings shared in articles.
GSum: A General Framework for Guided Neural Abstractive Summarization
Neural abstractive summarization models are flexible and can produce coherent summaries, but they are sometimes unfaithful and can be difficult to control.
Findings of the Fourth Workshop on Neural Generation and Translation
We describe the finding of the Fourth Workshop on Neural Generation and Translation, held in concert with the annual conference of the Association for Computational Linguistics (ACL 2020).
Findings of the Third Workshop on Neural Generation and Translation
This document describes the findings of the Third Workshop on Neural Generation and Translation, held in concert with the annual conference of the Empirical Methods in Natural Language Processing (EMNLP 2019).
Linguistic Versus Latent Relations for Modeling Coherent Flow in Paragraphs
In order to produce a coherent flow of text, we explore two forms of intersentential relations in a paragraph: one is a human-created linguistical relation that forms a structure (e. g., discourse tree) and the other is a relation from latent representation learned from the sentences themselves.
Latent Relation Language Models
In this paper, we propose Latent Relation Language Models (LRLMs), a class of language models that parameterizes the joint distribution over the words in a document and the entities that occur therein via knowledge graph relations.
Learning to Describe Unknown Phrases with Local and Global Contexts
When reading a text, it is common to become stuck on unfamiliar words and phrases, such as polysemous words with novel senses, rarely used idioms, internet slang, or emerging entities.
Learning to Describe Phrases with Local and Global Contexts
When reading a text, it is common to become stuck on unfamiliar words and phrases, such as polysemous words with novel senses, rarely used idioms, internet slang, or emerging entities.
Eve: A Gradient Based Optimization Method with Locally and Globally Adaptive Learning Rates
Adaptive gradient methods for stochastic optimization adjust the learning rate for each parameter locally.
