Search Results for author:
Found 13 papers, 10 papers with code
Towards Efficient and Exact Optimization of Language Model Alignment
We prove that EXO is guaranteed to optimize in the same direction as the RL algorithms asymptotically for arbitary parametrization of the policy, while enables efficient optimization by circumventing the complexities associated with RL algorithms.
Language Model Decoding as Direct Metrics Optimization
And most importantly, we prove that this induced distribution is guaranteed to improve the perplexity on human texts, which suggests a better approximation to the underlying distribution of human texts.
Tailoring Language Generation Models under Total Variation Distance
The standard paradigm of neural language generation adopts maximum likelihood estimation (MLE) as the optimizing method.
Curriculum-Based Self-Training Makes Better Few-Shot Learners for Data-to-Text Generation
Despite the success of text-to-text pre-trained models in various natural language generation (NLG) tasks, the generation performance is largely restricted by the number of labeled data in downstream tasks, particularly in data-to-text generation tasks.
LaMemo: Language Modeling with Look-Ahead Memory
Although Transformers with fully connected self-attentions are powerful to model long-term dependencies, they are struggling to scale to long texts with thousands of words in language modeling.
DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformer
Despite the recent advances in applying pre-trained language models to generate high-quality texts, generating long passages that maintain long-range coherence is yet challenging for these models.
JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs
Existing pre-trained models for knowledge-graph-to-text (KG-to-text) generation simply fine-tune text-to-text pre-trained models such as BART or T5 on KG-to-text datasets, which largely ignore the graph structure during encoding and lack elaborate pre-training tasks to explicitly model graph-text alignments.
Ranked #1 on
KG-to-Text Generation
on WebQuestions
CPM: A Large-scale Generative Chinese Pre-trained Language Model
However, applying GPT-3 to address Chinese NLP tasks is still challenging, as the training corpus of GPT-3 is primarily English, and the parameters are not publicly available.
Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths
Commonsense explanation generation aims to empower the machine's sense-making capability by generating plausible explanations to statements against commonsense.
Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph
Despite the success of generative pre-trained language models on a series of text generation tasks, they still suffer in cases where reasoning over underlying commonsense knowledge is required during generation.
SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge
To benefit the downstream tasks in sentiment analysis, we propose a novel language representation model called SentiLARE, which introduces word-level linguistic knowledge including part-of-speech tag and sentiment polarity (inferred from SentiWordNet) into pre-trained models.
Denoise while Aggregating: Collaborative Learning in Open-Domain Question Answering
The open-domain question answering (OpenQA) task aims to extract answers that match specific questions from a distantly supervised corpus.
Denoising Distantly Supervised Open-Domain Question Answering
Distantly supervised open-domain question answering (DS-QA) aims to find answers in collections of unlabeled text.
Ranked #2 on
Open-Domain Question Answering
on Quasar
