Search Results for author:
Found 20 papers, 12 papers with code
LLM360: Towards Fully Transparent Open-Source LLMs
The recent surge in open-source Large Language Models (LLMs), such as LLaMA, Falcon, and Mistral, provides diverse options for AI practitioners and researchers.
RedCoast: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs
In this work, we present RedCoast(Redco), a lightweight and user-friendly tool crafted to automate distributed training and inference for LLMs, as well as to simplify ML pipeline development.
SlimPajama-DC: Understanding Data Combinations for LLM Training
This paper aims to understand the impacts of various data combinations (e. g., web text, wikipedia, github, books) on the training of large language models using SlimPajama.
BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models
The resulting KGs as a symbolic interpretation of the source LMs also reveal new insights into the LMs' knowledge capacities.
Text Generation with Efficient (Soft) $Q$-Learning
We apply the approach to a wide range of text generation tasks, including learning from noisy/negative examples, adversarial attacks, and prompt generation.
Compression, Transduction, and Creation: A Unified Framework for Evaluating Natural Language Generation
Based on the nature of information change from input to output, we classify NLG tasks into compression (e. g., summarization), transduction (e. g., text rewriting), and creation (e. g., dialog).
On the Generation of Medical Dialogs for COVID-19
Training complex dialog generation models on small datasets bears high risk of overfitting.
Efficient (Soft) Q-Learning for Text Generation with Limited Good Data
We apply the approach to a wide range of novel text generation tasks, including learning from noisy/negative examples, adversarial attacks, and prompt generation.
Summarizing Text on Any Aspects: A Knowledge-Informed Weakly-Supervised Approach
Given a document and a target aspect (e. g., a topic of interest), aspect-based abstractive summarization attempts to generate a summary with respect to the aspect.
Progressive Generation of Long Text with Pretrained Language Models
However, as our systematic examination reveals, it is still challenging for such models to generate coherent long passages of text (e. g., 1000 tokens), especially when the models are fine-tuned to the target domain on a small corpus.
Automatic Text Summarization of COVID-19 Medical Research Articles using BERT and GPT-2
With the COVID-19 pandemic, there is a growing urgency for medical community to keep up with the accelerating growth in the new coronavirus-related literature.
On the Generation of Medical Dialogues for COVID-19
On these two datasets, we train several dialogue generation models based on Transformer, GPT, and BERT-GPT.
Schema-Guided Multi-Domain Dialogue State Tracking with Graph Attention Neural Networks
For multi-domain DST, the data sparsity problem is also a major obstacle due to the increased number of state candidates.
Ranked #12 on
Multi-domain Dialogue State Tracking
on MULTIWOZ 2.1
Learning Data Manipulation for Augmentation and Weighting
In this work, we propose a new method that supports learning different manipulation schemes with the same gradient-based algorithm.
AgentGraph: Towards Universal Dialogue Management with Structured Deep Reinforcement Learning
Experiments show that AgentGraph models significantly outperform traditional reinforcement learning approaches on most of the 18 tasks of the PyDial benchmark.
Connecting the Dots Between MLE and RL for Sequence Prediction
Reinforcement learning such as policy gradient addresses the issue but can have prohibitively poor exploration efficiency.
Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
The versatile toolkit also fosters technique sharing across different text generation tasks.
Structured Dialogue Policy with Graph Neural Networks
The proposed structured deep reinforcement learning is based on graph neural networks (GNN), which consists of some sub-networks, each one for a node on a directed graph.
Texar: A Modularized, Versatile, and Extensible Toolbox for Text Generation
The features make Texar particularly suitable for technique sharing and generalization across different text generation applications.
Autonomous Driving in Reality with Reinforcement Learning and Image Translation
Supervised learning is widely used in training autonomous driving vehicle.
