Search Results for author:
Found 31 papers, 21 papers with code
MuPT: A Generative Symbolic Music Pretrained Transformer
In this paper, we explore the application of Large Language Models (LLMs) to the pre-training of music.
VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?
Multimodal Large Language models (MLLMs) have shown promise in web-related tasks, but evaluating their performance in the web domain remains a challenge due to the lack of comprehensive benchmarks.
CodeEditorBench: Evaluating Code Editing Capability of Large Language Models
Large Language Models (LLMs) for code are rapidly evolving, with code editing emerging as a critical capability.
Long-context LLMs Struggle with Long In-context Learning
Our study reveals that long context understanding and reasoning is still a challenging task for the existing LLMs.
Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents
This iterative cycle of exploration and training fosters continued improvement in the agents.
StructLM: Towards Building Generalist Models for Structured Knowledge Grounding
Utilizing this dataset, we train a series of models, referred to as StructLM, based on the Mistral and the CodeLlama model family, ranging from 7B to 34B parameters.
Machine Unlearning of Pre-trained Large Language Models
This study investigates the concept of the `right to be forgotten' within the context of large language models (LLMs).
AttributionBench: How Hard is Automatic Attribution Evaluation?
Modern generative search engines enhance the reliability of large language model (LLM) responses by providing cited evidence.
OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement
However, open-source models often lack the execution capabilities and iterative refinement of advanced systems like the GPT-4 Code Interpreter.
Data Engineering for Scaling Language Models to 128K Context
We demonstrate that continual pretraining of the full model on 1B-5B tokens of such data is an effective and affordable strategy for scaling the context length of language models to 128K.
VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation
We evaluate VIESCORE on seven prominent tasks in conditional image tasks and found: (1) VIESCORE (GPT4-v) achieves a high Spearman correlation of 0. 3 with human evaluations, while the human-to-human correlation is 0. 45.
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning.
TableLlama: Towards Open Large Generalist Models for Tables
Towards that end, we construct TableInstruct, a new dataset with a variety of realistic tables and tasks, for instruction tuning and evaluating LLMs.
MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning
The MAmmoTH models are trained on MathInstruct, our meticulously curated instruction tuning dataset.
Roll Up Your Sleeves: Working with a Collaborative and Engaging Task-Oriented Dialogue System
We introduce TacoBot, a user-centered task-oriented digital assistant designed to guide users through complex real-world tasks with multiple steps.
Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate
Large language models (LLMs) such as ChatGPT and GPT-4 have shown impressive performance in complex reasoning tasks.
Automatic Evaluation of Attribution by Large Language Models
We manually curate a set of test examples covering 12 domains from a generative search engine, New Bing.
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe
Privacy concerns have attracted increasing attention in data-driven products due to the tendency of machine learning models to memorize sensitive training data.
Bootstrapping a User-Centered Task-Oriented Dialogue System
We present TacoBot, a task-oriented dialogue system built for the inaugural Alexa Prize TaskBot Challenge, which assists users in completing multi-step cooking and home improvement tasks.
Synthetic Question Value Estimation for Domain Adaptation of Question Answering
Synthesizing QA pairs with a question generator (QG) on the target domain has become a popular approach for domain adaptation of question answering (QA) models.
C-MORE: Pretraining to Answer Open-Domain Questions by Consulting Millions of References
And with our pretrained reader, the entire system improves by up to 4% in exact match.
Differential Privacy for Text Analytics via Natural Text Sanitization
The sanitized texts also contribute to our sanitization-aware pretraining and fine-tuning, enabling privacy-preserving natural language processing over the BERT language model with promising utility.
CliniQG4QA: Generating Diverse Questions for Domain Adaptation of Clinical Question Answering
Clinical question answering (QA) aims to automatically answer questions from medical professionals based on clinical texts.
COUGH: A Challenge Dataset and Models for COVID-19 FAQ Retrieval
For evaluation, we introduce Query Bank and Relevance Set, where the former contains 1, 236 human-paraphrased queries while the latter contains ~32 human-annotated FAQ items for each query.
PHICON: Improving Generalization of Clinical Text De-identification Models via Data Augmentation
De-identification is the task of identifying protected health information (PHI) in the clinical text.
Clinical Reading Comprehension: A Thorough Analysis of the emrQA Dataset
In this paper, we provide an in-depth analysis of this dataset and the clinical reading comprehension (CliniRC) task.
Practical Annotation Strategies for Question Answering Datasets
Here, remarkably, annotating a stratified subset with only 1. 2% of the original training set achieves 97. 7% of the performance as if the complete dataset was annotated.
Towards Making the Most of Context in Neural Machine Translation
Document-level machine translation manages to outperform sentence level models by a small margin, but have failed to be widely adopted.
Tensor Decomposition with Relational Constraints for Predicting Multiple Types of MicroRNA-disease Associations
To this end, we innovatively represent miRNA-disease-type triplets as a tensor and introduce Tensor Decomposition methods to solve the prediction task.
SurfCon: Synonym Discovery on Privacy-Aware Clinical Data
To solve the problem, we propose a new framework SurfCon that leverages two important types of information in the privacy-aware clinical data, i. e., the surface form information, and the global context information for synonym discovery.
Graph Embedding on Biomedical Networks: Methods, Applications, and Evaluations
Our experimental results demonstrate that the recent graph embedding methods achieve promising results and deserve more attention in the future biomedical graph analysis.
