Search Results for author:
Found 42 papers, 19 papers with code
Finetuned Multimodal Language Models Are High-Quality Image-Text Data Filters
Our MLM filter can generalize to different models and tasks, and be used as a drop-in replacement for CLIPScore.
Large Language Models as Zero-shot Dialogue State Tracker through Function Calling
We also show that by fine-tuning on a small collection of diverse task-oriented dialogues, we can equip modestly sized models, specifically a 13B parameter LLaMA2-Chat model, with function-calling capabilities and DST performance comparable to ChatGPT while maintaining their chat capabilities.
GPT-4V(ision) as a Generalist Evaluator for Vision-Language Tasks
Automatically evaluating vision-language tasks is challenging, especially when it comes to reflecting human judgments due to limitations in accounting for fine-grained details.
Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injection
In this work, we establish a benchmark to evaluate the robustness of instruction-following LLMs against prompt injection attacks.
Augmenting Language Models with Long-Term Memory
Such a decoupled memory design can easily cache and update long-term past contexts for memory retrieval without suffering from memory staleness.
STEPS: A Benchmark for Order Reasoning in Sequential Tasks
Therefore, to verify the order reasoning capability of current neural models in sequential tasks, we propose a challenging benchmark , named STEPS.
Graph Reasoning for Question Answering with Triplet Retrieval
State-of-the-art methods often utilize entities in questions to retrieve local subgraphs, which are then fed into KG encoder, e. g. graph neural networks (GNNs), to model their local structures and integrated into language models for question answering.
Bot or Human? Detecting ChatGPT Imposters with A Single Question
Large language models like GPT-4 have recently demonstrated impressive capabilities in natural language understanding and generation, enabling various applications including translation, essay writing, and chit-chatting.
Time Series as Images: Vision Transformer for Irregularly Sampled Time Series
This paper introduces a novel perspective by converting irregularly sampled time series into line graph images, then utilizing powerful pre-trained vision transformers for time series classification in the same way as image classification.
Language Model Detoxification in Dialogue with Contextualized Stance Control
To reduce the toxic degeneration in a pretrained Language Model (LM), previous work on Language Model detoxification has focused on reducing the toxicity of the generation itself (self-toxicity) without consideration of the context.
Improving Medical Predictions by Irregular Multimodal Electronic Health Records Modeling
Our method first addresses irregularity in each single modality by (1) modeling irregular time series by dynamically incorporating hand-crafted imputation embeddings into learned interpolation embeddings via a gating mechanism, and (2) casting a series of clinical note representations as multivariate irregular time series and tackling irregularity via a time attention mechanism.
Explanations from Large Language Models Make Small Reasoners Better
Integrating free-text explanations to in-context learning of large language models (LLM) is shown to elicit strong reasoning capabilities along with reasonable explanations.
Controllable Dialogue Simulation with In-Context Learning
Experimental results on the MultiWOZ dataset demonstrate that training a model on the simulated dialogues leads to even better performance than using the same amount of human-generated dialogues under the challenging low-resource settings, with as few as 85 dialogues as a seed.
Limitations of Language Models in Arithmetic and Symbolic Induction
LMs with tutor is able to deliver 100% accuracy in situations of OOD and repeating symbols, shedding new insights on the boundary of large LMs in induction.
Visually-Augmented Language Modeling
With the visually-augmented context, VaLM uses a visual knowledge fusion layer to enable multimodal grounded language modeling by attending to both text context and visual knowledge in images.
Task-adaptive Pre-training and Self-training are Complementary for Natural Language Understanding
Task-adaptive pre-training (TAPT) and Self-training (ST) have emerged as the major semi-supervised approaches to improve natural language understanding (NLU) tasks with massive amount of unlabeled data.
Semi-Supervised Hypothesis Transfer for Source-Free Domain Adaptation
Domain Adaptation has been widely used to deal with the distribution shift in vision, language, multimedia etc.
Lifelong Learning of Hate Speech Classification on Social Media
In this work, we propose lifelong learning of hate speech classification on social media.
Inductive Relation Prediction by BERT
Relation prediction in knowledge graphs is dominated by embedding based methods which mainly focus on the transductive setting.
Composite Re-Ranking for Efficient Document Search with BERT
By applying token encoding on top of a dual-encoder architecture, BECR separates the attentions between a query and a document while capturing the contextual semantics of a query.
Cross-modal Image Retrieval with Deep Mutual Information Maximization
Prior work usually uses a three-stage strategy to tackle this task: 1) extract the features of the inputs; 2) fuse the feature of the source image and its modified text to obtain fusion feature; 3) learn a similarity metric between the desired image and the source image + modified text by using deep metric learning.
Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Bases
To facilitate the development of KBQA models with stronger generalization, we construct and release a new large-scale, high-quality dataset with 64, 331 questions, GrailQA, and provide evaluation settings for all three levels of generalization.
Inter-Series Attention Model for COVID-19 Forecasting
COVID-19 pandemic has an unprecedented impact all over the world since early 2020.
CoCo: Controllable Counterfactuals for Evaluating Dialogue State Trackers
Dialogue state trackers have made significant progress on benchmark datasets, but their generalization capability to novel and realistic scenarios beyond the held-out conversations is less understood.
Ranked #2 on
Multi-domain Dialogue State Tracking
on MULTIWOZ 2.1
(using extra training data)
KGPT: Knowledge-Grounded Pre-Training for Data-to-Text Generation
We propose a knowledge-grounded pre-training (KGPT), which consists of two parts, 1) a general knowledge-grounded generation model to generate knowledge-enriched text.
Ranked #9 on
KG-to-Text Generation
on WebNLG 2.0 (Unconstrained)
Adaptive-Step Graph Meta-Learner for Few-Shot Graph Classification
The shared sub-structures between training classes and test classes are essential in few-shot graph classification.
Neural Assistant: Joint Action Prediction, Response Generation, and Latent Knowledge Reasoning
In this paper, we develop Neural Assistant: a single neural network model that takes conversation history and an external knowledge source as input and jointly produces both text response and action to be taken by the system as output.
You May Not Need Order in Time Series Forecasting
Time series forecasting with limited data is a challenging yet critical task.
Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting
Time series forecasting is an important problem across many domains, including predictions of solar plant energy output, electricity consumption, and traffic jam situation.
Ranked #27 on
Image Generation
on ImageNet 64x64
(Bits per dim metric)
Global Textual Relation Embedding for Relational Understanding
Pre-trained embeddings such as word embeddings and sentence embeddings are fundamental tools facilitating a wide range of downstream NLP tasks.
Ranked #142 on
Action Classification
on Kinetics-400
Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention
Semantically controlled neural response generation on limited-domain has achieved great performance.
Ranked #5 on
Data-to-Text Generation
on MULTIWOZ 2.1
How Large a Vocabulary Does Text Classification Need? A Variational Approach to Vocabulary Selection
Under deep neural networks, a pre-defined vocabulary is required to vectorize text inputs.
What It Takes to Achieve 100\% Condition Accuracy on WikiSQL
The SQL queries in WikiSQL are simple: Each involves one relation and does not have any join operation.
XL-NBT: A Cross-lingual Neural Belief Tracking Framework
Then, we pre-train a state tracker for the source language as a teacher, which is able to exploit easy-to-access parallel data.
DialSQL: Dialogue Based Structured Query Generation
The recent advance in deep learning and semantic parsing has significantly improved the translation accuracy of natural language questions to structured queries.
Variational Knowledge Graph Reasoning
Inferring missing links in knowledge graphs (KG) has attracted a lot of attention from the research community.
Recovering Question Answering Errors via Query Revision
The existing factoid QA systems often lack a post-inspection component that can help models recover from their own mistakes.
Cross-domain Semantic Parsing via Paraphrasing
Existing studies on semantic parsing mainly focus on the in-domain setting.
Global Relation Embedding for Relation Extraction
We study the problem of textual relation embedding with distant supervision.
Behavior Query Discovery in System-Generated Temporal Graphs
In this work, we investigate how to query temporal graphs and treat query formulation as a discriminative temporal graph pattern mining problem.
