Search Results for author:
Found 63 papers, 22 papers with code
Learning Adaptive Axis Attentions in Fine-tuning: Beyond Fixed Sparse Attention Patterns
We present a comprehensive study of sparse attention patterns in Transformer models.
Influence Diagram Bandits
We experiment with three structured bandit problems: cascading bandits, online learning to rank in the position-based model, and rank-1 bandits.
Few-Shot Class-Incremental Learning for Named Entity Recognition
In this work, we study a more challenging but practical problem, i. e., few-shot class-incremental learning for NER, where an NER model is trained with only few labeled samples of the new classes, without forgetting knowledge of the old ones.
Automatic Layout Planning for Visually-Rich Documents with Instruction-Following Models
Recent advancements in instruction-following models have made user interactions with models more user-friendly and efficient, broadening their applicability.
SOHES: Self-supervised Open-world Hierarchical Entity Segmentation
Using raw images as the sole training data, our method achieves unprecedented performance in self-supervised open-world segmentation, marking a significant milestone towards high-quality open-world entity segmentation in the absence of human-annotated masks.
BiLoRA: A Bi-level Optimization Framework for Overfitting-Resilient Low-Rank Adaptation of Large Pre-trained Models
Low-rank adaptation (LoRA) is a popular method for fine-tuning large-scale pre-trained models in downstream tasks by learning low-rank incremental matrices.
AutoLoRA: Automatically Tuning Matrix Ranks in Low-Rank Adaptation Based on Meta Learning
Large-scale pretraining followed by task-specific finetuning has achieved great success in various NLP tasks.
Downstream Task Guided Masking Learning in Masked Autoencoders Using Multi-Level Optimization
Masked Autoencoder (MAE) is a notable method for self-supervised pretraining in visual representation learning.
BLO-SAM: Bi-level Optimization Based Overfitting-Preventing Finetuning of SAM
Secondly, SAM faces challenges in excelling at specific downstream tasks, like medical imaging, due to a disparity between the distribution of its pretraining data, which predominantly consists of general-domain images, and the data used in downstream tasks.
Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints
Controllable layout generation refers to the process of creating a plausible visual arrangement of elements within a graphic design (e. g., document and web designs) with constraints representing design intentions.
Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes
Large language models (LLMs) have shown remarkable advances in language generation and understanding but are also prone to exhibiting harmful social biases.
Customization Assistant for Text-to-image Generation
Some existing methods do not require fine-tuning, while their performance are unsatisfactory.
VaQuitA: Enhancing Alignment in LLM-Assisted Video Understanding
Recent advancements in language-model-based video understanding have been progressing at a remarkable pace, spurred by the introduction of Large Language Models (LLMs).
Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information
Our work aims to address this concern by introducing a novel approach to detecting adversarial prompts at a token level, leveraging the LLM's capability to predict the next token's probability.
Improving a Named Entity Recognizer Trained on Noisy Data with a Few Clean Instances
To achieve state-of-the-art performance, one still needs to train NER models on large-scale, high-quality annotated data, an asset that is both costly and time-intensive to accumulate.
AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models
Safety alignment of Large Language Models (LLMs) can be compromised with manual jailbreak attacks and (automatic) adversarial attacks.
Bias and Fairness in Large Language Models: A Survey
Rapid advancements of large language models (LLMs) have enabled the processing, understanding, and generation of human-like text, with increasing integration into systems that touch our social sphere.
Knowledge Graph Prompting for Multi-Document Question Answering
Concurrently, the graph traversal agent acts as a local navigator that gathers pertinent context to progressively approach the question and guarantee retrieval quality.
Learning Navigational Visual Representations with Semantic Map Supervision
Being able to perceive the semantics and the spatial structure of the environment is essential for visual navigation of a household robot.
LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding
Instruction tuning unlocks the superior capability of Large Language Models (LLM) to interact with humans.
Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels
Remarkably, by incorporating conditional information from the powerful CLIP model, our method can boost the current SOTA accuracy by 10-20 absolute points in many cases.
Ranked #1 on
Image Classification
on Food-101N
(using extra training data)
Enhancing Detail Preservation for Customized Text-to-Image Generation: A Regularization-Free Approach
However, generating images of novel concept provided by the user input image is still a challenging task.
Few-Shot Dialogue Summarization via Skeleton-Assisted Prompt Transfer in Prompt Tuning
In this paper, we focus on improving the prompt transfer from dialogue state tracking to dialogue summarization and propose Skeleton-Assisted Prompt Transfer (SAPT), which leverages skeleton generation as extra supervision that functions as a medium connecting the distinct source and target task and resulting in the model's better consumption of dialogue state information.
DrugChat: Towards Enabling ChatGPT-Like Capabilities on Drug Molecule Graphs
The DrugChat system consists of a graph neural network (GNN), a large language model (LLM), and an adaptor.
Towards Building the Federated GPT: Federated Instruction Tuning
This repository offers a foundational framework for exploring federated fine-tuning of LLMs using heterogeneous instructions across diverse categories.
Robustness of Demonstration-based Learning Under Limited Data Scenario
Demonstration-based learning has shown great potential in stimulating pretrained language models' ability under limited data scenario.
STT: Soft Template Tuning for Few-Shot Adaptation
Significantly, it can even outperform the time- and resource-consuming fine-tuning method on sentiment classification tasks.
Federated Non-negative Matrix Factorization for Short Texts Topic Modeling with Mutual Information
Non-negative matrix factorization (NMF) based topic modeling is widely used in natural language processing (NLP) to uncover hidden topics of short text documents.
Towards Language-Free Training for Text-to-Image Generation
One of the major challenges in training text-to-image generation models is the need of a large number of high-quality text-image pairs.
LAFITE: Towards Language-Free Training for Text-to-Image Generation
One of the major challenges in training text-to-image generation models is the need of a large number of high-quality image-text pairs.
Ranked #2 on
Text-to-Image Generation
on Multi-Modal-CelebA-HQ
Graphine: A Dataset for Graph-aware Terminology Definition Generation
Unfortunately, the lack of large-scale terminology definition dataset hinders the process toward definition generation.
Improving Zero-shot Voice Style Transfer via Disentangled Representation Learning
Voice style transfer, also called voice conversion, seeks to modify one speaker's voice to generate speech as if it came from another (target) speaker.
Reinforcement Learning for Flexibility Design Problems
Flexibility design problems are a class of problems that appear in strategic decision-making across industries, where the objective is to design a ($e. g.$, manufacturing) network that affords flexibility and adaptivity.
SDA: Improving Text Generation with Self Data Augmentation
Data augmentation has been widely used to improve deep neural networks in many research fields, such as computer vision.
Semantic Matching for Sequence-to-Sequence Learning
In sequence-to-sequence models, classical optimal transport (OT) can be applied to semantically match generated sentences with target sentences.
Improving Text Generation with Student-Forcing Optimal Transport
An extension is further proposed to improve the OT learning, based on the structural and contextual information of the text sequences.
Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference
The neural attention mechanism plays an important role in many natural language processing applications.
Unsupervised Abstractive Dialogue Summarization for Tete-a-Tetes
High-quality dialogue-summary paired data is expensive to produce and domain-sensitive, making abstractive dialogue summarization a challenging task.
Influence Diagram Bandits: Variational Thompson Sampling for Structured Bandit Problems
We propose a novel framework for structured bandits, which we call an influence diagram bandit.
When does MAML Work the Best? An Empirical Study on Model-Agnostic Meta-Learning in NLP Applications
Model-Agnostic Meta-Learning (MAML), a model-agnostic meta-learning method, is successfully employed in NLP applications including few-shot text classification and multi-domain low-resource language generation.
Reward Constrained Interactive Recommendation with Natural Language Feedback
Text-based interactive recommendation provides richer user feedback and has demonstrated advantages over traditional interactive recommender systems.
Improving Adversarial Text Generation by Modeling the Distant Future
Auto-regressive text generation models usually focus on local fluency, and may cause inconsistent semantic meaning in long text generation.
Bayesian Meta Sampling for Fast Uncertainty Adaptation
Specifically, we propose a Bayesian meta sampling framework consisting of two main components: a meta sampler and a sample adapter.
GenDICE: Generalized Offline Estimation of Stationary Values
An important problem that arises in reinforcement learning and Monte Carlo methods is estimating quantities defined by the stationary distribution of a Markov chain.
Nested-Wasserstein Self-Imitation Learning for Sequence Generation
Reinforcement learning (RL) has been widely studied for improving sequence-generation models.
Learning Diverse Stochastic Human-Action Generators by Learning Smooth Latent Transitions
In this paper, we focus on skeleton-based action generation and propose to model smooth and diverse transitions on a latent space of action sequences with much lower dimensionality.
Ranked #4 on
Human action generation
on NTU RGB+D 2D
Text-Based Interactive Recommendation via Constraint-Augmented Reinforcement Learning
Text-based interactive recommendation provides richer user preferences and has demonstrated advantages over traditional interactive recommender systems.
Learning to Recommend from Sparse Data via Generative User Feedback
To address this, we propose a learning framework that improves collaborative filtering with a synthetic feedback loop (CF-SFL) to simulate the user feedback.
Improving Textual Network Learning with Variational Homophilic Embeddings
This paper considers a novel variational formulation of network embeddings, with special focus on textual networks.
Figure Captioning with Reasoning and Sequence-Level Training
In this work, we investigate the problem of figure captioning where the goal is to automatically generate a natural language description of the figure.
Topic-Guided Variational Auto-Encoder for Text Generation
We propose a topic-guided variational auto-encoder (TGVAE) model for text generation.
Topic-Guided Variational Autoencoders for Text Generation
We propose a topic-guided variational autoencoder (TGVAE) model for text generation.
Scalable Thompson Sampling via Optimal Transport
Thompson sampling (TS) is a class of algorithms for sequential decision-making, which requires maintaining a posterior distribution over a model.
Improving Sequence-to-Sequence Learning via Optimal Transport
Sequence-to-sequence models are commonly trained via maximum likelihood estimation (MLE).
Sequence Generation with Guider Network
Sequence generation with reinforcement learning (RL) has received significant attention recently.
Towards More Theoretically-Grounded Particle Optimization Sampling for Deep Learning
With such theoretical guarantees, SPOS can be safely and effectively applied on both Bayesian DL and deep RL tasks.
Stochastic Particle-Optimization Sampling and the Non-Asymptotic Convergence Theory
Particle-optimization-based sampling (POS) is a recently developed effective sampling technique that interactively updates a set of particles.
Policy Optimization as Wasserstein Gradient Flows
Policy optimization is a core component of reinforcement learning (RL), and most existing RL methods directly optimize parameters of a policy based on maximizing the expected total reward, or its surrogate.
Understanding and Accelerating Particle-Based Variational Inference
Particle-based variational inference methods (ParVIs) have gained attention in the Bayesian inference literature, for their capacity to yield flexible and accurate approximations.
Variational Inference and Model Selection with Generalized Evidence Bounds
Recent advances on the scalability and flexibility of variational inference have made it successful at unravelling hidden patterns in complex data.
A Unified Particle-Optimization Framework for Scalable Bayesian Sampling
There has been recent interest in developing scalable Bayesian sampling methods such as stochastic gradient MCMC (SG-MCMC) and Stein variational gradient descent (SVGD) for big-data analysis.
Learning Structural Weight Uncertainty for Sequential Decision-Making
Learning probability distributions on the weights of neural networks (NNs) has recently proven beneficial in many applications.
Particle Optimization in Stochastic Gradient MCMC
Stochastic gradient Markov chain Monte Carlo (SG-MCMC) has been increasingly popular in Bayesian learning due to its ability to deal with large data.
