Search Results for author:
Found 717 papers, 326 papers with code
Deep Streaming Label Learning
In order to fill in these research gaps, we propose a novel deep neural network (DNN) based framework, Deep Streaming Label Learning (DSLL), to classify instances with newly emerged labels effectively.
Redistributing Low-Frequency Words: Making the Most of Monolingual Data in Non-Autoregressive Translation
In this work, we provide an appealing alternative for NAT – monolingual KD, which trains NAT student on external monolingual data with AT teacher trained on the original bilingual data.
Label-Noise Robust Domain Adaptation
Domain adaptation aims to correct the classifiers when faced with distribution shift between source (training) and target (test) domains.
Polysemy Deciphering Network for Human-Object Interaction Detection
First, PD-Net augments human pose and spatial features for HOI detection using language priors, enabling the verb classifiers to receive language hints that reduce the intra-class variation of the same verb.
On Dropping Clusters to Regularize Graph Convolutional Neural Networks
Dropout has been widely adopted to regularize graph convolutional networks (GCNs) by randomly zeroing entries of the node feature vectors and obtains promising performance on various tasks.
LTF: A Label Transformation Framework for Correcting Label Shift
Distribution shift is a major obstacle to the deployment of current deep learning models on real-world problems.
The USYD-JD Speech Translation System for IWSLT2021
Our constrained system is based on a pipeline framework, i. e. ASR and NMT.
Hallucinating Visual Instances in Total Absentia
This seemingly minor difference in fact makes the HVITA a much challenging task, as the restoration algorithm would have to not only infer the category of the object in total absentia, but also hallucinate an object of which the appearance is consistent with the background.
A Survey on Self-Evolution of Large Language Models
Large language models (LLMs) have significantly advanced in various fields and intelligent agent applications.
Learning Feature Inversion for Multi-class Anomaly Detection under General-purpose COCO-AD Benchmark
Moreover, current metrics such as AU-ROC have nearly reached saturation on simple datasets, which prevents a comprehensive evaluation of different methods.
UniMix: Towards Domain Adaptive and Generalizable LiDAR Semantic Segmentation in Adverse Weather
We devote UniMix to two main setups: 1) unsupervised domain adaption, adapting the model from the clear weather source domain to the adverse weather target domain; 2) domain generalization, learning a model that generalizes well to unseen scenes in adverse weather.
Deepfake Generation and Detection: A Benchmark and Survey
Deepfake is a technology dedicated to creating highly realistic facial images and videos under specific conditions, which has significant application potential in fields such as entertainment, movie production, digital human creation, to name a few.
Object Detectors in the Open Environment: Challenges, Solutions, and Outlook
This paper aims to bridge this gap by conducting a comprehensive review and analysis of object detectors in open environments.
Contact-aware Human Motion Generation from Textual Descriptions
This paper addresses the problem of generating 3D interactive human motion from text.
Building Accurate Translation-Tailored LLMs with Language Aware Instruction Tuning
In this work, we design a two-stage fine-tuning algorithm to improve the instruction-following ability (especially the translation direction) of LLMs.
GLC++: Source-Free Universal Domain Adaptation through Global-Local Clustering and Contrastive Affinity Learning
GLC++ enhances the novel category clustering accuracy of GLC by 4. 3% in open-set scenarios on Office-Home.
MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining
However, transferring the pretrained models to downstream tasks may encounter task discrepancy due to their formulation of pretraining as image classification or object discrimination tasks.
Ranked #1 on
Semantic Segmentation
on SpaceNet 1
(using extra training data)
Aerial Scene Classification
Building change detection for remote sensing images
+13
Local-consistent Transformation Learning for Rotation-invariant Point Cloud Analysis
Equipped with LCRF and RPR, our LocoTrans is capable of learning local-consistent transformation and preserving local geometry, which benefits rotation invariance learning.
Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction
Experiments across various prompts, PLMs, and benchmarks show that our approach can not only correct the overfitted performance caused by prompt bias, but also significantly improve the prompt retrieval capability (up to 10% absolute performance gain).
When ControlNet Meets Inexplicit Masks: A Case Study of ControlNet on its Contour-following Ability
Subsequently, to enhance controllability with inexplicit masks, an advanced Shape-aware ControlNet consisting of a deterioration estimator and a shape-prior modulation block is devised.
Trajectory Consistency Distillation: Improved Latent Consistency Distillation by Semi-Linear Consistency Function with Trajectory Mapping
Consequently, we introduce Trajectory Consistency Distillation (TCD), which encompasses trajectory consistency function and strategic stochastic sampling.
A Survey on Knowledge Distillation of Large Language Models
In the era of Large Language Models (LLMs), Knowledge Distillation (KD) emerges as a pivotal methodology for transferring advanced capabilities from leading proprietary LLMs, such as GPT-4, to their open-source counterparts like LLaMA and Mistral.
Healthcare Copilot: Eliciting the Power of General LLMs for Medical Consultation
The copilot framework, which aims to enhance and tailor large language models (LLMs) for specific complex tasks without requiring fine-tuning, is gaining increasing attention from the community.
Towards Theoretical Understandings of Self-Consuming Generative Models
This paper tackles the emerging challenge of training generative models within a self-consuming loop, wherein successive generations of models are recursively trained on mixtures of real and synthetic data from previous generations.
Communication-Efficient Distributed Learning with Local Immediate Error Compensation
However, existing compression methods either perform only unidirectional compression in one iteration with higher communication cost, or bidirectional compression with slower convergence rate.
DB-LLM: Accurate Dual-Binarization for Efficient LLMs
Large language models (LLMs) have significantly advanced the field of natural language processing, while the expensive memory and computation consumption impede their practical deployment.
Revisiting Knowledge Distillation for Autoregressive Language Models
Knowledge distillation (KD) is a common approach to compress a teacher model to reduce its inference cost and memory footprint, by training a smaller student model.
ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decoding
With the development of instruction-tuned large language models (LLMs), improving the safety of LLMs has become more critical.
Continual Learning on Graphs: Challenges, Solutions, and Opportunities
To bridge the gap, we provide a comprehensive review of existing continual graph learning (CGL) algorithms by elucidating the different task settings and categorizing the existing methods based on their characteristics.
Mitigating Reward Hacking via Information-Theoretic Reward Modeling
Inspired by this finding, we propose the Integrated Cluster Deviation Score (ICDS), which quantifies deviations in the latent space, as an indicator of reward overoptimization to facilitate the development of online mitigation strategies.
Confronting Reward Overoptimization for Diffusion Models: A Perspective of Inductive and Primacy Biases
Then, we surprisingly discover that dormant neurons in our critic model act as a regularization against overoptimization, while active neurons reflect primacy bias in this setting.
Large Language Models as an Indirect Reasoner: Contrapositive and Contradiction for Automated Reasoning
The experimental results on popular LLMs, such as GPT-3. 5-turbo and Gemini-pro, show that our IR method enhances the overall accuracy of factual reasoning by 27. 33% and mathematical proof by 31. 43%, when compared with traditional DR methods.
A Survey on Transformer Compression
Model compression methods reduce the memory and computational cost of Transformer, which is a necessary step to implement large language/vision models on practical devices.
Poisson Process for Bayesian Optimization
BayesianOptimization(BO) is a sample-efficient black-box optimizer, and extensive methods have been proposed to build the absolute function response of the black-box function through a probabilistic surrogate model, including Tree-structured Parzen Estimator (TPE), random forest (SMAC), and Gaussian process (GP).
Representation Surgery for Multi-Task Model Merging
That is, there is a significant discrepancy in the representation distribution between the merged and individual models, resulting in poor performance of merged MTL.
FreDF: Learning to Forecast in Frequency Domain
Time series modeling is uniquely challenged by the presence of autocorrelation in both historical and label sequences.
Merging Multi-Task Models via Weight-Ensembling Mixture of Experts
A notable challenge is mitigating the interference between parameters of different models, which can substantially deteriorate performance.
Fine-Grained Zero-Shot Learning: Advances, Challenges, and Prospects
To enrich the literature of this domain and provide a sound basis for its future development, in this paper, we present a broad review of recent advances for fine-grained analysis in ZSL.
Hi-SAM: Marrying Segment Anything Model for Hierarchical Text Segmentation
In terms of the AMG mode, Hi-SAM segments text stroke foreground masks initially, then samples foreground points for hierarchical text mask generation and achieves layout analysis in passing.
Ranked #1 on
Hierarchical Text Segmentation
on HierText
Topology-aware Embedding Memory for Continual Learning on Expanding Networks
Memory replay based techniques have shown great success for continual learning with incrementally accumulated Euclidean data.
TD^2-Net: Toward Denoising and Debiasing for Dynamic Scene Graph Generation
To address the above problems, in this paper, we introduce a network named TD$^2$-Net that aims at denoising and debiasing for dynamic SGG.
Revisiting Demonstration Selection Strategies in In-Context Learning
In this work, we first revisit the factors contributing to this variance from both data and model aspects, and find that the choice of demonstration is both data- and model-dependent.
Solving Continual Offline Reinforcement Learning with Decision Transformer
We aim to investigate whether Decision Transformer (DT), another offline RL paradigm, can serve as a more suitable offline continuous learner to address these issues.
GoMatching: A Simple Baseline for Video Text Spotting via Long and Short Term Matching
In response to this issue, we propose to efficiently turn an off-the-shelf query-based image text spotter into a specialist on video and present a simple baseline termed GoMatching, which focuses the training efforts on tracking while maintaining strong recognition performance.
Intention Analysis Makes LLMs A Good Jailbreak Defender
Aligning large language models (LLMs) with human values, particularly in the face of stealthy and complex jailbreak attacks, presents a formidable challenge.
OOP: Object-Oriented Programming Evaluation Benchmark for Large Language Models
Advancing automated programming necessitates robust and comprehensive code generation benchmarks, yet current evaluation frameworks largely neglect object-oriented programming (OOP) in favor of functional programming (FP), e. g., HumanEval and MBPP.
WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledge
Sentiment analysis is rapidly advancing by utilizing various data modalities (e. g., text, image).
Hi-Map: Hierarchical Factorized Radiance Field for High-Fidelity Monocular Dense Mapping
In this paper, we introduce Hi-Map, a novel monocular dense mapping approach based on Neural Radiance Field (NeRF).
PanGu-$π$: Enhancing Language Model Architectures via Nonlinearity Compensation
We then demonstrate that the proposed approach is significantly effective for enhancing the model nonlinearity through carefully designed ablations; thus, we present a new efficient model architecture for establishing modern, namely, PanGu-$\pi$.
Pre-trained Trojan Attacks for Visual Recognition
This paper aims to raise awareness of the potential threats associated with applying PVMs in practical scenarios.
ConDaFormer: Disassembled Transformer with Local Structure Enhancement for 3D Point Cloud Understanding
Transformers have been recently explored for 3D point cloud understanding with impressive progress achieved.
Ranked #5 on
Semantic Segmentation
on S3DIS Area5
Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detection
Following this spirit, this paper explores plain ViT architecture for MUAD.
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion
At the upper level, we focus on learning a shared Concrete mask to identify the subspace, while at the inner level, model merging is performed to maximize the performance of the merged model.
Disentangled Interaction Representation for One-Stage Human-Object Interaction Detection
In this paper, we improve the performance of one-stage methods by enabling them to extract disentangled interaction representations.
Good Questions Help Zero-Shot Image Reasoning
QVix enables a wider exploration of visual scenes, improving the LVLMs' reasoning accuracy and depth in tasks such as visual question answering and visual entailment.
HandRefiner: Refining Malformed Hands in Generated Images by Diffusion-based Conditional Inpainting
Given a generated failed image due to malformed hands, we utilize ControlNet modules to re-inject such correct hand information.
One More Step: A Versatile Plug-and-Play Module for Rectifying Diffusion Schedule Flaws and Enhancing Low-Frequency Controls
By integrating a compact network and incorporating an additional simple yet effective step during inference, OMS elevates image fidelity and harmonizes the dichotomy between training and inference, while preserving original model parameters.
Task-Distributionally Robust Data-Free Meta-Learning
TDS leads to a biased meta-learner because of the skewed task distribution towards newly generated tasks.
DA-STC: Domain Adaptive Video Semantic Segmentation via Spatio-Temporal Consistency
To solve the challenge, we propose a novel DA-STC method for domain adaptive video semantic segmentation, which incorporates a bidirectional multi-level spatio-temporal fusion module and a category-aware spatio-temporal feature alignment module to facilitate consistent learning for domain-invariant features.
Optical Quantum Sensing for Agnostic Environments via Deep Learning
DQS incorporates two essential components: a Graph Neural Network (GNN) predictor and a trigonometric interpolation algorithm.
Multimodal deep representation learning for quantum cross-platform verification
Cross-platform verification, a critical undertaking in the realm of early-stage quantum computing, endeavors to characterize the similarity of two imperfect quantum devices executing identical algorithms, utilizing minimal measurements.
Zero-Shot Sharpness-Aware Quantization for Pre-trained Language Models
The key algorithm in solving ZSAQ is the SAM-SGA optimization, which aims to improve the quantization accuracy and model generalization via optimizing a minimax problem.
Stochastic Optimization for Non-convex Problem with Inexact Hessian Matrix, Gradient, and Function
In this paper, we explore a family of stochastic TR and ARC methods that can simultaneously provide inexact computations of the Hessian matrix, gradient, and function values.
Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer
The Mixture of Experts (MoE) has emerged as a highly successful technique in deep learning, based on the principle of divide-and-conquer to maximize model capacity without significant additional computational cost.
Merging Experts into One: Improving Computational Efficiency of Mixture of Experts
Although a sparse Mixture of Experts (MoE) can reduce the cost by activating a small subset of parameters (e. g., one expert) for each input, its computation escalates significantly if increasing the number of activated experts, limiting its practical utility.
Learn From Model Beyond Fine-Tuning: A Survey
LFM focuses on the research, modification, and design of FM based on the model interface, so as to better understand the model structure and weights (in a black box environment), and to generalize the model to downstream tasks.
Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stages
Plasticity, the ability of a neural network to evolve with new data, is crucial for high-performance and sample-efficient visual reinforcement learning (VRL).
PointHR: Exploring High-Resolution Architectures for 3D Point Cloud Segmentation
Significant progress has been made recently in point cloud segmentation utilizing an encoder-decoder framework, which initially encodes point clouds into low-resolution representations and subsequently decodes high-resolution predictions.
Ranked #5 on
Semantic Segmentation
on ScanNet
Parameter Efficient Multi-task Model Fusion with Partial Linearization
We demonstrate that our partial linearization technique enables a more effective fusion of multiple tasks into a single model, outperforming standard adapter tuning and task arithmetic alone.
Which mode is better for federated learning? Centralized or Decentralized
Both centralized and decentralized approaches have shown excellent performance and great application value in federated learning (FL).
Efficient Federated Prompt Tuning for Black-box Large Pre-trained Models
With the blowout development of pre-trained models (PTMs), the efficient tuning of these models for diverse downstream applications has emerged as a pivotal research concern.
AdaMerging: Adaptive Model Merging for Multi-Task Learning
This approach aims to autonomously learn the coefficients for model merging, either in a task-wise or layer-wise manner, without relying on the original training data.
One for All: Towards Training One Graph Model for All Classification Tasks
For in-context learning on graphs, OFA introduces a novel graph prompting paradigm that appends prompting substructures to the input graph, which enables it to address varied tasks without fine-tuning.
Unlikelihood Tuning on Negative Samples Amazingly Improves Zero-Shot Translation
Zero-shot translation (ZST), which is generally based on a multilingual neural machine translation model, aims to translate between unseen language pairs in training data.
Global Context Aggregation Network for Lightweight Saliency Detection of Surface Defects
To this end, we develop a Global Context Aggregation Network (GCANet) for lightweight saliency detection of surface defects on the encoder-decoder structure.
CINFormer: Transformer network with multi-stage CNN feature injection for surface defect segmentation
To address these issues, we propose a transformer network with multi-stage CNN (Convolutional Neural Network) feature injection for surface defect segmentation, which is a UNet-like structure named CINFormer.
Are Large Language Models Really Robust to Word-Level Perturbations?
Notably, we are surprised to discover that robustness tends to decrease as fine-tuning (SFT and RLHF) is conducted.
Graph Contrastive Learning Meets Graph Meta Learning: A Unified Method for Few-shot Node Tasks
In our study, we first identify two crucial advantages of contrastive learning compared to meta learning, including (1) the comprehensive utilization of graph nodes and (2) the power of graph augmentations.
FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Data
Federated learning is an emerging distributed machine learning method, enables a large number of clients to train a model without exchanging their local data.
Temporal Action Localization with Enhanced Instant Discriminability
Temporal action detection (TAD) aims to detect all action boundaries and their corresponding categories in an untrimmed video.
Ranked #1 on
Temporal Action Localization
on MultiTHUMOS
RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model
Text-to-image generation (TTI) refers to the usage of models that could process text input and generate high fidelity images based on text descriptions.
DARC: Distribution-Aware Re-Coloring Model for Generalizable Nucleus Segmentation
Second, we propose a new instance normalization method that is robust to the variation in foreground-background ratios.
Continual Learning From a Stream of APIs
Next, we train the CL model by minimizing the gap between the responses of the CL model and the black-box API on synthetic data, to transfer the API's knowledge to the CL model.
CktGNN: Circuit Graph Neural Network for Electronic Design Automation
The electronic design automation of analog circuits has been a longstanding challenge in the integrated circuit field due to the huge design space and complex design trade-offs among circuit specifications.
MerA: Merging Pretrained Adapters For Few-Shot Learning
Adapter tuning, which updates only a few parameters, has become a mainstream method for fine-tuning pretrained language models to downstream tasks.
Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models
We evaluate our method on both open and closed LLMs, and the experiments on the widely-used public dataset show that our method can generate more consistent responses in a long-context conversation.
ShadowNet for Data-Centric Quantum System Learning
Understanding the dynamics of large quantum systems is hindered by the curse of dimensionality.
Towards Understanding the Generalizability of Delayed Stochastic Gradient Descent
Stochastic gradient descent (SGD) performed in an asynchronous manner plays a crucial role in training large-scale machine learning models.
DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learning
To address the communication burden issues associated with federated learning (FL), decentralized federated learning (DFL) discards the central server and establishes a decentralized communication network, where each client communicates only with neighboring clients.
Deformable Mixer Transformer with Gating for Multi-Task Learning of Dense Prediction
In this work, we present a novel MTL model by combining both merits of deformable CNN and query-based Transformer with shared gating for multi-task learning of dense prediction.
Cross-modal & Cross-domain Learning for Unsupervised LiDAR Semantic Segmentation
In recent years, cross-modal domain adaptation has been studied on the paired 2D image and 3D LiDAR data to ease the labeling costs for 3D LiDAR semantic segmentation (3DLSS) in the target domain.
Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants
Beyond the normal case, long-tail class incremental learning and few-shot class incremental learning are also proposed to consider the data imbalance and data scarcity, respectively, which are common in real-world implementations and further exacerbate the well-known problem of catastrophic forgetting.
Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedup
Adaptive optimization has achieved notable success for distributed learning while extending adaptive optimizer to federated Learning (FL) suffers from severe inefficiency, including (i) rugged convergence due to inaccurate gradient estimation in global adaptive optimizer; (ii) client drifts exacerbated by local over-fitting with the local adaptive optimizer.
PNT-Edge: Towards Robust Edge Detection with Noisy Labels by Learning Pixel-level Noise Transitions
Exploiting the estimated noise transitions, our model, named PNT-Edge, is able to fit the prediction to clean labels.
Understanding Deep Neural Networks via Linear Separability of Hidden Layers
In this paper, we measure the linear separability of hidden layer outputs to study the characteristics of deep neural networks.
Patch-Wise Point Cloud Generation: A Divide-and-Conquer Approach
A generative model for high-fidelity point clouds is of great importance in synthesizing 3d environments for applications such as autonomous driving and robotics.
Heterogeneous Federated Learning: State-of-the-art and Research Challenges
Therefore, a systematic survey on this topic about the research challenges and state-of-the-art is essential.
Image Captions are Natural Prompts for Text-to-Image Models
With the rapid development of Artificial Intelligence Generated Content (AIGC), it has become common practice in many learning tasks to train or fine-tune large models on synthetic data due to the data-scarcity and privacy leakage problems.
Systematic Investigation of Sparse Perturbed Sharpness-Aware Minimization Optimizer
Sharpness-Aware Minimization (SAM) is a popular solution that smooths the loss landscape by minimizing the maximized change of training loss when adding a perturbation to the weight.
GraMMaR: Ground-aware Motion Model for 3D Human Motion Reconstruction
Demystifying complex human-ground interactions is essential for accurate and realistic 3D human motion reconstruction from RGB videos, as it ensures consistency between the humans and the ground plane.
Towards Open Vocabulary Learning: A Survey
To our knowledge, this is the first comprehensive literature review of open vocabulary learning.
FHA-Kitchens: A Novel Dataset for Fine-Grained Hand Action Recognition in Kitchen Scenes
We represent the action information in each hand interaction region as a triplet, resulting in a total of 878 action triplets.
Structured Cooperative Learning with Graphical Model Priors
We propose "Structured Cooperative Learning (SCooL)", in which a cooperation graph across devices is generated by a graphical model prior to automatically coordinate mutual learning between devices.
Learning Geometric Transformation for Point Cloud Completion
It exploits the repetitive geometric structures in common 3D objects to recover the complete shapes, which contains three sub-networks: geometric patch network, structure transformation network, and detail refinement network.
Ranked #4 on
Point Cloud Completion
on ShapeNet
Instructed Diffuser with Temporal Condition Guidance for Offline Reinforcement Learning
Recent works have shown the potential of diffusion models in computer vision and natural language processing.
Transition role of entangled data in quantum machine learning
Recent progress has highlighted its positive impact on learning quantum dynamics, wherein the integration of entanglement into quantum operations or measurements of quantum machine learning (QML) models leads to substantial reductions in training data size, surpassing a specified prediction error threshold.
Human-imperceptible, Machine-recognizable Images
Massive human-related data is collected to train neural networks for computer vision tasks.
Unsupervised Dense Retrieval with Relevance-Aware Contrastive Pre-Training
Dense retrievers have achieved impressive performance, but their demand for abundant training data limits their application scenarios.
MotionTrack: Learning Motion Predictor for Multiple Object Tracking
This challenge arises from two main factors: the insufficient discriminability of ReID features and the predominant utilization of linear motion models in MOT.
Ranked #4 on
Multi-Object Tracking
on SportsMOT
Decentralized SGD and Average-direction SAM are Asymptotically Equivalent
Decentralized stochastic gradient descent (D-SGD) allows collaborative learning on massive devices simultaneously without the control of a central server.
Extending the Design Space of Graph Neural Networks by Rethinking Folklore Weisfeiler-Lehman
We theoretically prove that even if we fix the space complexity to $O(n^k)$ (for any $k\geq 2$) in $(k, t)$-FWL, we can construct an expressiveness hierarchy up to solving the graph isomorphism problem.
Ranked #2 on
Graph Regression
on ZINC-500k
Collect-and-Distribute Transformer for 3D Point Cloud Analysis
In this paper, we propose a new transformer network equipped with a collect-and-distribute mechanism to communicate short- and long-range contexts of point clouds, which we refer to as CDFormer.
Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State Tracking
Zero-shot transfer learning for Dialogue State Tracking (DST) helps to handle a variety of task-oriented dialogue domains without the cost of collecting in-domain data.
Cocktail: Mixing Multi-Modality Controls for Text-Conditional Image Generation
In this work, we propose Cocktail, a pipeline to mix various modalities into one embedding, amalgamated with a generalized ControlNet (gControlNet), a controllable normalisation (ControlNorm), and a spatial guidance sampling method, to actualize multi-modal and spatially-refined control for text-conditional diffusion models.
DeepSolo++: Let Transformer Decoder with Explicit Points Solo for Multilingual Text Spotting
In this paper, we present DeepSolo++, a simple DETR-like baseline that lets a single decoder with explicit points solo for text detection, recognition, and script identification simultaneously.
Ranked #1 on
Text Spotting
on Inverse-Text
Learning to Learn from APIs: Black-Box Data-Free Meta-Learning
Data-free meta-learning (DFML) aims to enable efficient learning of new tasks by meta-learning from a collection of pre-trained models without access to the training data.
All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation
We hypothesize that this selective usage arises from the noise in pseudo-labels generated on unlabeled data.
Self-Evolution Learning for Discriminative Language Model Pretraining
Masked language modeling, widely used in discriminative language model (e. g., BERT) pretraining, commonly adopts a random masking strategy.
Revisiting Token Dropping Strategy in Efficient BERT Pretraining
Token dropping is a recently-proposed strategy to speed up the pretraining of masked language models, such as BERT, by skipping the computation of a subset of the input tokens at several middle layers.
Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Training
Personalized federated learning (PFL) aims to produce the greatest personalized model for each client to face an insurmountable problem--data heterogeneity in real FL systems.
Improving Heterogeneous Model Reuse by Density Estimation
This paper studies multiparty learning, aiming to learn a model using the private data of different participants.
VanillaNet: the Power of Minimalism in Deep Learning
In this study, we introduce VanillaNet, a neural network architecture that embraces elegance in design.
Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Tasks
However, most of the mixup methods do not consider the varying degree of learning difficulty in different stages of training and generate new samples with one hot labels, resulting in the model over confidence.
Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape
In federated learning (FL), a cluster of local clients are chaired under the coordination of the global server and cooperatively train one model with privacy protection.
Prompt-Tuning Decision Transformer with Preference Ranking
Our work contributes to the advancement of prompt-tuning approaches in RL, providing a promising direction for optimizing large RL agents for specific preference tasks.
MMoT: Mixture-of-Modality-Tokens Transformer for Composed Multimodal Conditional Image Synthesis
To this end, we propose to generate images conditioned on the compositions of multimodal control signals, where modalities are imperfectly complementary, i. e., composed multimodal conditional image synthesis (CMCIS).
Revolutionizing Agrifood Systems with Artificial Intelligence: A Survey
With the world population rapidly increasing, transforming our agrifood systems to be more productive, efficient, safe, and sustainable is crucial to mitigate potential food shortages.
SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model
In this study, we leverage SAM and existing RS object detection datasets to develop an efficient pipeline for generating a large-scale RS segmentation dataset, dubbed SAMRS.
Scalable Mask Annotation for Video Text Spotting
Video text spotting refers to localizing, recognizing, and tracking textual elements such as captions, logos, license plates, signs, and other forms of text within consecutive video frames.
Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacy
To defend the inference attacks and mitigate the sensitive information leakages in Federated Learning (FL), client-level Differentially Private FL (DPFL) is the de-facto standard for privacy protection by clipping local updates and adding random noise.
Deep Graph Reprogramming
In this paper, we explore a novel model reusing task tailored for graph neural networks (GNNs), termed as "deep graph reprogramming".
Compositional 3D Human-Object Neural Animation
Human-object interactions (HOIs) are crucial for human-centric scene understanding applications such as human-centric visual generation, AR/VR, and robotics.
Segment Anything in Non-Euclidean Domains: Challenges and Opportunities
The recent work known as Segment Anything (SA) has made significant strides in pushing the boundaries of semantic segmentation into the era of foundation models.
HKNAS: Classification of Hyperspectral Imagery Based on Hyper Kernel Neural Architecture Search
Recent neural architecture search (NAS) based approaches have made great progress in hyperspectral image (HSI) classification tasks.
DCN-T: Dual Context Network with Transformer for Hyperspectral Image Classification
Hyperspectral image (HSI) classification is challenging due to spatial variability caused by complex imaging conditions.
Event-based Simultaneous Localization and Mapping: A Comprehensive Survey
This paper presents a timely and comprehensive review of event-based vSLAM algorithms that exploit the benefits of asynchronous and irregular event streams for localization and mapping tasks.
UVA: Towards Unified Volumetric Avatar for View Synthesis, Pose rendering, Geometry and Texture Editing
Experiments on multiple human avatars demonstrate that our UVA achieves competitive results in novel view synthesis and novel pose rendering while enabling local and independent editing of geometry and appearance.
Deep Image Matting: A Comprehensive Survey
Image matting refers to extracting precise alpha matte from natural images, and it plays a critical role in various downstream applications, such as image editing.
On Efficient Training of Large-Scale Deep Learning Models: A Literature Review
The field of deep learning has witnessed significant progress, particularly in computer vision (CV), natural language processing (NLP), and speech.
Quantum Imitation Learning
Despite remarkable successes in solving various complex decision-making tasks, training an imitation learning (IL) algorithm with deep neural networks (DNNs) suffers from the high computation burden.
VTAE: Variational Transformer Autoencoder with Manifolds Learning
This weak projection, however, can be addressed by a Riemannian metric, and we show that geodesics computation and accurate interpolations between data samples on the Riemannian manifold can substantially improve the performance of deep generative models.
SimDistill: Simulated Multi-modal Distillation for BEV 3D Object Detection
Multi-view camera-based 3D object detection has become popular due to its low cost, but accurately inferring 3D geometry solely from camera data remains challenging and may lead to inferior performance.
Vision Transformer with Quadrangle Attention
Window-based attention has become a popular choice in vision transformers due to its superior performance, lower computational complexity, and less memory footprint.
Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models
To further improve the performance of LLMs on MT quality assessment, we investigate several prompting designs, and propose a new prompting method called \textbf{\texttt{Error Analysis Prompting}} (EAPrompt) by combining Chain-of-Thoughts (Wei et al., 2022) and Error Analysis (Lu et al., 2023).
Towards Making the Most of ChatGPT for Machine Translation
We show that: 1) The performance of ChatGPT depends largely on temperature, and a lower temperature usually can achieve better performance; 2) Emphasizing the task information can further improve ChatGPT's performance, particularly in complex MT tasks; 3) Introducing domain information can elicit ChatGPT's generalization ability and improve its performance in the specific domain; 4) ChatGPT tends to generate hallucinations for non-English-centric MT tasks, which can be partially addressed by our proposed prompts but still need to be highlighted for the MT/NLP community.
Architecture, Dataset and Model-Scale Agnostic Data-free Meta-Learning
The goal of data-free meta-learning is to learn useful prior knowledge from a collection of pre-trained models without accessing their training data.
Make Landscape Flatter in Differentially Private Federated Learning
Specifically, DP-FedSAM integrates Sharpness Aware Minimization (SAM) optimizer to generate local flatness models with better stability and weight perturbation robustness, which results in the small norm of local updates and robustness to DP noise, thereby improving the performance.
Deep Learning for Camera Calibration and Beyond: A Survey
In this paper, we provide a comprehensive survey of learning-based camera calibration techniques, by analyzing their strengths and limitations.
Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning
Visual Parameter-Efficient Fine-Tuning (PEFT) has become a powerful alternative for full fine-tuning so as to adapt pre-trained vision models to downstream tasks, which only tunes a small number of parameters while freezing the vast majority ones to ease storage burden and optimization difficulty.
Visual Prompt Based Personalized Federated Learning
Then, the local model is trained on the input composed of raw data and a visual prompt to learn the distribution information contained in the prompt.
Upcycling Models under Domain and Category Shift
We examine the superiority of our GLC on multiple benchmarks with different category shift scenarios, including partial-set, open-set, and open-partial-set DA.
Ranked #2 on
Universal Domain Adaptation
on VisDA2017
TriDet: Temporal Action Detection with Relative Boundary Modeling
In this paper, we present a one-stage framework TriDet for temporal action detection.
Ranked #2 on
Temporal Action Localization
on EPIC-KITCHENS-100
DDS2M: Self-Supervised Denoising Diffusion Spatio-Spectral Model for Hyperspectral Image Restoration
This is especially inappropriate for data-starved hyperspectral image (HSI) restoration.
Centroid-centered Modeling for Efficient Vision Transformer Pre-training
Our proposed approach, \textbf{CCViT}, leverages k-means clustering to obtain centroids for image modeling without supervised training of tokenizer model.
Graph Decision Transformer
Offline reinforcement learning (RL) is a challenging task, whose objective is to learn policies from static trajectory data without interacting with the environment.
Time Associated Meta Learning for Clinical Prediction
We view time-associated disease prediction as classification tasks at multiple time points.
ESceme: Vision-and-Language Navigation with Episodic Scene Memory
Vision-and-language navigation (VLN) simulates a visual agent that follows natural-language navigation instructions in real-world scenes.
OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML System
Automated machine learning (AutoML) seeks to build ML models with minimal human effort.
AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks
Integrating SAM with adaptive learning rate and momentum acceleration, dubbed AdaSAM, has already been explored empirically to train large-scale deep neural networks without theoretical guarantee due to the triple difficulties in analyzing the coupled perturbation step, adaptive learning rate and momentum step.
Subspace based Federated Unlearning
Federated learning (FL) enables multiple clients to train a machine learning model collaboratively without exchanging their local data.
Learning to Generalize Provably in Learning to Optimize
While the optimizer generalization has been recently studied, the optimizee generalization (or learning to generalize) has not been rigorously studied in the L2O context, which is the aim of this paper.
Fusion of Global and Local Knowledge for Personalized Federated Learning
Personalized federated learning, as a variant of federated learning, trains customized models for clients using their heterogeneously distributed data.
FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy
Federated learning is an emerging distributed machine learning framework which jointly trains a global model via a large number of local devices with data privacy protections.
Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT
Recently, ChatGPT has attracted great attention, as it can generate fluent and high-quality responses to human inquiries.
X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detection
In this paper, we take the first step toward the study of adversarial attacks targeted at X-ray prohibited item detection, and reveal the serious threats posed by such attacks in this safety-critical scenario.
Pseudo Contrastive Learning for Graph-based Semi-supervised Learning
Pseudo Labeling is a technique used to improve the performance of semi-supervised Graph Neural Networks (GNNs) by generating additional pseudo-labels based on confident predictions.
Bag of Tricks for Effective Language Model Pretraining and Downstream Adaptation: A Case Study on GLUE
This technical report briefly describes our JDExplore d-team's submission Vega v1 on the General Language Understanding Evaluation (GLUE) leaderboard, where GLUE is a collection of nine natural language understanding tasks, including question answering, linguistic acceptability, sentiment analysis, text similarity, paraphrase detection, and natural language inference.
Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks
Event cameras are bio-inspired sensors that capture the per-pixel intensity changes asynchronously and produce event streams encoding the time, pixel position, and polarity (sign) of the intensity changes.
FedABC: Targeting Fair Competition in Personalized Federated Learning
In particular, we adopt the ``one-vs-all'' training strategy in each client to alleviate the unfair competition between classes by constructing a personalized binary classification problem for each class.
Enhance Local Consistency in Federated Learning: A Multi-Step Inertial Momentum Approach
Federated learning (FL), as a collaborative distributed training paradigm with several edge computing devices under the coordination of a centralized server, is plagued by inconsistent local stationary points due to the heterogeneity of the local partial participation clients, which precipitates the local client-drifts problems and sparks off the unstable and slow convergence, especially on the aggravated heterogeneous dataset.
PointWavelet: Learning in Spectral Domain for 3D Point Cloud Analysis
In this paper, we introduce a new method, PointWavelet, to explore local graphs in the spectral domain via a learnable graph wavelet transform.
Improving the Model Consistency of Decentralized Federated Learning
To mitigate the privacy leakages and communication burdens of Federated Learning (FL), decentralized FL (DFL) discards the central server and each client only communicates with its neighbors in a decentralized communication network.
AniPixel: Towards Animatable Pixel-Aligned Human Avatar
In this paper, we propose AniPixel, a novel animatable and generalizable human avatar reconstruction method that leverages pixel-aligned features for body geometry prediction and RGB color blending.
Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning
In this paper, we deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse, which reveals that the last-layer features of the same class will collapse into a vertex, and the vertices of all classes are aligned with the classifier prototypes, which are formed as a simplex equiangular tight frame (ETF).
Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class-Incremental Learning
In this paper, we deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse, which reveals that the last-layer features of the same class will collapse into a vertex, and the vertices of all classes are aligned with the classifier prototypes, which are formed as a simplex equiangular tight frame (ETF).
Ranked #3 on
Few-Shot Class-Incremental Learning
on CUB-200-2011
SaFormer: A Conditional Sequence Modeling Approach to Offline Safe Reinforcement Learning
Offline safe RL is of great practical relevance for deploying agents in real-world applications.
Global Nash Equilibrium in Non-convex Multi-player Game: Theory and Algorithms
This paper takes conjugate transformation to the formulation of non-convex multi-player games, and casts the complementary problem into a variational inequality (VI) problem with a continuous pseudo-gradient mapping.
A Comprehensive Survey of Dataset Distillation
Dataset distillation, a dataset reduction method, addresses this problem by synthesizing a small typical dataset from substantial data and has attracted much attention from the deep learning community.
A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends
Deep supervised learning algorithms typically require a large volume of labeled data to achieve satisfactory performance.
PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentation
Third, inspired by Mask2Former, based on our meta-architecture, we propose Panoptic-PartFormer++ and design a new part-whole cross-attention scheme to boost part segmentation qualities further.
Class-Aware Patch Embedding Adaptation for Few-Shot Image Classification
This could significantly reduce the efficiency of a large family of few-shot learning algorithms, which have limited data and highly rely on the comparison of image patches.
Leverage Interactive Affinity for Affordance Learning
Perceiving potential "action possibilities" (i. e., affordance) regions of images and learning interactive functionalities of objects from human demonstration is a challenging task due to the diversity of human-object interactions.
Domain Specified Optimization for Deployment Authorization
As a complementary of SDPA, we also propose Target-Combined Deployment Authorization (TPDA), where unauthorized domains are partially accessible, and simplify the DSO method to a perturbation operation on the pseudo predictions, referred to as Target-Dependent Domain-Specified Optimization (TDSO).
Exploring Temporal Concurrency for Video-Language Representation Learning
Paired video and language data is naturally temporal concurrency, which requires the modeling of the temporal dynamics within each modality and the temporal alignment across modalities simultaneously.
Learnable Skeleton-Aware 3D Point Cloud Sampling
In this paper, we introduce a new skeleton-aware learning-to-sample method by learning object skeletons as the prior knowledge to preserve the object geometry and topology information during sampling.
Exploring the Relationship Between Architectural Design and Adversarially Robust Generalization
In particular, we comprehensively evaluated 20 most representative adversarially trained architectures on ImageNette and CIFAR-10 datasets towards multiple l_p-norm adversarial attacks.
From Images to Textual Prompts: Zero-Shot Visual Question Answering With Frozen Large Language Models
To address this issue, we propose Img2Prompt, a plug-and-play module that provides the prompts that can bridge the aforementioned modality and task disconnections, so that LLMs can perform zero-shot VQA tasks without end-to-end training.
Problem-Dependent Power of Quantum Neural Networks on Multi-Class Classification
Using these findings, we propose a method that uses loss dynamics to probe whether a QC may be more effective than a classical classifier on a particular learning task.
On Transforming Reinforcement Learning by Transformer: The Development Trajectory
Transformer, originally devised for natural language processing, has also attested significant success in computer vision.
From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Models
To address this issue, we propose \emph{Img2Prompt}, a plug-and-play module that provides the prompts that can bridge the aforementioned modality and task disconnections, so that LLMs can perform zero-shot VQA tasks without end-to-end training.
Toward Human-Like Evaluation for Natural Language Generation with Error Analysis
To this end, we augment BARTScore by incorporating the human-like error analysis strategies, namely BARTScore++, where the final score consists of both the evaluations of major errors and minor errors.
Original or Translated? On the Use of Parallel Data for Translation Quality Estimation
Machine Translation Quality Estimation (QE) is the task of evaluating translation output in the absence of human-written references.
Localizing Scan Targets from Human Pose for Autonomous Lung Ultrasound Imaging
Ultrasound is progressing toward becoming an affordable and versatile solution to medical imaging.
Safety Correction from Baseline: Towards the Risk-aware Policy in Robotics via Dual-agent Reinforcement Learning
On the other hand, the safe agent mimics the baseline agent for policy improvement and learns to fulfill safety constraints via off-policy RL tuning.
ContraFeat: Contrasting Deep Features for Semantic Discovery
In this paper, we propose a model that automates this process and achieves state-of-the-art semantic discovery performance.
Diff-Font: Diffusion Model for Robust One-Shot Font Generation
Specifically, a large stroke-wise dataset is constructed, and a stroke-wise diffusion model is proposed to preserve the structure and the completion of each generated character.
Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks
Despite a large number of reinforcement learning (RL) methods focusing on safety-critical tasks, there is still a lack of high-quality evaluation of those algorithms that adheres to safety constraints at each decision step under complex and unknown dynamics.
Generating Holistic 3D Human Motion from Speech
This work addresses the problem of generating 3D holistic body motions from human speech.
Ranked #2 on
Gesture Generation
on BEAT2
ViTPose++: Vision Transformer for Generic Body Pose Estimation
In this paper, we show the surprisingly good properties of plain vision transformers for body pose estimation from various aspects, namely simplicity in model structure, scalability in model size, flexibility in training paradigm, and transferability of knowledge between models, through a simple baseline model dubbed ViTPose.
Ranked #1 on
Animal Pose Estimation
on AP-10K
(using extra training data)
Learning to Learn Better for Video Object Segmentation
Recently, the joint learning framework (JOINT) integrates matching based transductive reasoning and online inductive learning to achieve accurate and robust semi-supervised video object segmentation (SVOS).
