Search Results for author:
Found 105 papers, 48 papers with code
Contrastive Quantization based Semantic Code for Generative Recommendation
Finally, we train and test semantic code with with generative retrieval on a sequential recommendation model.
Deep Pattern Network for Click-Through Rate Prediction
These patterns harbor substantial potential to significantly enhance CTR prediction performance.
AV-GAN: Attention-Based Varifocal Generative Adversarial Network for Uneven Medical Image Translation
In this paper, we propose the Attention-Based Varifocal Generative Adversarial Network (AV-GAN), which solves multiple problems in pathologic image translation tasks, such as uneven translation difficulty in different regions, mutual interference of multiple resolution information, and nuclear deformation.
Video Object Segmentation with Dynamic Query Modulation
Storing intermediate frame segmentations as memory for long-range context modeling, spatial-temporal memory-based methods have recently showcased impressive results in semi-supervised video object segmentation (SVOS).
GRA: Detecting Oriented Objects through Group-wise Rotating and Attention
GRA can adaptively capture fine-grained features of objects with diverse orientations, comprising two key components: Group-wise Rotating and Group-wise Attention.
Lodge: A Coarse to Fine Diffusion Network for Long Dance Generation Guided by the Characteristic Dance Primitives
In contrast, the second-stage is the local diffusion, which parallelly generates detailed motion sequences under the guidance of the dance primitives and choreographic rules.
Ranked #1 on
Motion Synthesis
on FineDance
MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models
Gesture synthesis is a vital realm of human-computer interaction, with wide-ranging applications across various fields like film, robotics, and virtual reality.
Follow-Your-Click: Open-domain Regional Image Animation via Short Prompts
Despite recent advances in image-to-video generation, better controllability and local animation are less explored.
Harmonious Group Choreography with Trajectory-Controllable Diffusion
Specifically, to tackle dancer collisions, we introduce a Dance-Beat Navigator capable of generating trajectories for multiple dancers based on the music, complemented by a Distance-Consistency loss to maintain appropriate spacing among trajectories within a reasonable threshold.
SEABO: A Simple Search-Based Method for Offline Imitation Learning
Offline reinforcement learning (RL) has attracted much attention due to its ability in learning from static offline datasets and eliminating the need of interacting with the environment.
Understanding What Affects Generalization Gap in Visual Reinforcement Learning: Theory and Empirical Evidence
Recently, there are many efforts attempting to learn useful policies for continuous control in visual reinforcement learning (RL).
BATON: Aligning Text-to-Audio Model with Human Preference Feedback
With the development of AI-Generated Content (AIGC), text-to-audio models are gaining widespread attention.
CreativeSynth: Creative Blending and Synthesis of Visual Arts based on Multimodal Diffusion
Large-scale text-to-image generative models have made impressive strides, showcasing their ability to synthesize a vast array of high-quality images.
Exploration and Anti-Exploration with Distributional Random Network Distillation
To address this issue, we introduce the Distributional RND (DRND), a derivative of the RND.
Text2Avatar: Text to 3D Human Avatar Generation with Codebook-Driven Body Controllable Attribute
Generating 3D human models directly from text helps reduce the cost and time of character modeling.
1st Place Solution for 5th LSVOS Challenge: Referring Video Object Segmentation
The recent transformer-based models have dominated the Referring Video Object Segmentation (RVOS) task due to the superior performance.
Exploring Multi-Modal Control in Music-Driven Dance Generation
Existing music-driven 3D dance generation methods mainly concentrate on high-quality dance generation, but lack sufficient control during the generation process.
Chain of Generation: Multi-Modal Gesture Synthesis via Cascaded Conditional Control
We introduce a novel method that separates priors from speech and employs multimodal priors as constraints for generating gestures.
Realistic Human Motion Generation with Cross-Diffusion Models
Our method integrates 3D and 2D information using a shared transformer network within the training of the diffusion model, unifying motion noise into a single feature space.
Semi-supervised Semantic Segmentation Meets Masked Modeling:Fine-grained Locality Learning Matters in Consistency Regularization
Semi-supervised semantic segmentation aims to utilize limited labeled images and abundant unlabeled images to achieve label-efficient learning, wherein the weak-to-strong consistency regularization framework, popularized by FixMatch, is widely used as a benchmark scheme.
WarpDiffusion: Efficient Diffusion Model for High-Fidelity Virtual Try-on
Image-based Virtual Try-On (VITON) aims to transfer an in-shop garment image onto a target person.
MagicStick: Controllable Video Editing via Control Handle Transformations
Yet succinct, our method is the first method to show the ability of video property editing from the pre-trained text-to-image model.
Bridging the Gap: A Unified Video Comprehension Framework for Moment Retrieval and Highlight Detection
Video Moment Retrieval (MR) and Highlight Detection (HD) have attracted significant attention due to the growing demand for video analysis.
Ranked #1 on
Highlight Detection
on YouTube Highlights
Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model
The direct preference optimization (DPO) method, effective in fine-tuning large language models, eliminates the necessity for a reward model.
Replay-enhanced Continual Reinforcement Learning
On the other hand, offline learning on replayed tasks while learning a new task may induce a distributional shift between the dataset and the learned policy on old tasks, resulting in forgetting.
Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model
These priors are subsequently utilized by RGformer to guide the decomposition of image features into their respective reflectance and illumination components.
The primacy bias in Model-based RL
The primacy bias in deep reinforcement learning (DRL), which refers to the agent's tendency to overfit early data and lose the ability to learn from new data, can significantly decrease the performance of DRL algorithms.
Consistent123: One Image to Highly Consistent 3D Asset Using Case-Aware Diffusion Priors
However, due to utilizing the case-agnostic rigid strategy, their generalization ability to arbitrary cases and the 3D consistency of reconstruction are still poor.
UniHead: Unifying Multi-Perception for Detection Heads
More precisely, our approach (1) introduces deformation perception, enabling the model to adaptively sample object features; (2) proposes a Dual-axial Aggregation Transformer (DAT) to adeptly model long-range dependencies, thereby achieving global perception; and (3) devises a Cross-task Interaction Transformer (CIT) that facilitates interaction between the classification and localization branches, thus aligning the two tasks.
Benchmarking Robustness and Generalization in Multi-Agent Systems: A Case Study on Neural MMO
We present the results of the second Neural MMO challenge, hosted at IJCAI 2022, which received 1600+ submissions.
Time-aligned Exposure-enhanced Model for Click-Through Rate Prediction
Click-Through Rate (CTR) prediction, crucial in applications like recommender systems and online advertising, involves ranking items based on the likelihood of user clicks.
Parallel Knowledge Enhancement based Framework for Multi-behavior Recommendation
In the fusion step, advanced neural networks are used to model the hierarchical correlations between user behaviors.
Strategic Preys Make Acute Predators: Enhancing Camouflaged Object Detectors by Generating Camouflaged Objects
On the prey side, we propose an adversarial training framework, Camouflageator, which introduces an auxiliary generator to generate more camouflaged objects that are harder for a COD method to detect.
Consistency Regularization for Generalizable Source-free Domain Adaptation
In this paper, we propose a consistency regularization framework to develop a more generalizable SFDA method, which simultaneously boosts model performance on both target training and testing datasets.
Ada-DQA: Adaptive Diverse Quality-aware Feature Acquisition for Video Quality Assessment
Video quality assessment (VQA) has attracted growing attention in recent years.
HQG-Net: Unpaired Medical Image Enhancement with High-Quality Guidance
Specifically, we extract features from an HQ image and explicitly insert the features, which are expected to encode HQ cues, into the enhancement network to guide the LQ enhancement with the variational normalization module.
Source-Free Domain Adaptive Fundus Image Segmentation with Class-Balanced Mean Teacher
This paper aims to address these two issues by proposing the Class-Balanced Mean Teacher (CBMT) model.
Zero-shot Preference Learning for Offline RL via Optimal Transport
In this paper, we propose a novel zero-shot preference-based RL algorithm that leverages labeled preference data from source tasks to infer labels for target tasks, eliminating the requirement for human queries.
Normalization Enhances Generalization in Visual Reinforcement Learning
Though normalization techniques have demonstrated huge success in supervised and unsupervised learning, their applications in visual RL are still scarce.
GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction
This paper aims to efficiently enable Large Language Models (LLMs) to use multimodal tools.
Off-Policy RL Algorithms Can be Sample-Efficient for Continuous Control via Sample Multiple Reuse
Empirical results show that SMR significantly boosts the sample efficiency of the base methods across most of the evaluated tasks without any hyperparameter tuning or additional tricks.
SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentation
To address this issue, we propose Semantic-assisted Object Cluster (SOC), which aggregates video content and textual guidance for unified temporal modeling and cross-modal alignment.
Ranked #2 on
Referring Expression Segmentation
on A2D Sentences
(using extra training data)
Weakly-Supervised Concealed Object Segmentation with SAM-based Pseudo Labeling and Multi-scale Feature Grouping
It remains a challenging task since (1) it is hard to distinguish concealed objects from the background due to the intrinsic similarity and (2) the sparsely-annotated training data only provide weak supervision for model learning.
Towards Realizing the Value of Labeled Target Samples: a Two-Stage Approach for Semi-Supervised Domain Adaptation
We further propose a consistency learning based mean teacher model to effectively adapt the learned UDA model using labeled and unlabeled target samples.
Semi-supervised Domain Adaptation
Unsupervised Domain Adaptation
Data-Efficient Image Quality Assessment with Attention-Panel Decoder
Blind Image Quality Assessment (BIQA) is a fundamental task in computer vision, which however remains unresolved due to the complex distortion conditions and diversified image contents.
Uncertainty-driven Trajectory Truncation for Data Augmentation in Offline Reinforcement Learning
To empirically show the advantages of TATU, we first combine it with two classical model-based offline RL algorithms, MOPO and COMBO.
Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videos
Generating text-editable and pose-controllable character videos have an imperious demand in creating various digital human.
Efficient Meshy Neural Fields for Animatable Human Avatars
Moreover, only minutes of optimization is enough for plausible reconstruction results.
BoxSnake: Polygonal Instance Segmentation with Box Supervision
Box-supervised instance segmentation has gained much attention as it requires only simple box annotations instead of costly mask or polygon annotations.
SSGD: A smartphone screen glass dataset for defect detection
Interactive devices with touch screen have become commonly used in various aspects of daily life, which raises the demand for high production quality of touch screen glass.
Compressed Interaction Graph based Framework for Multi-behavior Recommendation
However, it is challenging to explore multi-behavior data due to the unbalanced data distribution and sparse target behavior, which lead to the inadequate modeling of high-order relations when treating multi-behavior data ''as features'' and gradient conflict in multitask learning when treating multi-behavior data ''as labels''.
SemanticAC: Semantics-Assisted Framework for Audio Classification
In this paper, we propose SemanticAC, a semantics-assisted framework for Audio Classification to better leverage the semantic information.
Model-based Transfer Learning for Automatic Optical Inspection based on domain discrepancy
However, related research enhanced the network models by applying TL without considering the domain similarity among datasets, the data long-tailedness of a source dataset, and mainly used linear transformations to mitigate the lack of samples.
Adversarial Alignment for Source Free Object Detection
Source-free object detection (SFOD) aims to transfer a detector pre-trained on a label-rich source domain to an unlabeled target domain without seeing source data.
Emergent collective intelligence from massive-agent cooperation and competition
Since these learned group strategies arise from individual decisions without an explicit coordination mechanism, we claim that artificial collective intelligence emerges from massive-agent cooperation and competition.
Degradation-Resistant Unfolding Network for Heterogeneous Image Fusion
Heterogeneous image fusion (HIF) techniques aim to enhance image quality by merging complementary information from images captured by different sensors.
Camouflaged Object Detection With Feature Decomposition and Edge Reconstruction
COD is a challenging task due to the intrinsic similarity of camouflaged objects with the background, as well as their ambiguous boundaries.
FLAG3D: A 3D Fitness Activity Dataset with Language Instruction
With the continuously thriving popularity around the world, fitness activity analytic has become an emerging research topic in computer vision.
FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generation
To address these problems, we propose FineDance, which contains 14. 6 hours of music-dance paired data, with fine-grained hand motions, fine-grained genres (22 dance genres), and accurate posture.
SimVTP: Simple Video Text Pre-training with Masked Autoencoders
This paper presents SimVTP: a Simple Video-Text Pretraining framework via masked autoencoders.
Ranked #16 on
Moment Retrieval
on Charades-STA
Human-machine Interactive Tissue Prototype Learning for Label-efficient Histopathology Image Segmentation
Then, the encoder is used to map the images into the embedding space and generate pixel-level pseudo tissue masks by querying the tissue prototype dictionary.
Disentangling Past-Future Modeling in Sequential Recommendation via Dual Networks
Sequential recommendation (SR) plays an important role in personalized recommender systems because it captures dynamic and diverse preferences from users' real-time increasing behaviors.
State Advantage Weighting for Offline RL
We present state advantage weighting for offline reinforcement learning (RL).
Estimating Neural Reflectance Field from Radiance Field using Tree Structures
A high-quality NeRF decomposition relies on good geometry information extraction as well as good prior terms to properly resolve ambiguities between different components.
Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualization
A key challenge of continual reinforcement learning (CRL) in dynamic environments is to promptly adapt the RL agent's behavior as the environment changes over its lifetime, while minimizing the catastrophic forgetting of the learned information.
A Medical Semantic-Assisted Transformer for Radiographic Report Generation
Automated radiographic report generation is a challenging cross-domain task that aims to automatically generate accurate and semantic-coherence reports to describe medical images.
Neural Capture of Animatable 3D Human from Monocular Video
We present a novel paradigm of building an animatable 3D human representation from a monocular video input, such that it can be rendered in any unseen poses and views.
Coarse-to-Fine Knowledge-Enhanced Multi-Interest Learning Framework for Multi-Behavior Recommendation
More specifically, we propose a novel Coarse-to-fine Knowledge-enhanced Multi-interest Learning (CKML) framework to learn shared and behavior-specific interests for different behaviors.
Towards Better Dermoscopic Image Feature Representation Learning for Melanoma Classification
Deep learning-based melanoma classification with dermoscopic images has recently shown great potential in automatic early-stage melanoma diagnosis.
Double Check Your State Before Trusting It: Confidence-Aware Bidirectional Offline Model-Based Imagination
Model-based RL methods offer a richer dataset and benefit generalization by generating imaginary trajectories with either trained forward or reverse dynamics model.
Seeking Common Ground While Reserving Differences: Multiple Anatomy Collaborative Framework for Undersampled MRI Reconstruction
Four different implementations of anatomy-specific learners are presented and explored on the top of our framework in two MRI reconstruction networks.
Mildly Conservative Q-Learning for Offline Reinforcement Learning
The distribution shift between the learned policy and the behavior policy makes it necessary for the value function to stay conservative such that out-of-distribution (OOD) actions will not be severely overestimated.
OrdinalCLIP: Learning Rank Prompts for Language-Guided Ordinal Regression
In this paper, we propose to learn the rank concepts from the rich semantic CLIP latent space.
Ranked #1 on
Few-shot Age Estimation
on MORPH Album2
UniInst: Unique Representation for End-to-End Instance Segmentation
Existing instance segmentation methods have achieved impressive performance but still suffer from a common dilemma: redundant representations (e. g., multiple boxes, grids, and anchor points) are inferred for one instance, which leads to multiple duplicated predictions.
ScalableViT: Rethinking the Context-oriented Generalization of Vision Transformer
The vanilla self-attention mechanism inherently relies on pre-defined and steadfast computational dimensions.
Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RL
In this paper, we revisit the theoretical property of GCSL -- optimizing a lower bound of the goal reaching objective, and extend GCSL as a novel offline goal-conditioned RL algorithm.
Hybrid intelligence for dynamic job-shop scheduling with deep reinforcement learning and attention mechanism
The dynamic job-shop scheduling problem (DJSP) is a class of scheduling tasks that specifically consider the inherent uncertainties such as changing order requirements and possible machine breakdown in realistic smart manufacturing settings.
Value Activation for Bias Alleviation: Generalized-activated Deep Double Deterministic Policy Gradients
It is vital to accurately estimate the value function in Deep Reinforcement Learning (DRL) such that the agent could execute proper actions instead of suboptimal ones.
Implicit Feature Refinement for Instance Segmentation
Our IFR enjoys several advantages: 1) simulates an infinite-depth refinement network while only requiring parameters of single residual block; 2) produces high-level equilibrium instance features of global receptive field; 3) serves as a plug-and-play general module easily extended to most object recognition frameworks.
CLIP4Caption: CLIP for Video Caption
It is noted that our model is only trained on the MSR-VTT dataset.
Double-Uncertainty Guided Spatial and Temporal Consistency Regularization Weighting for Learning-based Abdominal Registration
Such convention has two limitations: (i) Besides the laborious grid search for the optimal fixed weight, the regularization strength of a specific image pair should be associated with the content of the images, thus the "one value fits all" training scheme is not ideal; (ii) Only spatially regularizing the transformation may neglect some informative clues related to the ill-posedness.
MHER: Model-based Hindsight Experience Replay
Replacing original goals with virtual goals generated from interaction with a trained dynamics model leads to a novel relabeling method, model-based relabeling (MBR).
A Self-Boosting Framework for Automated Radiographic Report Generation
On one hand, the image-text matching branch helps to learn highly text-correlated visual features for the report generation branch to output high quality reports.
Self-Supervised Video Hashing via Bidirectional Transformers
Most existing unsupervised video hashing methods are built on unidirectional models with less reliable training objectives, which underuse the correlations among frames and the similarity structure between videos.
A Coarse-to-Fine Instance Segmentation Network with Learning Boundary Representation
Experiments also show that our proposed method achieves competitive performance compared to existing boundary-based methods with a lightweight design and a simple pipeline.
Efficient Continuous Control with Double Actors and Regularized Critics
First, we uncover and demonstrate the bias alleviation property of double actors by building double actors upon single critic and double critics to handle overestimation bias in DDPG and underestimation bias in TD3 respectively.
Noisy Labels are Treasure: Mean-Teacher-Assisted Confident Learning for Hepatic Vessel Segmentation
Manually segmenting the hepatic vessels from Computer Tomography (CT) is far more expertise-demanding and laborious than other structures due to the low-contrast and complex morphology of vessels, resulting in the extreme lack of high-quality labeled data.
Reward function shape exploration in adversarial imitation learning: an empirical study
To ensure our results' reliability, we conduct the experiments on a series of Mujoco and Box2D continuous control tasks based on four different AILs.
Universal and Flexible Optical Aberration Correction Using Deep-Prior Based Deconvolution
High quality imaging usually requires bulky and expensive lenses to compensate geometric and chromatic aberrations.
Bias-reduced Multi-step Hindsight Experience Replay for Efficient Multi-goal Reinforcement Learning
Two main challenges in multi-goal reinforcement learning are sparse rewards and sample inefficiency.
Frequency-Aware Spatiotemporal Transformers for Video Inpainting Detection
In this paper, we propose a frequency-aware spatiotemporal transformers for deep In this paper, we propose a Frequency-Aware Spatiotemporal Transformer (FAST) for video inpainting detection, which aims to simultaneously mine the traces of video inpainting from spatial, temporal, and frequency domains.
Unsupervised Multimodal Image Registration with Adaptative Gradient Guidance
Multimodal image registration (MIR) is a fundamental procedure in many image-guided therapies.
Unimodal Cyclic Regularization for Training Multimodal Image Registration Networks
The loss function of an unsupervised multimodal image registration framework has two terms, i. e., a metric for similarity measure and regularization.
F3RNet: Full-Resolution Residual Registration Network for Deformable Image Registration
In this paper, we propose a novel unsupervised registration network, namely the Full-Resolution Residual Registration Network (F3RNet), for deformable registration of severely deformed organs.
Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration
Deformable image registration between Computed Tomography (CT) images and Magnetic Resonance (MR) imaging is essential for many image-guided therapies.
Disentangled Non-Local Neural Networks
This paper first studies the non-local block in depth, where we find that its attention computation can be split into two terms, a whitened pairwise term accounting for the relationship between two pixels and a unary term representing the saliency of every pixel.
Ranked #20 on
Semantic Segmentation
on Cityscapes test
(using extra training data)
Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration
The generative adversarial imitation learning (GAIL) has provided an adversarial learning framework for imitating expert policy from demonstrations in high-dimensional continuous tasks.
4D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Cameras
Our method enables a realtime online motion capture system running at 30fps using 5 cameras on a 5-person scene.
Ranked #8 on
3D Multi-Person Pose Estimation
on Shelf
Neural Architecture Search for Compressed Sensing Magnetic Resonance Image Reconstruction
Our proposed method can reach a better trade-off between computation cost and reconstruction performance for MR reconstruction problem with good generalizability and offer insights to design neural networks for other medical image applications.
PgNN: Physics-guided Neural Network for Fourier Ptychographic Microscopy
Since the optimal parameters of the PgNN can be derived by minimizing the difference between the model-generated images and real captured angle-varied images corresponding to the same scene, the proposed PgNN can get rid of the problem of massive training data as in traditional supervised methods.
On the Mathematical Understanding of ResNet with Feynman Path Integral
In this paper, we aim to understand Residual Network (ResNet) in a scientifically sound way by providing a bridge between ResNet and Feynman path integral.
Capture Dense: Markerless Motion Capture Meets Dense Pose Estimation
Specifically, we first introduce a novel markerless motion capture method that can take advantage of dense parsing capability provided by the dense pose detector.
Structure from Recurrent Motion: From Rigidity to Recurrency
This paper proposes a new method for Non-Rigid Structure-from-Motion (NRSfM) from a long monocular video sequence observing a non-rigid object performing recurrent and possibly repetitive dynamic action.
Joint Training of Cascaded CNN for Face Detection
Cascade has been widely used in face detection, where classifier with low computation cost can be firstly used to shrink most of the background while keeping the recall.
