Search Results for author:
Found 269 papers, 121 papers with code
Physical formula enhanced multi-task learning for pharmacokinetics prediction
Overall, our work illustrates the benefits and potential of using PEMAL in AIDD and other scenarios with data scarcity and noise.
Taming Stable Diffusion for Text to 360° Panorama Image Generation
Generative models, e. g., Stable Diffusion, have enabled the creation of photorealistic images from text prompts.
How Much Data are Enough? Investigating Dataset Requirements for Patch-Based Brain MRI Segmentation Tasks
This paper focuses on an early stage phase of deep learning research, prior to model development, and proposes a strategic framework for estimating the amount of annotated data required to train patch-based segmentation networks.
RS-Mamba for Large Remote Sensing Image Dense Prediction
RSM is specifically designed to capture the global context of remote sensing images with linear complexity, facilitating the effective processing of large VHR images.
Ranked #1 on
Road Segmentation
on Massachusetts Roads Dataset
(F1 metric)
Building change detection for remote sensing images
Change Detection
+1
RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agents
The ultimate goals of robotic learning is to acquire a comprehensive and generalizable robotic system capable of performing both seen skills within the training distribution and unseen skills in novel environments.
GVGEN: Text-to-3D Generation with Volumetric Representation
To simplify the generation of GaussianVolume and empower the model to generate instances with detailed 3D geometry, we propose a coarse-to-fine pipeline.
DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLM
We present DetToolChain, a novel prompting paradigm, to unleash the zero-shot object detection ability of multimodal large language models (MLLMs), such as GPT-4V and Gemini.
HVDistill: Transferring Knowledge from Images to Point Clouds via Unsupervised Hybrid-View Distillation
We present a hybrid-view-based knowledge distillation framework, termed HVDistill, to guide the feature learning of a point cloud neural network with a pre-trained image network in an unsupervised man- ner.
Agent3D-Zero: An Agent for Zero-shot 3D Understanding
The ability to understand and reason the 3D real world is a crucial milestone towards artificial general intelligence.
PoIFusion: Multi-Modal 3D Object Detection via Fusion at Points of Interest
In this work, we present PoIFusion, a simple yet effective multi-modal 3D object detection framework to fuse the information of RGB images and LiDAR point clouds at the point of interest (abbreviated as PoI).
LOCR: Location-Guided Transformer for Optical Character Recognition
Academic documents are packed with texts, equations, tables, and figures, requiring comprehensive understanding for accurate Optical Character Recognition (OCR).
Auxiliary Tasks Enhanced Dual-affinity Learning for Weakly Supervised Semantic Segmentation
We propose AuxSegNet+, a weakly supervised auxiliary learning framework to explore the rich information from these saliency maps and the significant inter-task correlation between saliency detection and semantic segmentation.
ConceptMath: A Bilingual Concept-wise Benchmark for Measuring Mathematical Reasoning of Large Language Models
This paper introduces ConceptMath, a bilingual (English and Chinese), fine-grained benchmark that evaluates concept-wise mathematical reasoning of Large Language Models (LLMs).
MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues
By conducting a detailed analysis of real multi-turn dialogue data, we construct a three-tier hierarchical ability taxonomy comprising 4208 turns across 1388 multi-turn dialogues in 13 distinct tasks.
NeRF-Det++: Incorporating Semantic Cues and Perspective-aware Depth Supervision for Indoor Multi-View 3D Detection
We project the freely available 3D segmentation annotations onto the 2D plane and leverage the corresponding 2D semantic maps as the supervision signal, significantly enhancing the semantic awareness of multi-view detectors.
Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!
Large language models (LLMs) need to undergo safety alignment to ensure safe conversations with humans.
Self-consistent Validation for Machine Learning Electronic Structure
Machine learning has emerged as a significant approach to efficiently tackle electronic structure problems.
Revealing Decurve Flows for Generalized Graph Propagation
\textbf{Secondly}, we propose the {\em Continuous Unified Ricci Curvature} (\textbf{CURC}), an extension of celebrated {\em Ollivier-Ricci Curvature} for directed and weighted graphs.
ChemLLM: A Chemical Large Language Model
ChemLLM beats GPT-3. 5 on all three principal tasks in chemistry, i. e., name conversion, molecular caption, and reaction prediction, and surpasses GPT-4 on two of them.
CasCast: Skillful High-resolution Precipitation Nowcasting via Cascaded Modelling
Precipitation nowcasting based on radar data plays a crucial role in extreme weather prediction and has broad implications for disaster management.
Point Cloud Matters: Rethinking the Impact of Different Observation Spaces on Robot Learning
In this study, we explore the influence of different observation spaces on robot learning, focusing on three predominant modalities: RGB, RGB-D, and point cloud.
Integration of cognitive tasks into artificial general intelligence test for large models
During the evolution of large models, performance evaluation is necessarily performed to assess their capabilities and ensure safety before practical application.
A Comprehensive Survey on 3D Content Generation
Recent years have witnessed remarkable advances in artificial intelligence generated content(AIGC), with diverse input modalities, e. g., text, image, video, audio and 3D.
ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast
Data-driven weather forecast based on machine learning (ML) has experienced rapid development and demonstrated superior performance in the global medium-range forecast compared to traditional physics-based dynamical models.
FengWu-GHR: Learning the Kilometer-scale Medium-range Global Weather Forecasting
Therefore, building a kilometer-scale global forecast model is a persistent pursuit in the meteorology domain.
From GPT-4 to Gemini and Beyond: Assessing the Landscape of MLLMs on Generalizability, Trustworthiness and Causality through Four Modalities
Multi-modal Large Language Models (MLLMs) have shown impressive abilities in generating reasonable responses with respect to multi-modal contents.
Observation-Guided Meteorological Field Downscaling at Station Scale: A Benchmark and a New Method
In this paper, we extend meteorological downscaling to arbitrary scattered station scales, establish a brand new benchmark and dataset, and retrieve meteorological states at any given station location from a coarse-resolution meteorological field.
Bilateral Reference for High-Resolution Dichotomous Image Segmentation
It comprises two essential components: the localization module (LM) and the reconstruction module (RM) with our proposed bilateral reference (BiRef).
Ranked #1 on
RGB Salient Object Detection
on HRSOD
(using extra training data)
Camouflaged Object Segmentation
Dichotomous Image Segmentation
+3
Merging Vision Transformers from Different Tasks and Domains
This work targets to merge various Vision Transformers (ViTs) trained on different tasks (i. e., datasets with different object categories) or domains (i. e., datasets with the same categories but different environments) into one unified model, yielding still good performance on each task or domain.
Partial Fine-Tuning: A Successor to Full Fine-Tuning for Vision Transformers
We first validate eight manually-defined partial fine-tuning strategies across kinds of datasets and vision transformer architectures, and find that some partial fine-tuning strategies (e. g., ffn only or attention only) can achieve better performance with fewer tuned parameters than full fine-tuning, and selecting appropriate layers is critical to partial fine-tuning.
Efficient Architecture Search via Bi-level Data Pruning
Improving the efficiency of Neural Architecture Search (NAS) is a challenging but significant task that has received much attention.
Towards an end-to-end artificial intelligence driven global weather forecasting system
Initial states are typically generated by traditional data assimilation components, which are computational expensive and time-consuming.
ContraNovo: A Contrastive Learning Approach to Enhance De Novo Peptide Sequencing
De novo peptide sequencing from mass spectrometry (MS) data is a critical task in proteomics research.
ResoNet: Robust and Explainable ENSO Forecasts with Hybrid Convolution and Transformer Networks
Recent studies have shown that deep learning (DL) models can skillfully predict the El Ni\~no-Southern Oscillation (ENSO) forecasts over 1. 5 years ahead.
FengWu-4DVar: Coupling the Data-driven Weather Forecasting Model with 4D Variational Assimilation
Weather forecasting is a crucial yet highly challenging task.
Point Transformer V3: Simpler, Faster, Stronger
This paper is not motivated to seek innovation within the attention mechanism.
Ranked #1 on
Semantic Segmentation
on S3DIS
(using extra training data)
UniDream: Unifying Diffusion Priors for Relightable Text-to-3D Generation
Recent advancements in text-to-3D generation technology have significantly advanced the conversion of textual descriptions into imaginative well-geometrical and finely textured 3D objects.
A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning
In this paper, from a novel perspective, we systematically study the challenges that remain in O2O RL and identify that the reason behind the slow improvement of the performance and the instability of online finetuning lies in the inaccurate Q-value estimation inherited from offline pretraining.
Hulk: A Universal Knowledge Translator for Human-Centric Tasks
Human-centric perception tasks, e. g., pedestrian detection, skeleton-based action recognition, and pose estimation, have wide industrial applications, such as metaverse and sports analysis.
Ranked #1 on
Pedestrian Image Caption
on CUHK-PEDES
GPT4Vis: What Can GPT-4 Do for Zero-shot Visual Recognition?
Our study centers on the evaluation of GPT-4's linguistic and visual capabilities in zero-shot visual recognition tasks: Firstly, we explore the potential of its generated rich textual descriptions across various categories to enhance recognition performance without any training.
Point Cloud Pre-training with Diffusion Models
This generator aggregates the features extracted by the backbone and employs them as the condition to guide the point-to-point recovery from the noisy point cloud, thereby assisting the backbone in capturing both local and global geometric priors as well as the global point density distribution of the object.
Octavius: Mitigating Task Interference in MLLMs via LoRA-MoE
While this phenomenon has been overlooked in previous work, we propose a novel and extensible framework, called Octavius, for comprehensive studies and experimentation on multimodal learning with Multimodal Large Language Models (MLLMs).
GUPNet++: Geometry Uncertainty Propagation Network for Monocular 3D Object Detection
It models the uncertainty propagation relationship of the geometry projection during training, improving the stability and efficiency of the end-to-end model learning.
I$^2$MD: 3D Action Representation Learning with Inter- and Intra-modal Mutual Distillation
Different from existing distillation solutions that transfer the knowledge of a pre-trained and fixed teacher to the student, in CMD, the knowledge is continuously updated and bidirectionally distilled between modalities during pre-training.
MaskMA: Towards Zero-Shot Multi-Agent Decision Making with Mask-Based Collaborative Learning
Building a single generalist agent with strong zero-shot capability has recently sparked significant advancements.
PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm
In this paper, we introduce a novel universal 3D pre-training framework designed to facilitate the acquisition of efficient 3D representation, thereby establishing a pathway to 3D foundational models.
Ranked #1 on
3D Semantic Segmentation
on ScanNet++
(using extra training data)
UniPAD: A Universal Pre-training Paradigm for Autonomous Driving
In the context of autonomous driving, the significance of effective feature learning is widely acknowledged.
Rethinking the BERT-like Pretraining for DNA Sequences
To address this research gap, we first conducted a series of exploratory experiments and gained several insightful observations: 1) In the fine-tuning phase of downstream tasks, when using K-mer overlapping tokenization instead of K-mer non-overlapping tokenization, both overlapping and non-overlapping pretraining weights show consistent performance improvement. 2) During the pre-training process, using K-mer overlapping tokenization quickly produces clear K-mer embeddings and reduces the loss to a very low level, while using K-mer non-overlapping tokenization results in less distinct embeddings and continuously decreases the loss.
Towards Fair and Comprehensive Comparisons for Image-Based 3D Object Detection
In this work, we build a modular-designed codebase, formulate strong training recipes, design an error diagnosis toolbox, and discuss current methods for image-based 3D object detection.
Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization
A single language model (LM), despite aligning well with an average labeler through reinforcement learning from human feedback (RLHF), may not universally suit diverse human preferences.
Understanding Masked Autoencoders From a Local Contrastive Perspective
Masked AutoEncoder (MAE) has revolutionized the field of self-supervised learning with its simple yet effective masking and reconstruction strategies.
RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models
The advent of Large Language Models (LLMs) has paved the way for complex tasks such as role-playing, which enhances user interactions by enabling models to imitate various characters.
NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Space
Monocular 3D Semantic Scene Completion (SSC) has garnered significant attention in recent years due to its potential to predict complex semantics and geometry shapes from a single image, requiring no 3D inputs.
3D Semantic Scene Completion from a single 2D image
3D Semantic Scene Completion from a single RGB image
Improving Multiple Sclerosis Lesion Segmentation Across Clinical Sites: A Federated Learning Approach with Noise-Resilient Training
Our approach enables collaboration among multiple clinical sites without compromising data privacy under a federated learning paradigm that incorporates a noise-robust training strategy based on label correction.
DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior
We present DiffBIR, a general restoration pipeline that could handle different blind image restoration tasks in a unified framework.
Ranked #1 on
Blind Face Restoration
on LFW
Boosting Residual Networks with Group Knowledge
Specifically, we implicitly divide all subnets into hierarchical groups by subnet-in-subnet sampling, aggregate the knowledge of different subnets in each group during training, and exploit upper-level group knowledge to supervise lower-level subnet groups.
STEERER: Resolving Scale Variations for Counting and Localization via Selective Inheritance Learning
Scale variation is a deep-rooted problem in object counting, which has not been effectively addressed by existing scale-aware algorithms.
Masked Motion Predictors are Strong 3D Action Representation Learners
To be specific, the proposed MAMP takes as input the masked spatio-temporal skeleton sequence and predicts the corresponding temporal motion of the masked human joints.
Ranked #5 on
Skeleton Based Action Recognition
on NTU RGB+D 120
Experts Weights Averaging: A New General Training Scheme for Vision Transformers
As Vision Transformers (ViTs) are gradually surpassing CNNs in various visual tasks, one may question: if a training scheme specifically for ViTs exists that can also achieve performance improvement without increasing inference cost?
MCTformer+: Multi-Class Token Transformer for Weakly Supervised Semantic Segmentation
Building upon the observation that the attended regions of the one-class token in the standard vision transformer can contribute to a class-agnostic localization map, we explore the potential of the transformer model to capture class-specific attention for class-discriminative object localization by learning multiple class tokens.
Object Localization
Weakly supervised Semantic Segmentation
+1
Theoretically Guaranteed Policy Improvement Distilled from Model-Based Planning
Model-based reinforcement learning (RL) has demonstrated remarkable successes on a range of continuous control tasks due to its high sample efficiency.
A general Temperature-Guided Language model to engineer enhanced Stability and Activity in Proteins
Designing protein mutants with high stability and activity is a critical yet challenging task in protein engineering.
Meta-Transformer: A Unified Framework for Multimodal Learning
Multimodal learning aims to build models that can process and relate information from multiple modalities.
What Can Simple Arithmetic Operations Do for Temporal Modeling?
We conduct comprehensive ablation studies on the instantiation of ATMs and demonstrate that this module provides powerful temporal modeling capability at a low computational cost.
Ranked #4 on
Action Recognition
on Something-Something V1
UniG3D: A Unified 3D Object Generation Dataset
However, the quality and diversity of existing 3D object generation methods are constrained by the inadequacies of existing 3D object datasets, including issues related to text quality, the incompleteness of multi-modal data representation encompassing 2D rendered images and 3D assets, as well as the size of the dataset.
MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators
Generating realistic human motion from given action descriptions has experienced significant advancements because of the emerging requirement of digital humans.
Adaptive Hierarchical SpatioTemporal Network for Traffic Forecasting
In this work, we propose an Adaptive Hierarchical SpatioTemporal Network (AHSTN) to promote traffic forecasting by exploiting the spatial hierarchy and modeling multi-scale spatial correlations.
Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions
This paper strives to resolve this problem by proposing a new instruct-ReID task that requires the model to retrieve images according to the given image or language instructions.
LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmark
To the best of our knowledge, we present one of the very first open-source endeavors in the field, LAMM, encompassing a Language-Assisted Multi-Modal instruction tuning dataset, framework, and benchmark.
Bi-LRFusion: Bi-Directional LiDAR-Radar Fusion for 3D Dynamic Object Detection
LiDAR and Radar are two complementary sensing approaches in that LiDAR specializes in capturing an object's 3D shape while Radar provides longer detection ranges as well as velocity hints.
PVT-SSD: Single-Stage 3D Object Detector with Point-Voxel Transformer
The key to associating the two different representations is our introduced input-dependent Query Initialization module, which could efficiently generate reference points and content queries.
Clothes-Invariant Feature Learning by Causal Intervention for Clothes-Changing Person Re-identification
Clothes-invariant feature extraction is critical to the clothes-changing person re-identification (CC-ReID).
Stimulative Training++: Go Beyond The Performance Limits of Residual Networks
In this work, we aim to re-investigate the training process of residual networks from a novel social psychology perspective of loafing, and further propose a new training scheme as well as three improved strategies for boosting residual networks beyond their performance limits.
FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead
We present FengWu, an advanced data-driven global medium-range weather forecast system based on Artificial Intelligence (AI).
Automatically Predict Material Properties with Microscopic Image Example Polymer Compatibility
Computer image recognition with machine learning method can make up the defects of artificial judging, giving accurate and quantitative judgement.
HumanBench: Towards General Human-centric Perception with Projector Assisted Pretraining
Specifically, we propose a \textbf{HumanBench} based on existing datasets to comprehensively evaluate on the common ground the generalization abilities of different pretraining methods on 19 datasets from 6 diverse downstream tasks, including person ReID, pose estimation, human parsing, pedestrian attribute recognition, pedestrian detection, and crowd counting.
Ranked #1 on
Pedestrian Attribute Recognition
on PA-100K
(using extra training data)
UniHCP: A Unified Model for Human-Centric Perceptions
When adapted to a specific task, UniHCP achieves new SOTAs on a wide range of human-centric tasks, e. g., 69. 8 mIoU on CIHP for human parsing, 86. 18 mA on PA-100K for attribute prediction, 90. 3 mAP on Market1501 for ReID, and 85. 8 JI on CrowdHuman for pedestrian detection, performing better than specialized models tailored for each task.
Ranked #1 on
Pose Estimation
on MS-COCO
Multi-Scale Control Signal-Aware Transformer for Motion Synthesis without Phase
As past poses often contain useful auxiliary hints, in this paper, we propose a task-agnostic deep learning method, namely Multi-scale Control Signal-aware Transformer (MCS-T), with an attention based encoder-decoder architecture to discover the auxiliary information implicitly for synthesizing controllable motion without explicitly requiring auxiliary information such as phase.
Saliency Guided Contrastive Learning on Scene Images
Despite being feasible, recent works largely overlooked discovering the most discriminative regions for contrastive learning to object representations in scene images.
Learning from pseudo-labels: deep networks improve consistency in longitudinal brain volume estimation
An accurate and robust quantitative measurement of brain volume change is paramount for translational research and clinical applications.
Fast-BEV: A Fast and Strong Bird's-Eye View Perception Baseline
Our Fast-BEV consists of five parts, We novelly propose (1) a lightweight deployment-friendly view transformation which fast transfers 2D image feature to 3D voxel space, (2) an multi-scale image encoder which leverages multi-scale information for better performance, (3) an efficient BEV encoder which is particularly designed to speed up on-vehicle inference.
$β$-DARTS++: Bi-level Regularization for Proxy-robust Differentiable Architecture Search
To address the robustness problem, we first benchmark different NAS methods under a wide range of proxy data, proxy channels, proxy layers and proxy epochs, since the robustness of NAS under different kinds of proxies has not been explored before.
Open-Set Fine-Grained Retrieval via Prompting Vision-Language Evaluator
PLEor could leverage pre-trained CLIP model to infer the discrepancies encompassing both pre-defined and unknown subcategories, called category-specific discrepancies, and transfer them to the backbone network trained in the close-set scenarios.
Semi-Supervised Semantic Segmentation under Label Noise via Diverse Learning Groups
Each learning group consists of a teacher network, a student network and a novel filter module.
Crossing the Gap: Domain Generalization for Image Captioning
In this paper, we introduce a new setting called Domain Generalization for Image Captioning (DGIC), where the data from the target domain is unseen in the learning process.
Learning Multi-Modal Class-Specific Tokens for Weakly Supervised Dense Object Localization
Weakly supervised dense object localization (WSDOL) relies generally on Class Activation Mapping (CAM), which exploits the correlation between the class weights of the image classifier and the pixel-level features.
Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?
Most existing text-video retrieval methods focus on cross-modal matching between the visual content of videos and textual query sentences.
Ranked #7 on
Video Retrieval
on VATEX
Ponder: Point Cloud Pre-training via Neural Rendering
We propose a novel approach to self-supervised learning of point cloud representations by differentiable neural rendering.
Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Models
In this paper, we propose a novel framework called BIKE, which utilizes the cross-modal bridge to explore bidirectional knowledge: i) We introduce the Video Attribute Association mechanism, which leverages the Video-to-Text knowledge to generate textual auxiliary attributes for complementing video recognition.
Ranked #1 on
Zero-Shot Action Recognition
on ActivityNet
MM-3DScene: 3D Scene Understanding by Customizing Masked Modeling with Informative-Preserved Reconstruction and Self-Distilled Consistency
Besides, such scenes with progressive masking ratios can also serve to self-distill their intrinsic spatial consistency, requiring to learn the consistent representations from unmasked areas.
3D Point Cloud Pre-training with Knowledge Distillation from 2D Images
The recent success of pre-trained 2D vision models is mostly attributable to learning from large-scale datasets.
EPCL: Frozen CLIP Transformer is An Efficient Point Cloud Encoder
These token embeddings are concatenated with a task token and fed into the frozen CLIP transformer to learn point cloud representation.
GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Clouds
In contrast to previous 3D MAE frameworks, which either design a complex decoder to infer masked information from maintained regions or adopt sophisticated masking strategies, we instead propose a much simpler paradigm.
Reconstructing Hand-Held Objects from Monocular Video
The key idea is that the hand motion naturally provides multiple views of the object and the motion can be reliably estimated by a hand pose tracker.
ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependency
In the learning phase, each agent minimizes the TD error that is dependent on how the subsequent agents have reacted to their chosen action.
Ranked #1 on
SMAC
on SMAC 3s5z_vs_3s6z
3D-QueryIS: A Query-based Framework for 3D Instance Segmentation
Previous top-performing methods for 3D instance segmentation often maintain inter-task dependencies and the tendency towards a lack of robustness.
Boosting Semi-Supervised 3D Object Detection with Semi-Sampling
When training a teacher-student semi-supervised framework, we randomly select gt samples and pseudo samples to both labeled frames and unlabeled frames, making a strong data augmentation for them.
The Equalization Losses: Gradient-Driven Training for Long-tailed Object Recognition
Long-tail distribution is widely spread in real-world applications.
Stimulative Training of Residual Networks: A Social Psychology Perspective of Loafing
In this work, we aim to re-investigate the training process of residual networks from a novel social psychology perspective of loafing, and further propose a new training strategy to strengthen the performance of residual networks.
CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-training
To address this issue, we propose CLIP2Point, an image-depth pre-training method by contrastive learning to transfer CLIP to the 3D domain, and adapt it to point cloud classification.
Ranked #3 on
Training-free 3D Point Cloud Classification
on ScanObjectNN
(using extra training data)
Towards Frame Rate Agnostic Multi-Object Tracking
In this paper, we propose a Frame Rate Agnostic MOT framework with a Periodic training Scheme (FAPS) to tackle the FraMOT problem for the first time.
ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild
We propose a single-network approach, termed ZoomNet, to take into account the hierarchical structure of the full human body and solve the scale variation of different body parts.
Ranked #2 on
2D Human Pose Estimation
on COCO-WholeBody
An Empirical Study of Pseudo-Labeling for Image-based 3D Object Detection
We hope this work can provide insights for the image-based 3D detection community under a semi-supervised setting.
Fine-grained Retrieval Prompt Tuning
In this paper, we develop Fine-grained Retrieval Prompt Tuning (FRPT), which steers a frozen pre-trained model to perform the fine-grained retrieval task from the perspectives of sample prompting and feature adaptation.
3D Interacting Hand Pose Estimation by Hand De-occlusion and Removal
Unlike most previous works that directly predict the 3D poses of two interacting hands simultaneously, we propose to decompose the challenging interacting hand pose estimation task and estimate the pose of each hand separately.
NSNet: Non-saliency Suppression Sampler for Efficient Video Recognition
On the video level, a temporal attention module is learned under dual video-level supervisions on both the salient and the non-salient representations.
Ranked #4 on
Action Recognition
on ActivityNet
Pose for Everything: Towards Category-Agnostic Pose Estimation
In this paper, we introduce the task of Category-Agnostic Pose Estimation (CAPE), which aims to create a pose estimation model capable of detecting the pose of any class of object given only a few samples with keypoint definition.
Ranked #4 on
2D Pose Estimation
on MP-100
Fast-MoCo: Boost Momentum-based Contrastive Learning with Combinatorial Patches
Contrastive-based self-supervised learning methods achieved great success in recent years.
Action Recognition With Motion Diversification and Dynamic Selection
To address this issue, we introduce a Motion Diversification and Selection (MoDS) module to generate diversified spatio-temporal motion features and then select the suitable motion representation dynamically for categorizing the input video.
Ranked #18 on
Action Recognition
on Something-Something V1
Revisiting Classifier: Transferring Vision-Language Models for Video Recognition
In this study, we focus on transferring knowledge for video classification tasks.
Ranked #1 on
Action Recognition
on ActivityNet
TransVG++: End-to-End Visual Grounding with Language Conditioned Vision Transformer
For another, we devise Language Conditioned Vision Transformer that removes external fusion modules and reuses the uni-modal ViT for vision-language fusion at the intermediate layers.
Better Teacher Better Student: Dynamic Prior Knowledge for Knowledge Distillation
Besides, our DPK makes the performance of the student model positively correlated with that of the teacher model, which means that we can further boost the accuracy of students by applying larger teachers.
Domain Invariant Masked Autoencoders for Self-supervised Learning from Multi-domains
While recent self-supervised learning methods have achieved good performances with evaluation set on the same domain as the training set, they will have an undesirable performance decrease when tested on a different domain.
MS Lesion Segmentation: Revisiting Weighting Mechanisms for Federated Learning
In addition, the segmentation loss function in each client is also re-weighted according to the lesion volume for the data during training.
Not All Tokens Are Equal: Human-centric Visual Analysis via Token Clustering Transformer
Vision transformers have achieved great successes in many computer vision tasks.
Ranked #4 on
2D Human Pose Estimation
on COCO-WholeBody
Unsupervised Learning of Accurate Siamese Tracking
As unlimited self-supervision signals can be obtained by tracking a video along a cycle in time, we investigate evolving a Siamese tracker by tracking videos forward-backward.
SeCo: Separating Unknown Musical Visual Sounds with Consistency Guidance
Recent years have witnessed the success of deep learning on the visual sound separation task.
DR.VIC: Decomposition and Reasoning for Video Individual Counting
Instead of relying on the Multiple Object Tracking (MOT) techniques, we propose to solve the problem by decomposing all pedestrians into the initial pedestrians who existed in the first frame and the new pedestrians with separate identities in each following frame.
Backbone is All Your Need: A Simplified Architecture for Visual Object Tracking
Exploiting a general-purpose neural architecture to replace hand-wired designs or inductive biases has recently drawn extensive interest.
Multi-class Token Transformer for Weakly Supervised Semantic Segmentation
To this end, we propose a Multi-class Token Transformer, termed as MCTformer, which uses multiple class tokens to learn interactions between the class tokens and the patch tokens.
$β$-DARTS: Beta-Decay Regularization for Differentiable Architecture Search
Neural Architecture Search~(NAS) has attracted increasingly more attention in recent years because of its capability to design deep neural networks automatically.
3D Object Detection from Images for Autonomous Driving: A Survey
3D object detection from images, one of the fundamental and challenging problems in autonomous driving, has received increasing attention from both industry and academia in recent years.
Trajectory Forecasting from Detection with Uncertainty-Aware Motion Encoding
Trajectories obtained from object detection and tracking are inevitably noisy, which could cause serious forecasting errors to predictors built on ground truth trajectories.
MonoDistill: Learning Spatial Features for Monocular 3D Object Detection
Finally, this LiDAR Net can serve as the teacher to transfer the learned knowledge to the baseline model.
Pseudo-Labeled Auto-Curriculum Learning for Semi-Supervised Keypoint Localization
PL approaches apply pseudo-labels to unlabeled data, and then train the model with a combination of the labeled and pseudo-labeled data iteratively.
RePre: Improving Self-Supervised Vision Transformer with Reconstructive Pre-training
Recently, self-supervised vision transformers have attracted unprecedented attention for their impressive representation learning ability.
b-DARTS: Beta-Decay Regularization for Differentiable Architecture Search
Neural Architecture Search (NAS) has attracted increasingly more attention in recent years because of its capability to design deep neural network automatically.
Accelerating Neural Network Optimization Through an Automated Control Theory Lens
We conduct experiments both from a control theory lens through a phase locus verification and from a network training lens on several models, including CNNs, Transformers, MLPs, and on benchmark datasets.
Revisiting the Transferability of Supervised Pretraining: an MLP Perspective
The pretrain-finetune paradigm is a classical pipeline in visual learning.
A Continuous Mapping For Augmentation Design
Automated data augmentation (ADA) techniques have played an important role in boosting the performance of deep models.
Unsupervised Contrastive Learning with Simple Transformation for 3D Point Cloud Data
Though a number of point cloud learning methods have been proposed to handle unordered points, most of them are supervised and require labels for training.
Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm
Recently, large-scale Contrastive Language-Image Pre-training (CLIP) has attracted unprecedented attention for its impressive zero-shot recognition ability and excellent transferability to downstream tasks.
Deep Instance Segmentation with Automotive Radar Detection Points
Automotive radar provides reliable environmental perception in all-weather conditions with affordable cost, but it hardly supplies semantic and geometry information due to the sparsity of radar detection points.
Improving the Transferability of Supervised Pretraining with an MLP Projector
The pretrain-finetune paradigm is a classical pipeline in visual learning.
Graph-Based 3D Multi-Person Pose Estimation Using Multi-View Images
Following the top-down paradigm, we decompose the task into two stages, i. e. person localization and pose estimation.
Ranked #2 on
3D Multi-Person Pose Estimation
on Panoptic
(using extra training data)
Towards Balanced Learning for Instance Recognition
In this work, we carefully revisit the standard training practice of detectors, and find that the detection performance is often limited by the imbalance during the training process, which generally consists in three levels - sample level, feature level, and objective level.
BN-NAS: Neural Architecture Search with Batch Normalization
We present BN-NAS, neural architecture search with Batch Normalization (BN-NAS), to accelerate neural architecture search (NAS).
PSViT: Better Vision Transformer via Token Pooling and Attention Sharing
Then, a compact set of the possible combinations for different token pooling and attention sharing mechanisms are constructed.
Geometry Uncertainty Projection Network for Monocular 3D Object Detection
In this paper, we propose a Geometry Uncertainty Projection Network (GUP Net) to tackle the error amplification problem at both inference and training stages.
3D Object Detection From Monocular Images
Depth Estimation
+3
Learning Geometry-Guided Depth via Projective Modeling for Monocular 3D Object Detection
In this paper, we propose to learn geometry-guided depth estimation with projective modeling to advance monocular 3D object detection.
Ranked #10 on
Monocular 3D Object Detection
on KITTI Cars Moderate
Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentation
Motivated by the significant inter-task correlation, we propose a novel weakly supervised multi-task framework termed as AuxSegNet, to leverage saliency detection and multi-label image classification as auxiliary tasks to improve the primary task of semantic segmentation using only image-level ground-truth labels.
GLiT: Neural Architecture Search for Global and Local Image Transformer
We introduce the first Neural Architecture Search (NAS) method to find a better transformer architecture for image recognition.
Ranked #498 on
Image Classification
on ImageNet
Mutual CRF-GNN for Few-Shot Learning
In this MCGN, the labels and features of support data are used by the CRF for inferring GNN affinities in a principled and probabilistic way.
AutoSampling: Search for Effective Data Sampling Schedules
Data sampling acts as a pivotal role in training deep learning models.
ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search
Human pose estimation has achieved significant progress in recent years.
Ranked #23 on
Pose Estimation
on COCO test-dev
(using extra training data)
Layerwise Optimization by Gradient Decomposition for Continual Learning
The gradient for update should be close to the gradient of the new task, consistent with the gradients shared by all old tasks, and orthogonal to the space spanned by the gradients specific to the old tasks.
PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop
Regression-based methods have recently shown promising results in reconstructing human meshes from monocular images.
Ranked #5 on
3D Human Pose Estimation
on AGORA
(using extra training data)
3D human pose and shape estimation
3D Human Reconstruction
+2
Delving into Localization Errors for Monocular 3D Object Detection
Estimating 3D bounding boxes from monocular images is an essential component in autonomous driving, while accurate 3D object detection from this kind of data is very challenging.
3D Object Detection From Monocular Images
Autonomous Driving
+3
Gradient Regularized Contrastive Learning for Continual Domain Adaptation
To better understand this issue, we study the problem of continual domain adaptation, where the model is presented with a labelled source domain and a sequence of unlabelled target domains.
Real-Time Visual Object Tracking via Few-Shot Learning
Visual Object Tracking (VOT) can be seen as an extended task of Few-Shot Learning (FSL).
Higher Performance Visual Tracking with Dual-Modal Localization
Visual Object Tracking (VOT) has synchronous needs for both robustness and accuracy.
Probabilistic Graph Attention Network with Conditional Kernels for Pixel-Wise Prediction
In contrast to previous works directly considering multi-scale feature maps obtained from the inner layers of a primary CNN architecture, and simply fusing the features with weighted averaging or concatenation, we propose a probabilistic graph attention network structure based on a novel Attention-Gated Conditional Random Fields (AG-CRFs) model for learning and fusing multi-scale representations in a principled manner.
Aggregation With Feature Detection
Aggregating features from different depths of a network is widely adopted to improve the network capability.
Inception Convolution with Efficient Dilation Search
To develop a practical method for learning complex inception convolution based on the data, a simple but effective search algorithm, referred to as efficient dilation optimization (EDO), is developed.
DETR for Crowd Pedestrian Detection
Furthermore, the bipartite match of ED harms the training efficiency due to the large ground truth number in crowd scenes.
Direct Depth Learning Network for Stereo Matching
To make the Adaptive-Grained Depth Refinement stage robust to the coarse depth and adaptive to the depth range of the points, the Granularity Uncertainty is introduced to Adaptive-Grained Depth Refinement stage.
Full Matching on Low Resolution for Disparity Estimation
To this end, we first propose to decompose the full matching task into multiple stages of the cost aggregation module.
Temporal-Channel Transformer for 3D Lidar-Based Video Object Detection in Autonomous Driving
As a special design of this transformer, the information encoded in the encoder is different from that in the decoder, i. e. the encoder encodes temporal-channel information of multiple frames while the decoder decodes the spatial-channel information for the current frame in a voxel-wise manner.
Evolving Search Space for Neural Architecture Search
The automation of neural architecture design has been a coveted alternative to human experts.
Adaptive Gradient Method with Resilience and Momentum
Several variants of stochastic gradient descent (SGD) have been proposed to improve the learning effectiveness and efficiency when training deep neural networks, among which some recent influential attempts would like to adaptively control the parameter-wise learning rate (e. g., Adam and RMSProp).
Category-specific Semantic Coherency Learning for Fine-grained Image Recognition
Existing deep learning based weakly supervised fine-grained image recognition (WFGIR) methods usually pick out the discriminative regions from the high-level feature (HLF) maps directly.
Once Quantization-Aware Training: High Performance Extremely Low-bit Architecture Search
Therefore, we propose to combine Network Architecture Search methods with quantization to enjoy the merits of the two sides.
Improving Auto-Augment via Augmentation-Wise Weight Sharing
On CIFAR-10, we achieve a top-1 error rate of 1. 24%, which is currently the best performing single model without extra training data.
Once Quantized for All: Progressively Searching for Quantized Compact Models
Automatic search of Quantized Neural Networks (QNN) has attracted a lot of attention.
SAMOT: Switcher-Aware Multi-Object Tracking and Still Another MOT Measure
Besides, we propose a new MOT evaluation measure, Still Another IDF score (SAIDF), aiming to focus more on identity issues. This new measure may overcome some problems of the previous measures and provide a better insight for identity issues in MOT.
Improving Deep Video Compression by Resolution-adaptive Flow Coding
In this work, we propose a new framework called Resolution-adaptive Flow Coding (RaFC) to effectively compress the flow maps globally and locally, in which we use multi-resolution representations instead of single-resolution representations for both the input flow maps and the output motion features of the MV encoder.
Exploring the Hierarchy in Relation Labels for Scene Graph Generation
By assigning each relationship a single label, current approaches formulate the relationship detection as a classification problem.
BriNet: Towards Bridging the Intra-class and Inter-class Gaps in One-Shot Segmentation
(2) The object categories at the training and inference stages have no overlap, leaving the inter-class gap.
Rethinking Pseudo-LiDAR Representation
Based on this observation, we design an image based CNN detector named Patch-Net, which is more generalized and can be instantiated as pseudo-LiDAR based 3D detectors.
Whole-Body Human Pose Estimation in the Wild
This paper investigates the task of 2D human whole-body pose estimation, which aims to localize dense landmarks on the entire human body including face, hands, body, and feet.
Ranked #8 on
2D Human Pose Estimation
on COCO-WholeBody
Differentiable Hierarchical Graph Grouping for Multi-Person Pose Estimation
The modules of HGG can be trained end-to-end with the keypoint detection network and is able to supervise the grouping process in a hierarchical manner.
Ranked #3 on
Keypoint Detection
on OCHuman
3D Human Mesh Regression with Dense Correspondence
This paper proposes a model-free 3D human mesh estimation framework, named DecoMR, which explicitly establishes the dense correspondence between the mesh and the local image features in the UV space (i. e. a 2D space used for texture mapping of 3D mesh).
Ranked #1 on
3D Human Reconstruction
on Surreal
Scope Head for Accurate Localization in Object Detection
Existing anchor-based and anchor-free object detectors in multi-stage or one-stage pipelines have achieved very promising detection performance.
Cheaper Pre-training Lunch: An Efficient Paradigm for Object Detection
In this paper, we propose a general and efficient pre-training paradigm, Montage pre-training, for object detection.
Location-Aware Feature Selection Text Detection Network
However, they are behind in accuracy comparing with recent segmentation-based text detectors.
Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition
Spatial-temporal graphs have been widely used by skeleton-based action recognition algorithms to model human action dynamics.
Ranked #4 on
3D Action Recognition
on Assembly101
Content Adaptive and Error Propagation Aware Deep Video Compression
Therefore, the encoder is adaptive to different video contents and achieves better compression performance by reducing the domain gap between the training and testing datasets.
Channel Pruning Guided by Classification Loss and Feature Importance
To this end, we propose a new strategy to suppress the influence of unimportant features (i. e., the features will be removed at the next pruning stage).
Equalization Loss for Long-Tailed Object Recognition
Based on it, we propose a simple but effective loss, named equalization loss, to tackle the problem of long-tailed rare categories by simply ignoring those gradients for rare categories.
Ranked #17 on
Long-tail Learning
on CIFAR-10-LT (ρ=10)
EcoNAS: Finding Proxies for Economical Neural Architecture Search
While many methods have been proposed to improve the efficiency of NAS, the search progress is still laborious because training and evaluating plausible architectures over large search space is time-consuming.
Learning 3D Human Shape and Pose from Dense Body Parts
Reconstructing 3D human shape and pose from monocular images is challenging despite the promising results achieved by the most recent learning-based methods.
Ranked #76 on
3D Human Pose Estimation
on 3DPW
(MPJPE metric)
3D human pose and shape estimation
3D Human Reconstruction
+3
Computation Reallocation for Object Detection
However, classification allocation pattern is usually adopted directly to object detector, which is proved to be sub-optimal.
A Shape Transformation-based Dataset Augmentation Framework for Pedestrian Detection
Alternatively, to access much more natural-looking pedestrians, we propose to augment pedestrian detection datasets by transforming real pedestrians from the same dataset into different shapes.
TRB: A Novel Triplet Representation for Understanding 2D Human Body
In this paper, we propose the Triplet Representation for Body (TRB) -- a compact 2D human body representation, with skeleton keypoints capturing human pose information and contour keypoints containing human shape information.
Improving One-shot NAS by Suppressing the Posterior Fading
In this paper, we analyse existing weight sharing one-shot NAS approaches from a Bayesian point of view and identify the posterior fading problem, which compromises the effectiveness of shared weights.
IntersectGAN: Learning Domain Intersection for Generating Images with Multiple Attributes
For example, given two image domains $X_1$ and $X_2$ with certain attributes, the intersection $X_1 \cap X_2$ denotes a new domain where images possess the attributes from both $X_1$ and $X_2$ domains.
GradNet: Gradient-Guided Network for Visual Object Tracking
In this work, we propose a novel gradient-guided network to exploit the discriminative information in gradients and update the template in the siamese network through feed-forward and backward operations.
Ranked #3 on
Visual Object Tracking
on OTB-2015
(Precision metric)
Structured Modeling of Joint Deep Feature and Prediction Refinement for Salient Object Detection
We formulate the CRF graphical model that involves message-passing of feature-feature, feature-prediction, and prediction-prediction, from the coarse scale to the finer scale, to update the features and the corresponding predictions.
Crowd Counting with Deep Structured Scale Integration Network
Automatic estimation of the number of people in unconstrained crowded scenes is a challenging task and one major difficulty stems from the huge scale variation of people.
Improving Description-based Person Re-identification by Multi-granularity Image-text Alignments
Firstly, the global-global alignment in the Global Contrast (GC) module is for matching the global contexts of images and descriptions.
Ranked #19 on
Text based Person Retrieval
on CUHK-PEDES
MMDetection: Open MMLab Detection Toolbox and Benchmark
In this paper, we introduce the various features of this toolbox.
Improving Action Localization by Progressive Cross-stream Cooperation
Specifically, we first generate a larger set of region proposals by combining the latest region proposals from both streams, from which we can readily obtain a larger set of labelled training samples to help learn better action detection models.
Online Hyper-parameter Learning for Auto-Augmentation Strategy
Data augmentation is critical to the success of modern deep learning techniques.
AM-LFS: AutoML for Loss Function Search
The key contribution of this work is the design of search space which can guarantee the generalization and transferability on different vision tasks by including a bunch of existing prevailing loss functions in a unified formulation.
Contextualized Spatial-Temporal Network for Taxi Origin-Destination Demand Prediction
Finally, a GCC module is applied to model the correlation between all regions by computing a global correlation feature as a weighted sum of all regional features, with the weights being calculated as the similarity between the corresponding region pairs.
PRUNING WITH HINTS: AN EFFICIENT FRAMEWORK FOR MODEL ACCELERATION
Hints can improve the performance of student model by transferring knowledge from teacher model.
Box-driven Class-wise Region Masking and Filling Rate Guided Loss for Weakly Supervised Semantic Segmentation
To address this problem, it is a good choice to learn to segment with weak supervision from bounding boxes.
Weakly-supervised Learning
Weakly supervised Semantic Segmentation
+1
Libra R-CNN: Towards Balanced Learning for Object Detection
In this work, we carefully revisit the standard training practice of detectors, and find that the detection performance is often limited by the imbalance during the training process, which generally consists in three levels - sample level, feature level, and objective level.
Ranked #149 on
Object Detection
on COCO test-dev
Feature Intertwiner for Object Detection
We argue that the reliable set could guide the feature learning of the less reliable set during training - in spirit of student mimicking teacher behavior and thus pushing towards a more compact class centroid in the feature space.
Ranked #134 on
Object Detection
on COCO test-dev
