Search Results for author:
Found 63 papers, 24 papers with code
Efficient Action Counting with Dynamic Queries
Temporal repetition counting aims to quantify the repeated action cycles within a video.
OpenMEDLab: An Open-source Platform for Multi-modality Foundation Models in Medicine
The emerging trend of advancing generalist artificial intelligence, such as GPTv4 and Gemini, has reshaped the landscape of research (academia and industry) in machine learning and many other research areas.
Language Models Represent Beliefs of Self and Others
Understanding and attributing mental states, known as Theory of Mind (ToM), emerges as a fundamental capability for human social reasoning.
Real-time Holistic Robot Pose Estimation with Unknown States
We propose an end-to-end pipeline for real-time, holistic robot pose estimation from a single RGB image, even in the absence of known robot states.
Ranked #1 on
Robot Pose Estimation
on DREAM-dataset
Efficient Selective Audio Masked Multimodal Bottleneck Transformer for Audio-Video Classification
To learn from multimodal videos effectively, in this work, we propose a novel audio-video recognition approach termed audio video Transformer, AVT, leveraging the effective spatio-temporal representation by the video Transformer to improve action recognition accuracy.
Efficient Multiscale Multimodal Bottleneck Transformer for Audio-Video Classification
In recent years, researchers combine both audio and video signals to deal with challenges where actions are not well represented or captured by visual cues.
TPC-ViT: Token Propagation Controller for Efficient Vision Transformer
Previous approaches that employ gradual token reduction to address this challenge assume that token redundancy in one layer implies redundancy in all the following layers.
Deformable Audio Transformer for Audio Event Detection
Hence, we introduce a learnable input adaptor to alleviate this issue, and DATAR achieves state-of-the-art performance.
ChimpACT: A Longitudinal Dataset for Understanding Chimpanzee Behaviors
ChimpACT is both comprehensive and challenging, consisting of 163 videos with a cumulative 160, 500 frames, each richly annotated with detection, identification, pose estimation, and fine-grained spatiotemporal behavior labels.
A Multi-Scale Spatial Transformer U-Net for Simultaneously Automatic Reorientation and Segmentation of 3D Nuclear Cardiac Images
Accurate reorientation and segmentation of the left ventricular (LV) is essential for the quantitative analysis of myocardial perfusion imaging (MPI), in which one critical step is to reorient the reconstructed transaxial nuclear cardiac images into standard short-axis slices for subsequent image processing.
UPL-SFDA: Uncertainty-aware Pseudo Label Guided Source-Free Domain Adaptation for Medical Image Segmentation
The different predictions in these duplicated heads are used to obtain pseudo labels for unlabeled target-domain images and their uncertainty to identify reliable pseudo labels.
BROW: Better featuRes fOr Whole slide image based on self-distillation
Hence, we proposed BROW, a foundation model for extracting better feature representations for WSIs, which can be conveniently adapted to downstream tasks without or with slight fine-tuning.
Classification of lung cancer subtypes on CT images with synthetic pathological priors
The accurate diagnosis on pathological subtypes for lung cancer is of significant importance for the follow-up treatments and prognosis managements.
Human Motion Generation: A Survey
In this survey, we present a comprehensive literature review of human motion generation, which, to the best of our knowledge, is the first of its kind in this field.
AutoShot: A Short Video Dataset and State-of-the-Art Shot Boundary Detection
For video content creation and understanding, the shot boundary detection (SBD) is one of the most essential components in various scenarios.
Ranked #1 on
Camera shot boundary detection
on ClipShots
Selective Structured State-Spaces for Long-Form Video Understanding
To address this limitation, we present a novel Selective S4 (i. e., S5) model that employs a lightweight mask generator to adaptively select informative image tokens resulting in more efficient and accurate modeling of long-term spatiotemporal dependencies in videos.
Ranked #2 on
Video Classification
on Breakfast
3D Human Mesh Estimation from Virtual Markers
The advanced motion capture systems solve the problem by placing dense physical markers on the body surface, which allows to extract realistic meshes from their non-rigid motions.
Ranked #1 on
3D Human Pose Estimation
on Surreal
Multiscale Audio Spectrogram Transformer for Efficient Audio Classification
Audio event has a hierarchical architecture in both time and frequency and can be grouped together to construct more abstract semantic audio classes.
Ranked #12 on
Audio Classification
on VGGSound
Dynamic Inference With Grounding Based Vision and Language Models
For example, recent image and language models with more than 200M parameters have been proposed to learn visual grounding in the pre-training step and show impressive results on downstream vision and language tasks.
GFPose: Learning 3D Human Pose Prior with Gradient Fields
During the denoising process, GFPose implicitly incorporates pose priors in gradients and unifies various discriminative and generative tasks in an elegant framework.
Intelligent Computing: The Latest Advances, Challenges and Future
In recent years, we have witnessed the emergence of intelligent computing, a new computing paradigm that is reshaping traditional computing and promoting digital revolution in the era of big data, artificial intelligence and internet-of-things with new computing theories, architectures, methods, systems, and applications.
MONAI: An open-source framework for deep learning in healthcare
For AI models to be used clinically, they need to be made safe, reproducible and robust, and the underlying software framework must be aware of the particularities (e. g. geometry, physiology, physics) of medical data being processed.
MotionBERT: A Unified Perspective on Learning Human Motion Representations
We present a unified perspective on tackling various human-centric video tasks by learning human motion representations from large-scale and heterogeneous data resources.
Ranked #1 on
Monocular 3D Human Pose Estimation
on Human3.6M
(using extra training data)
AVT: Audio-Video Transformer for Multimodal Action Recognition
AVT uses a combination of video and audio signals to improve action recognition accuracy, leveraging the effective spatio-temporal representation by the video Transformer.
Ranked #4 on
Multi-modal Classification
on VGG-Sound
Multiscale Multimodal Transformer for Multimodal Action Recognition
In this work, we develop a multiscale multimodal Transformer (MMT) that employs hierarchical representation learning.
Ranked #1 on
Multi-modal Classification
on VGG-Sound
Anti-Retroactive Interference for Lifelong Learning
Second, according to the similarity between incremental knowledge and base knowledge, we design an adaptive fusion of incremental knowledge, which helps the model allocate capacity to the knowledge of different difficulties.
CelebV-HQ: A Large-Scale Video Facial Attributes Dataset
Large-scale datasets have played indispensable roles in the recent success of face generation/editing and significantly facilitated the advances of emerging research fields.
Ranked #1 on
Unconditional Video Generation
on CelebV-HQ
Faster VoxelPose: Real-time 3D Human Pose Estimation by Orthographic Projection
While the voxel-based methods have achieved promising results for multi-person 3D pose estimation from multi-cameras, they suffer from heavy computation burdens, especially for large scenes.
Ranked #5 on
3D Multi-Person Pose Estimation
on Campus
Confidence Dimension for Deep Learning based on Hoeffding Inequality and Relative Evaluation
Research on the generalization ability of deep neural networks (DNNs) has recently attracted a great deal of attention.
Adversarial Contrastive Self-Supervised Learning
Recently, learning from vast unlabeled data, especially self-supervised learning, has been emerging and attracted widespread attention.
Associative Adversarial Learning Based on Selective Attack
We show that our method improves the recognition accuracy of adversarial training on ImageNet by 8. 32% compared with the baseline.
MoCaNet: Motion Retargeting in-the-wild via Canonicalization Networks
Trained with the canonicalization operations and the derived regularizations, our method learns to factorize a skeleton sequence into three independent semantic subspaces, i. e., motion, structure, and view angle.
Self-Supervised Monocular Depth and Ego-Motion Estimation in Endoscopy: Appearance Flow to the Rescue
Recently, self-supervised learning technology has been applied to calculate depth and ego-motion from monocular videos, achieving remarkable performance in autonomous driving scenarios.
Towards Comprehensive Monocular Depth Estimation: Multiple Heads Are Better Than One
In this work, we investigate into the phenomenon and propose to integrate the strengths of multiple weak depth predictor to build a comprehensive and accurate depth predictor, which is critical for many real-world applications, e. g., 3D reconstruction.
Joint Channel and Weight Pruning for Model Acceleration on Moblie Devices
In this paper, we present a unified framework with Joint Channel pruning and Weight pruning (JCW), and achieves a better Pareto-frontier between the latency and accuracy than previous model compression approaches.
SpeechNAS: Towards Better Trade-off between Latency and Accuracy for Large-Scale Speaker Verification
Recently, x-vector has been a successful and popular approach for speaker verification, which employs a time delay neural network (TDNN) and statistics pooling to extract speaker characterizing embedding from variable-length utterances.
Ranked #1 on
Speaker Verification
on VoxCeleb1
Shifted Chunk Transformer for Spatio-Temporal Representational Learning
However, the pure-Transformer based spatio-temporal learning can be prohibitively costly on memory and computation to extract fine-grained features from a tiny patch.
Federated Whole Prostate Segmentation in MRI with Personalized Neural Architectures
Building robust deep learning-based models requires diverse training data, ideally from several sources.
Test-Time Training for Deformable Multi-Scale Image Registration
Registration is a fundamental task in medical robotics and is often a crucial step for many downstream tasks such as motion analysis, intra-operative tracking and image segmentation.
Deformable Gabor Feature Networks for Biomedical Image Classification
In recent years, deep learning has dominated progress in the field of medical image analysis.
Federated Semi-Supervised Learning for COVID Region Segmentation in Chest CT using Multi-National Data from China, Italy, Japan
To facilitate CT analysis, recent efforts have focused on computer-aided characterization and diagnosis, which has shown promising results.
Cycle-Consistent Adversarial Autoencoders for Unsupervised Text Style Transfer
Unsupervised text style transfer is full of challenges due to the lack of parallel data and difficulties in content preservation.
Multi-Domain Image Completion for Random Missing Input Data
Multi-domain data are widely leveraged in vision applications taking advantage of complementary information from different modalities, e. g., brain tumor segmentation from multi-parametric magnetic resonance imaging (MRI).
Cardiac Segmentation on Late Gadolinium Enhancement MRI: A Benchmark Study from Multi-Sequence Cardiac MR Segmentation Challenge
In addition, the paired MS-CMR images could enable algorithms to combine the complementary information from the other sequences for the segmentation of LGE CMR.
LAMP: Large Deep Nets with Automated Model Parallelism for Image Segmentation
In this work, we introduce Large deep 3D ConvNets with Automated Model Parallelism (LAMP) and investigate the impact of both input's and deep 3D ConvNets' size on segmentation accuracy.
TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting
We present a lightweight video motion retargeting approach TransMoMo that is capable of transferring motion of a person in a source video realistically to another video of a target person.
NeurReg: Neural Registration and Its Application to Image Segmentation
Furthermore, we design three segmentation frameworks based on the proposed registration framework: 1) atlas-based segmentation, 2) joint learning of both segmentation and registration tasks, and 3) multi-task learning with atlas-based segmentation as an intermediate feature.
Cardiac Segmentation of LGE MRI with Noisy Labels
In the first step, we register a small set of five LGE cardiac magnetic resonance (CMR) images with ground truth labels to a set of 40 target LGE CMR images without annotation.
Privacy-preserving Federated Brain Tumour Segmentation
Due to medical data privacy regulations, it is often infeasible to collect and share patient data in a centralised data lake.
Neural Multi-Scale Self-Supervised Registration for Echocardiogram Dense Tracking
In this work, we propose a neural multi-scale self-supervised registration (NMSR) method for automated myocardial and cardiac blood flow dense tracking.
Deep Learning for Automated Medical Image Analysis
Second, we will demonstrate how to use the weakly labeled data for the mammogram breast cancer diagnosis by efficiently design deep learning for multi-instance learning.
AnatomyNet: Deep Learning for Fast and Fully Automated Whole-volume Segmentation of Head and Neck Anatomy
Methods: Our deep learning model, called AnatomyNet, segments OARs from head and neck CT images in an end-to-end fashion, receiving whole-volume HaN CT images as input and generating masks of all OARs of interest in one shot.
DeepEM: Deep 3D ConvNets With EM For Weakly Supervised Pulmonary Nodule Detection
Recently deep learning has been witnessing widespread adoption in various medical image applications.
DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification
DeepLung consists of two components, nodule detection (identifying the locations of candidate nodules) and classification (classifying candidate nodules into benign or malignant).
Ranked #5 on
Lung Nodule Classification
on LIDC-IDRI
Adversarial Deep Structured Nets for Mass Segmentation from Mammograms
Mass segmentation provides effective morphological features which are important for mass diagnosis.
DeepLung: 3D Deep Convolutional Nets for Automated Pulmonary Nodule Detection and Classification
Considering the 3D nature of lung CT data, two 3D networks are designed for the nodule detection and classification respectively.
Automated Pulmonary Nodule Detection And Classification
Classification
+1
Deep Multi-instance Networks with Sparse Label Assignment for Whole Mammogram Classification
Inspired by the success of using deep convolutional features for natural image analysis and multi-instance learning (MIL) for labeling a set of instances/patches, we propose end-to-end trained deep multi-instance networks for mass classification based on whole mammogram without the aforementioned ROIs.
Leak Event Identification in Water Systems Using High Order CRF
Today, detection of anomalous events in civil infrastructures (e. g. water pipe breaks and leaks) is time consuming and often takes hours or days.
Deep Multi-instance Networks with Sparse Label Assignment for Whole Mammogram Classification
Inspired by the success of using deep convolutional features for natural image analysis and multi-instance learning for labeling a set of instances/patches, we propose end-to-end trained deep multi-instance networks for mass classification based on whole mammogram without the aforementioned costly need to annotate the training data.
Adversarial Deep Structural Networks for Mammographic Mass Segmentation
Experimental results on two public datasets, INbreast and DDSM-BCRP, show that our end-to-end network combined with adversarial training achieves the-state-of-the-art results.
Co-occurrence Feature Learning for Skeleton based Action Recognition using Regularized Deep LSTM Networks
Skeleton based action recognition distinguishes human actions using the trajectories of skeleton joints, which provide a very good representation for describing actions.
Deep Trans-layer Unsupervised Networks for Representation Learning
Compared to traditional deep learning methods, the implemented feature learning method has much less parameters and is validated in several typical experiments, such as digit recognition on MNIST and MNIST variations, object recognition on Caltech 101 dataset and face verification on LFW dataset.
Constrained Extreme Learning Machines: A Study on Classification Cases
Extreme learning machine (ELM) is an extremely fast learning method and has a powerful performance for pattern recognition tasks proven by enormous researches and engineers.
