Search Results for author:
Found 244 papers, 127 papers with code
Learning a Category-level Object Pose Estimator without Pose Annotations
Instead of using manually annotated images, we leverage diffusion models (e. g., Zero-1-to-3) to generate a set of images under controlled pose differences and propose to learn our object pose estimator with those images.
ViTamin: Designing Scalable Vision Models in the Vision-Language Era
To this end, we introduce ViTamin, a new vision models tailored for VLMs.
Exploiting Structural Consistency of Chest Anatomy for Unsupervised Anomaly Detection in Radiography Images
To this end, we propose a Simple Space-Aware Memory Matrix for In-painting and Detecting anomalies from radiography images (abbreviated as SimSID).
A Bayesian Approach to OOD Robustness in Image Classification
This enables us to learn a transitional dictionary of vMF kernels that are intermediate between the source and target domains and train the generative model on this dictionary using the annotations on the source domain, followed by iterative refinement.
From Pixel to Cancer: Cellular Automata in Computed Tomography
AI for cancer detection encounters the bottleneck of data scarcity, annotation difficulty, and low prevalence of early tumors.
Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis
X-ray is widely applied for transmission imaging due to its stronger penetration than natural light.
Towards Generalizable Tumor Synthesis
Tumor synthesis enables the creation of artificial tumors in medical images, facilitating the training of AI models for tumor detection and segmentation.
Leveraging AI Predicted and Expert Revised Annotations in Interactive Segmentation: Continual Tuning or Full Training?
Human experts revise the annotations predicted by AI, and in turn, AI improves its predictions by learning from these revised annotations.
PaLM2-VAdapter: Progressively Aligned Language Model Makes a Strong Vision-language Adapter
Our research undertakes a thorough exploration of the state-of-the-art perceiver resampler architecture and builds a strong baseline.
Source-Free and Image-Only Unsupervised Domain Adaptation for Category Level Object Pose Estimation
We focus on individual locally robust mesh vertex features and iteratively update them based on their proximity to corresponding features in the target domain even when the global pose is not correct.
Ranked #1 on
Unsupervised Domain Adaptation
on OOD-CV
SPFormer: Enhancing Vision Transformer with Superpixel Representation
In this work, we introduce SPFormer, a novel Vision Transformer enhanced by superpixel representation.
HISR: Hybrid Implicit Surface Representation for Photorealistic 3D Human Reconstruction
This representation is composed of two surface layers that represent opaque and translucent regions on the clothed human body.
A Semantic Space is Worth 256 Language Descriptions: Make Stronger Segmentation Models with Descriptive Properties
Instead of relying solely on category-specific annotations, ProLab uses descriptive properties grounded in common sense knowledge for supervising segmentation models.
Continual Adversarial Defense
In response to the rapidly evolving nature of adversarial attacks against visual classifiers on a monthly basis, numerous defenses have been proposed to generalize against as many known attacks as possible.
Causal-CoG: A Causal-Effect Look at Context Generation for Boosting Multi-modal Language Models
We propose Causal Context Generation, Causal-CoG, which is a prompting strategy that engages contextual information to enhance precise VQA during inference.
Rejuvenating image-GPT as Strong Visual Representation Learners
This paper enhances image-GPT (iGPT), one of the pioneering works that introduce autoregressive pretraining to predict next pixels for visual representation learning.
SCLIP: Rethinking Self-Attention for Dense Vision-Language Inference
Specifically, we replace the traditional self-attention block of CLIP vision encoder's last layer by our CSA module and reuse its pretrained projection matrices of query, key, and value, leading to a training-free adaptation approach for CLIP's zero-shot semantic segmentation.
Sequential Modeling Enables Scalable Learning for Large Vision Models
We introduce a novel sequential modeling approach which enables learning a Large Vision Model (LVM) without making use of any linguistic data.
Prompt-Based Exemplar Super-Compression and Regeneration for Class-Incremental Learning
Replay-based methods in class-incremental learning (CIL) have attained remarkable success, as replaying the exemplars of old classes can significantly mitigate catastrophic forgetting.
MaXTron: Mask Transformer with Trajectory Attention for Video Panoptic Segmentation
To alleviate the issue, we propose to adapt the trajectory attention for both the dense pixel features and object queries, aiming to improve the short-term and long-term tracking results, respectively.
Ranked #1 on
Video Panoptic Segmentation
on VIPSeg
Learning Part Segmentation from Synthetic Animals
We then benchmark Syn-to-Real animal part segmentation from SAP to PartImageNet, namely SynRealPart, with existing semantic segmentation domain adaptation methods and further improve them as our second contribution.
Instruct2Attack: Language-Guided Semantic Adversarial Attacks
We propose Instruct2Attack (I2A), a language-guided semantic attack that generates semantically meaningful perturbations according to free-form language instructions.
IG Captioner: Information Gain Captioners are Strong Zero-shot Classifiers
To tackle this problem, we redesign the scoring objective for the captioner to alleviate the distributional bias and focus on measuring the gain of information brought by the visual inputs.
Structure-Aware Sparse-View X-ray 3D Reconstruction
In this paper, we propose a framework, Structure-Aware X-ray Neural Radiodensity Fields (SAX-NeRF), for sparse-view X-ray 3D reconstruction.
Ranked #1 on
Low-Dose X-Ray Ct Reconstruction
on X3D
De-Diffusion Makes Text a Strong Cross-Modal Interface
We demonstrate text as a strong cross-modal interface.
3D-Aware Visual Question Answering about Parts, Poses and Occlusions
In this work, we introduce the task of 3D-aware VQA, which focuses on challenging questions that require a compositional reasoning over the 3D structure of visual scenes.
Synthetic Data as Validation
Specifically, the DSC score for liver tumor segmentation improves from 26. 7% (95% CI: 22. 6%-30. 9%) to 34. 5% (30. 8%-38. 2%) when evaluated on an in-domain dataset and from 31. 1% (26. 0%-36. 2%) to 35. 4% (32. 1%-38. 7%) on an out-domain dataset.
Acquiring Weak Annotations for Tumor Localization in Temporal and Volumetric Data
In summary, this research proposes an efficient annotation strategy for tumor detection and localization that is less accurate than per-pixel annotations but useful for creating large-scale datasets for screening tumors in various medical modalities.
3D TransUNet: Advancing Medical Image Segmentation through Vision Transformers
In this paper, we extend the 2D TransUNet architecture to a 3D network by building upon the state-of-the-art nnU-Net architecture, and fully exploring Transformers' potential in both the encoder and decoder design.
FedConv: Enhancing Convolutional Neural Networks for Handling Data Heterogeneity in Federated Learning
Federated learning (FL) is an emerging paradigm in machine learning, where a shared model is collaboratively learned using data from multiple devices to mitigate the risk of data leakage.
Understanding Pan-Sharpening via Generalized Inverse
By the generalized inverse theory, we derived two forms of general inverse matrix formulations that can correspond to the two prominent classes of Pan-sharpening methods, that is, component substitution and multi-resolution analysis methods.
Boosting Dermatoscopic Lesion Segmentation via Diffusion Models with Visual and Textual Prompts
Image synthesis approaches, e. g., generative adversarial networks, have been popular as a form of data augmentation in medical image analysis tasks.
Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shape
Animal3D consists of 3379 images collected from 40 mammal species, high-quality annotations of 26 keypoints, and importantly the pose and shape parameters of the SMAL model.
Ranked #1 on
Animal Pose Estimation
on Animal3D
3D-Aware Neural Body Fitting for Occlusion Robust 3D Human Pose Estimation
Motivated by the recent success of generative models in rigid object pose estimation, we propose 3D-aware Neural Body Fitting (3DNBF) - an approximate analysis-by-synthesis approach to 3D human pose estimation with SOTA performance and occlusion robustness.
Early Detection and Localization of Pancreatic Cancer by Label-Free Tumor Synthesis
We further investigate the per-voxel segmentation performance of pancreatic tumors if AI is trained on a combination of CT scans with synthetic tumors and CT scans with annotated large tumors at an advanced stage.
SwinMM: Masked Multi-view with Swin Transformers for 3D Medical Image Segmentation
To address this limitation, we present Masked Multi-view with Swin Transformers (SwinMM), a novel multi-view pipeline for enabling accurate and data-efficient self-supervised medical image analysis.
Generating Images with 3D Annotations Using Diffusion Models
With explicit 3D geometry control, we can easily change the 3D structures of the objects in the generated images and obtain ground-truth 3D annotations automatically.
Compositor: Bottom-up Clustering and Compositing for Robust Part and Object Segmentation
Compositor achieves state-of-the-art performance on PartImageNet and Pascal-Part by outperforming previous methods by around 0. 9% and 1. 3% on PartImageNet, 0. 4% and 1. 7% on Pascal-Part in terms of part and object mIoU and demonstrates better robustness against occlusion by around 4. 4% and 7. 1% on part and object respectively.
Discovering Failure Modes of Text-guided Diffusion Models via Adversarial Search
(2) We find regions in the latent space that lead to distorted images independent of the text prompt, suggesting that parts of the latent space are not well-structured.
Continual Learning for Abdominal Multi-Organ and Tumor Segmentation
The ability to dynamically extend a model to new data and classes is critical for multiple organ and tumor segmentation.
Neural Textured Deformable Meshes for Robust Analysis-by-Synthesis
Human vision demonstrates higher robustness than current AI algorithms under out-of-distribution scenarios.
Robust Category-Level 3D Pose Estimation from Synthetic Data
In this work, we aim to narrow the performance gap between models trained on synthetic data and few real images and fully supervised models trained on large-scale data.
Robust 3D-aware Object Classification via Discriminative Render-and-Compare
Using differentiable rendering, we estimate the 3D object pose by minimizing the reconstruction error between the mesh and the feature representation of the target image.
AbdomenAtlas-8K: Annotating 8,000 CT Volumes for Multi-Organ Segmentation in Three Weeks
Annotating medical images, particularly for organ segmentation, is laborious and time-consuming.
OOD-CV-v2: An extended Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images
Enhancing the robustness of vision algorithms in real-world scenarios is challenging.
Diffusion Models as Masked Autoencoders
There has been a longstanding belief that generation can facilitate a true understanding of visual data.
Label-Free Liver Tumor Segmentation
We demonstrate that AI models can accurately segment liver tumors without the need for manual annotation by using synthetic tumors in CT scans.
Ranked #1 on
Tumor Segmentation
on LiTS17
InstMove: Instance Motion for Object-centric Video Segmentation
A common solution is to use optical flow to provide motion information, but essentially it only considers pixel-level motion, which still relies on appearance similarity and hence is often inaccurate under occlusion and fast movement.
PoseExaminer: Automated Testing of Out-of-Distribution Robustness in Human Pose and Shape Estimation
We introduce a learning-based testing method, termed PoseExaminer, that automatically diagnoses HPS algorithms by searching over the parameter space of human pose images to find the failure modes.
CancerUniT: Towards a Single Unified Model for Effective Detection, Segmentation, and Diagnosis of Eight Major Cancers Using a Large Collection of CT Scans
Human readers or radiologists routinely perform full-body multi-organ multi-disease detection and diagnosis in clinical practice, while most medical AI systems are built to focus on single organs with a narrow list of a few diseases.
Benchmarking Robustness in Neural Radiance Fields
We find that NeRF-based models are significantly degraded in the presence of corruption, and are more sensitive to a different set of corruptions than image recognition models.
Learning Road Scene-level Representations via Semantic Region Prediction
In this work, we tackle two vital tasks in automated driving systems, i. e., driver intent prediction and risk object identification from egocentric images.
CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection
The proposed model is developed from an assembly of 14 datasets, using a total of 3, 410 CT scans for training and then evaluated on 6, 162 external CT scans from 3 additional datasets.
Ranked #1 on
Organ Segmentation
on BTCV
Unleashing the Power of Visual Prompting At the Pixel Level
This paper presents a simple and effective visual prompting method for adapting pre-trained models to downstream recognition tasks.
AsyInst: Asymmetric Affinity with DepthGrad and Color for Box-Supervised Instance Segmentation
Through systematic analysis, we found that the commonly used pairwise affinity loss has two limitations: (1) it works with color affinity but leads to inferior performance with other modalities such as depth gradient, (2)the original affinity loss does not prevent trivial predictions as intended but actually accelerates this process due to the affinity loss term being symmetric.
Localization vs. Semantics: Visual Representations in Unimodal and Multimodal Models
Despite the impressive advancements achieved through vision-and-language pretraining, it remains unclear whether this joint learning paradigm can help understand each individual modality.
Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoning
Visual Question Answering (VQA) models often perform poorly on out-of-distribution data and struggle on domain generalization.
LUMix: Improving Mixup by Better Modelling Label Uncertainty
LUMix is simple as it can be implemented in just a few lines of code and can be universally applied to any deep networks \eg CNNs and Vision Transformers, with minimal computational cost.
SMAUG: Sparse Masked Autoencoder for Efficient Video-Language Pre-training
Coupling all these designs allows our method to enjoy both competitive performances on text-to-video retrieval and video question answering tasks, and much less pre-training costs by 1. 9X or more.
Synthetic Tumors Make AI Segment Tumors Better
We develop a novel strategy to generate synthetic tumors.
Delving into Masked Autoencoders for Multi-Label Thorax Disease Classification
We hope that this study can direct future research on the application of Transformers to a larger variety of medical imaging tasks.
1st Place Solution of The Robust Vision Challenge 2022 Semantic Segmentation Track
The model is trained on a composite dataset consisting of images from 9 datasets (ADE20K, Cityscapes, Mapillary Vistas, ScanNet, VIPER, WildDash 2, IDD, BDD, and COCO) with a simple dataset balancing strategy.
Context-Enhanced Stereo Transformer
We construct our stereo depth estimation model, Context Enhanced Stereo Transformer (CSTR), by plugging CEP into the state-of-the-art stereo depth estimation method Stereo Transformer.
MOAT: Alternating Mobile Convolution and Attention Brings Strong Vision Models
The tiny-MOAT family is also benchmarked on downstream tasks, serving as a baseline for the community.
Ranked #1 on
Object Detection
on MS COCO
Robust Category-Level 6D Pose Estimation with Coarse-to-Fine Rendering of Neural Features
We consider the problem of category-level 6D pose estimation from a single RGB image.
Masked Autoencoders Enable Efficient Knowledge Distillers
For example, by distilling the knowledge from an MAE pre-trained ViT-L into a ViT-B, our method achieves 84. 0% ImageNet top-1 accuracy, outperforming the baseline of directly distilling a fine-tuned ViT-L by 1. 2%.
Explicit Occlusion Reasoning for Multi-person 3D Human Pose Estimation
Inspired by the remarkable ability of humans to infer occluded joints from visible cues, we develop a method to explicitly model this process that significantly improves bottom-up multi-person human pose estimation with or without occlusions.
Ranked #10 on
3D Multi-Person Pose Estimation (absolute)
on MuPoTS-3D
3D Human Pose Estimation
3D Multi-Person Pose Estimation (absolute)
+2
In Defense of Online Models for Video Instance Segmentation
In recent years, video instance segmentation (VIS) has been largely advanced by offline models, while online models gradually attracted less attention possibly due to their inferior performance.
Ranked #9 on
Video Instance Segmentation
on YouTube-VIS validation
(using extra training data)
kMaX-DeepLab: k-means Mask Transformer
However, we observe that most existing transformer-based vision models simply borrow the idea from NLP, neglecting the crucial difference between languages and images, particularly the extremely large sequence length of spatially flattened pixel features.
Ranked #2 on
Panoptic Segmentation
on COCO test-dev
Unsupervised Domain Adaptation through Shape Modeling for Medical Image Segmentation
Previous methods proposed Variational Autoencoder (VAE) based models to learn the distribution of shape for a particular organ and used it to automatically evaluate the quality of a segmentation prediction by fitting it into the learned shape distribution.
CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentation
We propose Clustering Mask Transformer (CMT-DeepLab), a transformer-based framework for panoptic segmentation designed around clustering.
Ranked #6 on
Panoptic Segmentation
on COCO test-dev
A Simple Data Mixing Prior for Improving Self-Supervised Learning
More notably, our SDMP is the first method that successfully leverages data mixing to improve (rather than hurt) the performance of Vision Transformers in the self-supervised setting.
VoGE: A Differentiable Volume Renderer using Gaussian Ellipsoids for Analysis-by-Synthesis
The Gaussian reconstruction kernels have been proposed by Westover (1990) and studied by the computer graphics community back in the 90s, which gives an alternative representation of object 3D geometry from meshes and point clouds.
In Defense of Image Pre-Training for Spatiotemporal Recognition
Inspired by this observation, we hypothesize that the key to effectively leveraging image pre-training lies in the decomposition of learning spatial and temporal features, and revisiting image pre-training as the appearance prior to initializing 3D kernels.
Fast AdvProp
Specifically, our modifications in Fast AdvProp are guided by the hypothesis that disentangled learning with adversarial examples is the key for performance improvements, while other training recipes (e. g., paired clean and adversarial training samples, multi-step adversarial attackers) could be largely simplified.
SwapMix: Diagnosing and Regularizing the Over-Reliance on Visual Context in Visual Question Answering
By swapping the context object features, the model reliance on context can be suppressed effectively.
CP2: Copy-Paste Contrastive Pretraining for Semantic Segmentation
Recent advances in self-supervised contrastive learning yield good image-level representation, which favors classification tasks but usually neglects pixel-level detailed information, leading to unsatisfactory transfer performance to dense prediction tasks such as semantic segmentation.
DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection
In this paper, we propose two novel techniques: InverseAug that inverses geometric-related augmentations, e. g., rotation, to enable accurate geometric alignment between lidar points and image pixels, and LearnableAlign that leverages cross-attention to dynamically capture the correlations between image and lidar features during fusion.
Point-Level Region Contrast for Object Detection Pre-Training
In this work we present point-level region contrast, a self-supervised pre-training approach for the task of object detection.
Lite Vision Transformer with Enhanced Self-Attention
We propose Lite Vision Transformer (LVT), a novel light-weight transformer network with two enhanced self-attention mechanisms to improve the model performances for mobile deployment.
Masked Feature Prediction for Self-Supervised Visual Pre-Training
We present Masked Feature Prediction (MaskFeat) for self-supervised pre-training of video models.
Ranked #8 on
Action Recognition
on AVA v2.2
(using extra training data)
MT-TransUNet: Mediating Multi-Task Tokens in Transformers for Skin Lesion Segmentation and Classification
However, these approaches formulated skin cancer diagnosis as a simple classification task, dismissing the potential benefit from lesion segmentation.
PartImageNet: A Large, High-Quality Dataset of Parts
To help address this problem, we propose PartImageNet, a large, high-quality dataset with part segmentation annotations.
OOD-CV: A Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images
One reason is that existing robustness benchmarks are limited, as they either rely on synthetic data or ignore the effects of individual nuisance factors.
Learning from Temporal Gradient for Semi-supervised Action Recognition
Our method achieves the state-of-the-art performance on three video action recognition benchmarks (i. e., Kinetics-400, UCF-101, and HMDB-51) under several typical semi-supervised settings (i. e., different ratios of labeled data).
TransMix: Attend to Mix for Vision Transformers
The confidence of the label will be larger if the corresponding input image is weighted higher by the attention map.
Occluded Video Instance Segmentation: Dataset and ICCV 2021 Challenge
To promote the development of occlusion understanding, we collect a large-scale dataset called OVIS for video instance segmentation in the occluded scenario.
Searching for TrioNet: Combining Convolution with Local and Global Self-Attention
In order to effectively search in this huge architecture space, we propose Hierarchical Sampling for better training of the supernet.
iBOT: Image BERT Pre-Training with Online Tokenizer
We present a self-supervised framework iBOT that can perform masked prediction with an online tokenizer.
Ranked #1 on
Unsupervised Image Classification
on ImageNet
Are Transformers More Robust Than CNNs?
Transformer emerges as a powerful tool for visual recognition.
Ranked #1 on
Adversarial Robustness
on Stylized ImageNet
Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Pose
The model is initialized from a few labelled images and is subsequently used to synthesize feature representations of unseen 3D views.
A Light-weight Interpretable Compositional Model for Nuclei Detection and Weakly-Supervised Segmentation
It only requires annotations on isolated nucleus, rather than on all nuclei in the dataset.
Nuisance-Label Supervision: Robustness Improvement by Free Labels
Nuisance factors are those irrelevant to a task, and an ideal model should be invariant to them.
Calibrating Concepts and Operations: Towards Symbolic Reasoning on Real Images
Our experiments show CCO substantially boosts the performance of neural symbolic methods on real images.
Image BERT Pre-training with Online Tokenizer
The success of language Transformers is primarily attributed to the pretext task of masked language modeling (MLM), where texts are first tokenized into semantically meaningful pieces.
SAME: Deformable Image Registration based on Self-supervised Anatomical Embeddings
In this work, we introduce a fast and accurate method for unsupervised 3D medical image registration.
RobustART: Benchmarking Robustness on Architecture Design and Training Techniques
Thus, we propose RobustART, the first comprehensive Robustness investigation benchmark on ImageNet regarding ARchitecture design (49 human-designed off-the-shelf architectures and 1200+ networks from neural architecture search) and Training techniques (10+ techniques, e. g., data augmentation) towards diverse noises (adversarial, natural, and system noises).
Exploring Simple 3D Multi-Object Tracking for Autonomous Driving
3D multi-object tracking in LiDAR point clouds is a key ingredient for self-driving vehicles.
Locally Enhanced Self-Attention: Combining Self-Attention and Convolution as Local and Context Terms
Self-Attention has become prevalent in computer vision models.
Progressive Stage-wise Learning for Unsupervised Feature Representation Enhancement
For a given unsupervised task, we design multilevel tasks and define different learning stages for the deep network.
Simulated Adversarial Testing of Face Recognition Models
In this work, we propose a framework for learning how to test machine learning algorithms using simulators in an adversarial manner in order to find weaknesses in the model before deploying it in critical scenarios.
Glance-and-Gaze Vision Transformer
It is motivated by the Glance and Gaze behavior of human beings when recognizing objects in natural scenes, with the ability to efficiently model both long-range dependencies and local context.
Rethinking Re-Sampling in Imbalanced Semi-Supervised Learning
In particular, we decouple the training of the representation and the classifier, and systematically investigate the effects of different data re-sampling techniques when training the whole network including a classifier as well as fine-tuning the feature extractor only.
Visual analogy: Deep learning versus compositional models
We compared human performance to that of two recent deep learning models (Siamese Network and Relation Network) directly trained to solve these analogy problems, as well as to that of a compositional model that assesses relational similarity between part-based representations.
Auto-FedAvg: Learnable Federated Averaging for Multi-Institutional Medical Image Segmentation
In this work, we design a new data-driven approach, namely Auto-FedAvg, where aggregation weights are dynamically adjusted, depending on data distributions across data silos and the current training progress of the models.
Self-Supervised Pillar Motion Learning for Autonomous Driving
Autonomous driving can benefit from motion behavior comprehension when interacting with diverse traffic participants in highly dynamic environments.
A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation
To deal with the large shape variance, we introduce Articulated Signed Distance Functions (A-SDF) to represent articulated shapes with a disentangled latent space, where we have separate codes for encoding shape and articulation.
CateNorm: Categorical Normalization for Robust Medical Image Segmentation
We propose a new normalization strategy, named categorical normalization (CateNorm), to normalize the activations according to categorical statistics.
Learning Part Segmentation through Unsupervised Domain Adaptation from Synthetic Vehicles
We believe our dataset provides a rich testbed to study UDA for part segmentation and will help to significantly push forward research in this area.
Weakly Supervised Instance Segmentation for Videos with Temporal Mask Consistency
However, existing approaches which rely only on image-level class labels predominantly suffer from errors due to (a) partial segmentation of objects and (b) missing object predictions.
Understanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mapping
Additionally, current approaches that deal with forgetting ignore the problem of catastrophic remembering, i. e. the worsening ability to discriminate between data from different tasks.
Ranked #1 on
Continual Learning
on ImageNet-50 (5 tasks)
CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning
Semi-supervised learning on class-imbalanced data, although a realistic problem, has been under studied.
Occluded Video Instance Segmentation: A Benchmark
On the OVIS dataset, the highest AP achieved by state-of-the-art algorithms is only 16. 3, which reveals that we are still at a nascent stage for understanding objects, instances, and videos in a real-world scenario.
Ranked #39 on
Video Instance Segmentation
on OVIS validation
NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation
Using differentiable rendering we estimate the 3D object pose by minimizing the reconstruction error between NeMo and the feature representation of the target image.
CORL: Compositional Representation Learning for Few-Shot Classification
In particular, during meta-learning, we train a knowledge base that consists of a dictionary of component representations and a dictionary of component activation maps that encode common spatial activation patterns of components.
Mask Guided Matting via Progressive Refinement Network
We propose Mask Guided (MG) Matting, a robust matting framework that takes a general coarse mask as guidance.
ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentation
We name this joint task as Depth-aware Video Panoptic Segmentation, and propose a new evaluation metric along with two derived datasets for it, which will be made available to the public.
Ranked #1 on
Video Panoptic Segmentation
on Cityscapes-VPS
(using extra training data)
Depth-aware Video Panoptic Segmentation
Monocular Depth Estimation
+2
Robust Instance Segmentation through Reasoning about Multi-Object Occlusion
The improved segmentation masks are, in turn, integrated into the network in a top-down manner to improve the image classification.
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers
As a result, MaX-DeepLab shows a significant 7. 1% PQ gain in the box-free regime on the challenging COCO dataset, closing the gap between box-based and box-free methods for the first time.
Ranked #12 on
Panoptic Segmentation
on COCO test-dev
Unsupervised Part Discovery via Feature Alignment
Specifically, given a training image, we find a set of similar images that show instances of the same object category in the same pose, through an affine alignment of their corresponding feature maps.
Robustness Out of the Box: Compositional Representations Naturally Defend Against Black-Box Patch Attacks
Second, we find that compositional deep networks, which have part-based representations that lead to innate robustness to natural occlusion, are robust to patch attacks on PASCAL3D+ and the German Traffic Sign Recognition Benchmark, without adversarial training.
Batch Normalization with Enhanced Linear Transformation
Batch normalization (BN) is a fundamental unit in modern deep networks, in which a linear transformation module was designed for improving BN's flexibility of fitting complex data distributions.
Can Temporal Information Help with Contrastive Self-Supervised Learning?
To this end, we present Temporal-aware Contrastive self-supervised learningTaCo, as a general paradigm to enhance video CSL.
Amodal Segmentation through Out-of-Task and Out-of-Distribution Generalization with a Bayesian Model
Moreover, by leveraging an outlier process, Bayesian models can further generalize out-of-distribution to segment partially occluded objects and to predict their amodal object boundaries.
Shape-Texture Debiased Neural Network Training
To prevent models from exclusively attending on a single cue in representation learning, we augment training data with images with conflicting shape and texture information (eg, an image of chimpanzee shape but with lemon texture) and, most importantly, provide the corresponding supervisions from shape and texture simultaneously.
Ranked #598 on
Image Classification
on ImageNet
CO2: Consistent Contrast for Unsupervised Visual Representation Learning
Regarding the similarity of the query crop to each crop from other images as "unlabeled", the consistency term takes the corresponding similarity of a positive crop as a pseudo label, and encourages consistency between these two similarities.
CoKe: Localized Contrastive Learning for Robust Keypoint Detection
In this paper, we introduce a contrastive learning framework for keypoint detection (CoKe).
Lymph Node Gross Tumor Volume Detection in Oncology Imaging via Relationship Learning Using Graph Neural Network
Specifically, we first utilize a 3D convolutional neural network with ROI-pooling to extract the GTV$_{LN}$'s instance-wise appearance features.
Lymph Node Gross Tumor Volume Detection and Segmentation via Distance-based Gating using 3D CT/PET Imaging in Radiotherapy
Finding, identifying and segmenting suspicious cancer metastasized lymph nodes from 3D multi-modality imaging is a clinical task of paramount importance.
ASAP-Net: Attention and Structure Aware Point Cloud Sequence Segmentation
In this paper, we further improve spatio-temporal point cloud feature learning with a flexible module called ASAP considering both attention and structure information across frames, which we find as two important factors for successful segmentation in dynamic point clouds.
Probabilistic Multi-modal Trajectory Prediction with Lane Attention for Autonomous Vehicles
As for vehicles, their trajectories are significantly influenced by the lane geometry and how to effectively use the lane information is of active interest.
Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation
Experiments on the NIH pancreas segmentation dataset and a multi-organ segmentation dataset show state-of-the-art performance of the proposed framework on semi-supervised medical image segmentation.
Compositional Convolutional Neural Networks: A Robust and Interpretable Model for Object Recognition under Occlusion
The structure of the compositional model enables CompositionalNets to decompose images into objects and context, as well as to further decompose object representations in terms of individual parts and the objects' pose.
Smooth Adversarial Training
SAT also works well with larger networks: it helps EfficientNet-L1 to achieve 82. 2% accuracy and 58. 6% robustness on ImageNet, outperforming the previous state-of-the-art defense by 9. 5% for accuracy and 11. 6% for robustness.
DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution
In this paper, we explore this mechanism in the backbone design for object detection.
Ranked #2 on
Object Detection
on AI-TOD
Detecting Scatteredly-Distributed, Small, andCritically Important Objects in 3D OncologyImaging via Decision Stratification
We focus on the detection and segmentation of oncology-significant (or suspicious cancer metastasized) lymph nodes (OSLNs), which has not been studied before as a computational task.
JSSR: A Joint Synthesis, Segmentation, and Registration System for 3D Multi-Modal Image Alignment of Large-scale Pathological CT Scans
We extensively evaluate our JSSR system on a large-scale medical image dataset containing 1, 485 patient CT imaging studies of four different phases (i. e., 5, 940 3D CT scans with pathological livers) on the registration, segmentation and synthesis tasks.
Robust Object Detection under Occlusion with Context-Aware CompositionalNets
In this work, we propose to overcome two limitations of CompositionalNets which will enable them to detect partially occluded objects: 1) CompositionalNets, as well as other DCNN architectures, do not explicitly separate the representation of the context from the object itself.
Domain Adaptive Relational Reasoning for 3D Multi-Organ Segmentation
In this paper, we present a novel unsupervised domain adaptation (UDA) method, named Domain Adaptive Relational Reasoning (DARR), to generalize 3D multi-organ segmentation models to medical data collected from different scanners and/or protocols (domains).
Organ at Risk Segmentation for Head and Neck Cancer using Stratified Learning and Neural Architecture Search
This is the goal of our work, where we introduce stratified organ at risk segmentation (SOARS), an approach that stratifies OARs into anchor, mid-level, and small & hard (S&H) categories.
PatchAttack: A Black-box Texture-based Attack with Reinforcement Learning
PatchAttack induces misclassifications by superimposing small textured patches on the input image.
Context-Aware Group Captioning via Self-Attention and Contrastive Features
In this paper, we introduce a new task, context-aware group captioning, which aims to describe a group of target images in the context of another group of related reference images.
Neural Architecture Search for Lightweight Non-Local Networks
However, it has been rarely explored to embed the NL blocks in mobile neural networks, mainly due to the following challenges: 1) NL blocks generally have heavy computation cost which makes it difficult to be applied in applications where computational resources are limited, and 2) it is an open problem to discover an optimal configuration to embed NL blocks into mobile neural networks.
Ranked #60 on
Neural Architecture Search
on ImageNet
Are Labels Necessary for Neural Architecture Search?
Existing neural network architectures in computer vision -- whether designed by humans or by machines -- were typically found using both images and their associated labels.
Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation
The ability to detect failures and anomalies are fundamental requirements for building reliable systems for computer vision applications, especially safety-critical applications of semantic segmentation, such as autonomous driving and medical image analysis.
Ranked #8 on
Anomaly Detection
on Road Anomaly
(using extra training data)
Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation
In this paper, we attempt to remove this constraint by factorizing 2D self-attention into two 1D self-attentions.
Ranked #4 on
Panoptic Segmentation
on Cityscapes val
(using extra training data)
Compositional Convolutional Neural Networks: A Deep Architecture with Innate Robustness to Partial Occlusion
Inspired by the success of compositional models at classifying partially occluded objects, we propose to integrate compositional models and DCNNs into a unified deep model with innate robustness to partial occlusion.
When Radiology Report Generation Meets Knowledge Graph
In addition, we proposed a new evaluation metric for radiology image reporting with the assistance of the same composed graph.
Object as Hotspots: An Anchor-Free 3D Object Detection Approach via Firing of Hotspots
Accurate 3D object detection in LiDAR based point clouds suffers from the challenges of data sparsity and irregularities.
Ranked #3 on
3D Object Detection
on KITTI Pedestrians Moderate
AtomNAS: Fine-Grained End-to-End Neural Architecture Search
We propose a fine-grained search space comprised of atomic blocks, a minimal search unit that is much smaller than the ones used in recent NAS algorithms.
Ranked #61 on
Neural Architecture Search
on ImageNet
Learning from Synthetic Animals
Despite great success in human parsing, progress for parsing other deformable articulated objects, like animals, is still limited by the lack of labeled data.
Identity Preserve Transform: Understand What Activity Classification Models Have Learnt
This can explain why an activity classification model usually fails to generalize to datasets it is not trained on.
DASZL: Dynamic Action Signatures for Zero-shot Learning
There are many realistic applications of activity recognition where the set of potential activity descriptions is combinatorially large.
RSA: Randomized Simulation as Augmentation for Robust Human Action Recognition
In this paper, we propose the Randomized Simulation as Augmentation (RSA) framework which augments real-world training data with synthetic data to improve the robustness of action recognition networks.
Identifying Model Weakness with Adversarial Examiner
Different from the existing strategy to always give the same (distribution of) test data, the adversarial examiner will dynamically select the next test data to hand out based on the testing history so far, with the goal being to undermine the model's performance.
Deeply Shape-guided Cascade for Instance Segmentation
The key to a successful cascade architecture for precise instance segmentation is to fully leverage the relationship between bounding box detection and mask segmentation across multiple stages.
Adversarial Examples Improve Image Recognition
We show that AdvProp improves a wide range of models on various image recognition tasks and performs better when the models are bigger.
Ranked #4 on
Domain Generalization
on VizWiz-Classification
Rethinking Normalization and Elimination Singularity in Neural Networks
To address this issue, we propose BatchChannel Normalization (BCN), which uses batch knowledge to avoid the elimination singularities in the training of channel-normalized models.
Localizing Occluders with Compositional Convolutional Networks
Our experimental results demonstrate that the proposed extensions increase the model's performance at localizing occluders as well as at classifying partially occluded objects.
Grouped Spatial-Temporal Aggregation for Efficient Action Recognition
This decomposition is more parameter-efficient and enables us to quantitatively analyze the contributions of spatial and temporal features in different layers.
Universal Physical Camouflage Attacks on Object Detectors
In this paper, we study physical adversarial attacks on object detectors in the wild.
TDAPNet: Prototype Network with Recurrent Top-Down Attention for Robust Object Classification under Partial Occlusion
Despite deep convolutional neural networks' great success in object classification, it suffers from severe generalization performance drop under occlusion due to the inconsistency between training and testing data.
Hyper-Pairing Network for Multi-Phase Pancreatic Ductal Adenocarcinoma Segmentation
Pancreatic ductal adenocarcinoma (PDAC) is one of the most lethal cancers with an overall five-year survival rate of 8%.
FusionNet: Incorporating Shape and Texture for Abnormality Detection in 3D Abdominal CT Scans
Second, a FusionNet is proposed to take both the binary mask and CT image as input and perform a binary classification.
Deep Differentiable Random Forests for Age Estimation
Both of them connect split nodes to the top layer of convolutional neural networks (CNNs) and deal with inhomogeneous data by jointly learning input-dependent data partitions at the split nodes and age distributions at the leaf nodes.
Multi-Scale Attentional Network for Multi-Focal Segmentation of Active Bleed after Pelvic Fractures
Trauma is the worldwide leading cause of death and disability in those younger than 45 years, and pelvic fractures are a major source of morbidity and mortality.
Intriguing properties of adversarial training at scale
This two-domain hypothesis may explain the issue of BN when training with a mixture of clean and adversarial images, as estimating normalization statistics of this mixture distribution is challenging.
V-NAS: Neural Architecture Search for Volumetric Medical Image Segmentation
Deep learning algorithms, in particular 2D and 3D fully convolutional neural networks (FCNs), have rapidly become the mainstream methodology for volumetric medical image segmentation.
Combining Compositional Models and Deep Networks For Robust Object Classification under Occlusion
In this work, we combine DCNNs and compositional object models to retain the best of both approaches: a discriminative model that is robust to partial occlusion and mask attacks.
Robustness of Object Recognition under Extreme Occlusion in Humans and Computational Models
We test both humans and the above-mentioned computational models in a challenging task of object recognition under extreme occlusion, where target objects are heavily occluded by irrelevant real objects in real backgrounds.
Structured Prediction using cGANs with Fusion Discriminator
We propose the fusion discriminator, a single unified framework for incorporating conditional information into a generative adversarial network (GAN) for a variety of distinct structured prediction tasks, including image synthesis, semantic segmentation, and depth estimation.
Prior-aware Neural Network for Partially-Supervised Multi-Organ Segmentation
Accurate multi-organ abdominal CT segmentation is essential to many clinical applications such as computer-aided intervention.
Semantic-Aware Knowledge Preservation for Zero-Shot Sketch-Based Image Retrieval
Sketch-based image retrieval (SBIR) is widely recognized as an important vision problem which implies a wide range of real-world applications.
An Alarm System For Segmentation Algorithm Based On Shape Model
Motivated by this, in this paper, we learn a feature space using the shape information which is a strong prior shared among different datasets and robust to the appearance variation of input data. The shape feature is captured using a Variational Auto-Encoder (VAE) network that trained with only the ground truth masks.
Micro-Batch Training with Batch-Channel Normalization and Weight Standardization
Batch Normalization (BN) has become an out-of-box technique to improve deep network training.
Ranked #76 on
Instance Segmentation
on COCO minival
Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
Therefore, we propose to search the network level structure in addition to the cell level structure, which forms a hierarchical architecture search space.
Ranked #7 on
Semantic Segmentation
on PASCAL VOC 2012 val
CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressions
Yet there has been evidence that current benchmark datasets suffer from bias, and current state-of-the-art models cannot be easily evaluated on their intermediate reasoning process.
Ranked #1 on
Referring Expression Segmentation
on CLEVR-Ref+
ELASTIC: Improving CNNs with Dynamic Scaling Policies
We formulate the scaling policy as a non-linear function inside the network's structure that (a) is learned from data, (b) is instance specific, (c) does not add extra computation, and (d) can be applied on any network architecture.
Feature Denoising for Improving Adversarial Robustness
This study suggests that adversarial perturbations on images lead to noise in the features constructed by these networks.
Ranked #1 on
Adversarial Defense
on CAAD 2018
Learning Transferable Adversarial Examples via Ghost Networks
The critical principle of ghost networks is to apply feature-level perturbations to an existing model to potentially create a huge set of diverse models.
Neural Rejuvenation: Improving Deep Network Training by Enhancing Computational Resource Utilization
By simply replacing standard optimizers with Neural Rejuvenation, we are able to improve the performances of neural networks by a very large margin while using similar training efforts and maintaining their original resource usages.
3D Semi-Supervised Learning with Uncertainty-Aware Multi-View Co-Training
Meanwhile, a fully-supervised method based on our approach achieved state-of-the-art performances on both the LiTS liver tumor segmentation and the Medical Segmentation Decathlon (MSD) challenge, demonstrating the robustness and value of our framework, even when fully supervised training is feasible.
Semantic Part Detection via Matching: Learning to Generalize to Novel Viewpoints from Limited Training Data
In particular, this enables images in the training dataset to be matched to a virtual 3D model of the object (for simplicity, we assume that the object viewpoint can be estimated by standard techniques).
Robust Face Detection via Learning Small Faces on Hard Images
In this paper, we propose that the robustness of a face detector against hard faces can be improved by learning small faces on hard images.
Ranked #8 on
Face Detection
on WIDER Face (Hard)
OriNet: A Fully Convolutional Network for 3D Human Pose Estimation
We use limb orientations as a new way to represent 3D poses and bind the orientation together with the bounding box of each limb region to better associate images and predictions.
Ranked #76 on
3D Human Pose Estimation
on MPI-INF-3DHP
(AUC metric)
Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding
Performance on the five tasks of depth estimation, optical flow estimation, odometry, moving object segmentation and scene flow estimation shows that our approach outperforms other SoTA methods.
Weakly Supervised Region Proposal Network and Object Detection
The Convolutional Neural Network (CNN) based region proposal generation method (i. e. region proposal network), trained using bounding box annotations, is an essential component in modern fully supervised object detectors.
Rethinking Monocular Depth Estimation with Adversarial Training
Most existing approaches treat depth estimation as a regression problem with a local pixel-wise loss function.
PCL: Proposal Cluster Learning for Weakly Supervised Object Detection
The iterative instance classifier refinement is implemented online using multiple streams in convolutional neural networks, where the first is an MIL network and the others are for instance classifier refinement supervised by the preceding one.
Ranked #1 on
Weakly Supervised Object Detection
on ImageNet
Resisting Large Data Variations via Introspective Transformation Network
Experimental results verify that our approach significantly improves the ability of deep networks to resist large variations between training and testing data and achieves classification accuracy improvements on several benchmark datasets, including MNIST, affNIST, SVHN, CIFAR-10 and miniImageNet.
Knowledge Distillation in Generations: More Tolerant Teachers Educate Better Students
We focus on the problem of training a deep neural network in generations.
SampleAhead: Online Classifier-Sampler Communication for Learning from Synthesized Data
But, this raises an important problem in active vision: given an {\bf infinite} data space, how to effectively sample a {\bf finite} subset to train a visual classifier?
Adversarial Attacks and Defences Competition
To accelerate research on adversarial examples and robustness of machine learning classifiers, Google Brain organized a NIPS 2017 competition that encouraged researchers to develop new methods to generate adversarial examples as well as to develop new ways to defend against them.
Scene Graph Parsing as Dependency Parsing
The scene graphs generated by our learned neural dependency parser achieve an F-score similarity of 49. 67% to ground truth graphs on our evaluation set, surpassing best previous approaches by 5%.
Improving Transferability of Adversarial Examples with Input Diversity
We hope that our proposed attack strategy can serve as a strong benchmark baseline for evaluating the robustness of networks to adversaries and the effectiveness of different defense methods in the future.
Deep Co-Training for Semi-Supervised Image Recognition
We present Deep Co-Training, a deep learning based method inspired by the Co-Training framework.
Unleashing the Potential of CNNs for Interpretable Few-Shot Learning
Our models are based on the idea of encoding objects in terms of visual concepts, which are interpretable visual cues represented by the feature vectors within CNNs.
Deep Regression Forests for Age Estimation
Age estimation from facial images is typically cast as a nonlinear regression problem.
Ranked #6 on
Age Estimation
on FGNET
Progressive Neural Architecture Search
We propose a new method for learning the structure of convolutional neural networks (CNNs) that is more efficient than recent state-of-the-art methods based on reinforcement learning and evolutionary algorithms.
Ranked #15 on
Neural Architecture Search
on NAS-Bench-201, ImageNet-16-120
(Accuracy (Val) metric)
