Search Results for author:
Found 96 papers, 66 papers with code
UniDepth: Universal Monocular Metric Depth Estimation
However, the remarkable accuracy of recent MMDE methods is confined to their training domains.
Ranked #1 on
Monocular Depth Estimation
on NYU-Depth V2
(using extra training data)
DexDribbler: Learning Dexterous Soccer Manipulation via Dynamic Supervision
Learning dexterous locomotion policy for legged robots is becoming increasingly popular due to its ability to handle diverse terrains and resemble intelligent behaviors.
S$^3$M-Net: Joint Learning of Semantic Segmentation and Stereo Matching for Autonomous Driving
Hence, in this article, we introduce S$^3$M-Net, a novel joint learning framework developed to perform semantic segmentation and stereo matching simultaneously.
ICGNet: A Unified Approach for Instance-Centric Grasping
Executing a successful grasp in a cluttered environment requires multiple levels of scene understanding: First, the robot needs to analyze the geometric properties of individual objects to find feasible grasps.
MuRF: Multi-Baseline Radiance Fields
We present Multi-Baseline Radiance Fields (MuRF), a general feed-forward approach to solving sparse view synthesis under multiple different baseline settings (small and large baselines, and different number of input views).
Gaussian Grouping: Segment and Edit Anything in 3D Scenes
To address this issue, we propose Gaussian Grouping, which extends Gaussian Splatting to jointly reconstruct and segment anything in open-world 3D scenes.
Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding
The real-world deployment of an autonomous driving system requires its components to run on-board and in real-time, including the motion prediction module that predicts the future trajectories of surrounding traffic participants.
COOLer: Class-Incremental Learning for Appearance-Based Multiple Object Tracking
Continual learning allows a model to learn multiple tasks sequentially while retaining the old knowledge without the training data of the preceding tasks.
DARTH: Holistic Test-time Adaptation for Multiple Object Tracking
However, the nature of a MOT system is manifold - requiring object detection and instance association - and adapting all its components is non-trivial.
Distilling ODE Solvers of Diffusion Models into Smaller Steps
Notably, our method incurs negligible computational overhead compared to previous distillation techniques, facilitating straightforward and rapid integration with existing samplers.
Video Task Decathlon: Unifying Image and Video Tasks in Autonomous Driving
VTD is a promising new direction for exploring the unification of perception tasks in autonomous driving.
Three Ways to Improve Verbo-visual Fusion for Dense 3D Visual Grounding
A common formulation to tackle 3D visual grounding is grounding-by-detection, where localization is done via bounding boxes.
R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras
We propose R3D3, a multi-camera system for dense 3D reconstruction and ego-motion estimation.
MolGrapher: Graph-based Visual Recognition of Chemical Structures
In addition, we introduce a large-scale benchmark of annotated real molecule images, USPTO-30K, to spur research on this critical topic.
Video OWL-ViT: Temporally-consistent open-world localization in video
Our model is end-to-end trainable on video data and enjoys improved temporal consistency compared to tracking-by-detection baselines, while retaining the open-world capabilities of the backbone detector.
Dual Aggregation Transformer for Image Super-Resolution
Based on the above idea, we propose a novel Transformer model, Dual Aggregation Transformer (DAT), for image SR. Our DAT aggregates features across spatial and channel dimensions, in the inter-block and intra-block dual manner.
Ranked #6 on
Image Super-Resolution
on Manga109 - 4x upscaling
Strategic Preys Make Acute Predators: Enhancing Camouflaged Object Detectors by Generating Camouflaged Objects
On the prey side, we propose an adversarial training framework, Camouflageator, which introduces an auxiliary generator to generate more camouflaged objects that are harder for a COD method to detect.
Cascade-DETR: Delving into High-Quality Universal Object Detection
While dominating on the COCO benchmark, recent Transformer-based detection methods are not competitive in diverse domains.
Segment Anything Meets Point Tracking
The Segment Anything Model (SAM) has established itself as a powerful zero-shot image segmentation model, enabled by efficient point-centric annotation and prompt-based models.
SSCBench: Monocular 3D Semantic Scene Completion Benchmark in Street Views
Monocular scene understanding is a foundational component of autonomous systems.
3D Semantic Scene Completion
3D Semantic Scene Completion from a single 2D image
Segment Anything in High Quality
HQ-SAM is only trained on the introduced detaset of 44k masks, which takes only 4 hours on 8 GPUs.
Ranked #1 on
Zero-Shot Instance Segmentation
on LVIS v1.0 val
Condition-Invariant Semantic Segmentation
Motivated by these findings, we propose to leverage stylization in performing feature-level adaptation by aligning the internal network features extracted by the encoder of the network from the original and the stylized view of each input image with a novel feature invariance loss.
Maskomaly:Zero-Shot Mask Anomaly Segmentation
We present a simple and practical framework for anomaly segmentation called Maskomaly.
How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontiers
Our method propagates semantics on the scene graphs based on language priors and scene statistics to introduce semantic knowledge to the geometric frontiers.
OVTrack: Open-Vocabulary Multiple Object Tracking
This leaves contemporary MOT methods limited to a small set of pre-defined object categories.
iDisc: Internal Discretization for Monocular Depth Estimation
Our method sets the new state of the art with significant improvements on NYU-Depth v2 and KITTI, outperforming all published methods on the official KITTI benchmark.
Ranked #2 on
Surface Normals Estimation
on NYU Depth v2
Mask-Free Video Instance Segmentation
A consistency loss is then enforced on the found matches.
TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction
We present TrafficBots, a multi-agent policy built upon motion prediction and end-to-end driving, and based on TrafficBots we obtain a world model tailored for the planning module of autonomous vehicles.
A Multiplicative Value Function for Safe and Efficient Reinforcement Learning
To this end, we propose a safe model-free RL algorithm with a novel multiplicative value function consisting of a safety critic and a reward critic.
Scaling Vision Transformers to 22 Billion Parameters
The scaling of Transformers has driven breakthrough capabilities for language models.
Ranked #1 on
Zero-Shot Transfer Image Classification
on ObjectNet
Uncertainty-Driven Dense Two-View Structure from Motion
This work introduces an effective and practical solution to the dense two-view structure from motion (SfM) problem.
BiBench: Benchmarking and Analyzing Network Binarization
Network binarization emerges as one of the most promising compression approaches offering extraordinary computation and memory savings by minimizing the bit-width.
3DPPE: 3D Point Positional Encoding for Transformer-based Multi-Camera 3D Object Detection
Although 3D measurements are not available at the inference time of monocular 3D object detection, 3DPPE uses predicted depth to approximate the real point positions.
CC-3DT: Panoramic 3D Object Tracking via Cross-Camera Fusion
To track the 3D locations and trajectories of the other traffic participants at any given time, modern autonomous vehicles are equipped with multiple cameras that cover the vehicle's full surroundings.
3DPPE: 3D Point Positional Encoding for Multi-Camera 3D Object Detection Transformers
Although 3D measurements are not available at the inference time of monocular 3D object detection, 3DPPE uses predicted depth to approximate the real point positions.
Unifying Flow, Stereo and Depth Estimation
We present a unified formulation and model for three motion and 3D perception tasks: optical flow, rectified stereo matching and unrectified stereo depth estimation from posed images.
Ranked #1 on
Optical Flow Estimation
on Sintel-clean
Normalization Perturbation: A Simple Domain Generalization Method for Real-World Domain Shifts
Thus, we propose to perturb the channel statistics of source domain features to synthesize various latent styles, so that the trained deep model can perceive diverse potential domains and generalizes well even without observations of target domain data in training.
Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racing
The resulting policy directly infers control commands with feature representations learned from raw images, forgoing the need for globally-consistent state estimation, trajectory planning, and handcrafted control design.
Composite Learning for Robust and Effective Dense Predictions
In this paper, we find that jointly training a dense prediction (target) task with a self-supervised (auxiliary) task can consistently improve the performance of the target task, while eliminating the need for labeling auxiliary tasks.
Ranked #102 on
Semantic Segmentation
on NYU Depth v2
QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Tracking
In this paper, we present Quasi-Dense Similarity Learning, which densely samples hundreds of object regions on a pair of images for contrastive learning.
Ranked #4 on
Multiple Object Tracking
on BDD100K test
Fast Hierarchical Learning for Few-Shot Object Detection
These approaches however suffer from ``catastrophic forgetting'' issue due to finetuning of base detector, leading to sub-optimal performance on the base classes.
Uncertainty Guided Policy for Active Robotic 3D Reconstruction using Neural Radiance Fields
In this paper, we tackle the problem of active robotic 3D reconstruction of an object.
Spatio-Temporal Action Detection Under Large Motion
Current methods for spatiotemporal action tube detection often extend a bounding box proposal at a given keyframe into a 3D temporal cuboid and pool features from nearby frames.
Video Mask Transfiner for High-Quality Video Instance Segmentation
While Video Instance Segmentation (VIS) has seen rapid progress, current approaches struggle to predict high-quality masks with accurate boundary details.
Ranked #1 on
Video Instance Segmentation
on HQ-YTVIS
Tracking Every Thing in the Wild
Our experiments show that TETA evaluates trackers more comprehensively, and TETer achieves significant improvements on the challenging large-scale datasets BDD100K and TAO compared to the state-of-the-art.
Ranked #4 on
Multi-Object Tracking
on TAO
SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation
Adapting to a continuously evolving environment is a safety-critical challenge inevitably faced by all autonomous driving systems.
Learning Online Multi-Sensor Depth Fusion
Multi-sensor depth fusion is able to substantially improve the robustness and accuracy of 3D reconstruction methods, but existing techniques are not robust enough to handle sensors which operate with diverse value ranges as well as noise and outlier statistics.
LiDAR Snowfall Simulation for Robust 3D Object Detection
Due to the difficulty of collecting and annotating training data in this setting, we propose a physically based method to simulate the effect of snowfall on real clear-weather LiDAR point clouds.
Ranked #1 on
3D Object Detection
on Heavy Snowfall
Transforming Model Prediction for Tracking
Optimization based tracking methods have been widely successful by integrating a target model prediction module, providing effective global reasoning by minimizing an objective function.
Ranked #19 on
Visual Object Tracking
on LaSOT
(Precision metric)
Generative Cooperative Learning for Unsupervised Video Anomaly Detection
Video anomaly detection is well investigated in weakly-supervised and one-class classification (OCC) settings.
Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondences
We propose Probabilistic Warp Consistency, a weakly-supervised learning objective for semantic matching.
RePaint: Inpainting using Denoising Diffusion Probabilistic Models
In this work, we propose RePaint: A Denoising Diffusion Probabilistic Model (DDPM) based inpainting approach that is applicable to even extreme masks.
SAGA: Stochastic Whole-Body Grasping with Contact
Given an initial pose and the generated whole-body grasping pose as the start and end of the motion respectively, we design a novel contact-aware generative motion infilling module to generate a diverse set of grasp-oriented motions.
Mask Transfiner for High-Quality Instance Segmentation
Instead of operating on regular dense tensors, our Mask Transfiner decomposes and represents the image regions as a quadtree.
Ranked #1 on
Instance Segmentation
on BDD100K val
Normalizing Flow as a Flexible Fidelity Objective for Photo-Realistic Super-resolution
Super-resolution is an ill-posed problem, where a ground-truth high-resolution image represents only one possibility in the space of plausible solutions.
Dense Prediction with Attentive Feature Aggregation
Our experiments show consistent and significant improvements on challenging semantic segmentation benchmarks, including Cityscapes, BDD100K, and Mapillary Vistas, at negligible computational and parameter overhead.
TACS: Taxonomy Adaptive Cross-Domain Semantic Segmentation
In many real-world settings, the target domain task requires a different taxonomy than the one imposed by the source domain.
End-to-End Urban Driving by Imitating a Reinforcement Learning Coach
Our end-to-end agent achieves a 78% success rate while generalizing to a new town and new weather on the NoCrash-dense benchmark and state-of-the-art performance on the challenging public routes of the CARLA LeaderBoard.
Deep Reparametrization of Multi-Frame Super-Resolution and Denoising
The deep reparametrization allows us to directly model the image formation process in the latent space, and to integrate learned image priors into the prediction.
Ranked #4 on
Burst Image Super-Resolution
on BurstSR
On the Practicality of Deterministic Epistemic Uncertainty
We find that, while DUMs scale to realistic vision tasks and perform well on OOD detection, the practicality of current methods is undermined by poor calibration under distributional shifts.
Out of Distribution (OOD) Detection
Semantic Segmentation
+1
Prototypical Cross-Attention Networks for Multiple Object Tracking and Segmentation
We propose Prototypical Cross-Attention Network (PCAN), capable of leveraging rich spatio-temporal information for online multiple object tracking and segmentation.
Ranked #1 on
Video Instance Segmentation
on BDD100K val
Multi-Object Tracking and Segmentation
Multiple Object Track and Segmentation
+3
Robust Object Detection via Instance-Level Temporal Cycle Confusion
Building reliable object detectors that are robust to domain shifts, such as various changes in context, viewpoint, and object appearances, is critical for real-world applications.
Warp Consistency for Unsupervised Learning of Dense Correspondences
From our observations and empirical results, we design a general unsupervised objective employing two of the derived constraints.
Monocular Quasi-Dense 3D Object Tracking
Experiments on our proposed simulation data and real-world benchmarks, including KITTI, nuScenes, and Waymo datasets, show that our tracking framework offers robust object association and tracking on urban-driving scenarios.
Ranked #7 on
Multiple Object Tracking
on KITTI Tracking test
Exploring Cross-Image Pixel Contrast for Semantic Segmentation
Inspired by the recent advance in unsupervised contrastive representation learning, we propose a pixel-wise contrastive framework for semantic segmentation in the fully supervised setting.
Instance-Aware Predictive Navigation in Multi-Agent Environments
To decide the action at each step, we seek the action sequence that can lead to safe future states based on the prediction module outputs by repeatedly sampling likely action sequences.
Quasi-Dense Similarity Learning for Multiple Object Tracking
Compared to methods with similar detectors, it boosts almost 10 points of MOTA and significantly decreases the number of ID switches on BDD100K and Waymo datasets.
Ranked #1 on
One-Shot Object Detection
on PASCAL VOC 2012 val
Frustratingly Simple Few-Shot Object Detection
Such a simple approach outperforms the meta-learning methods by roughly 2~20 points on current benchmarks and sometimes even doubles the accuracy of the prior methods.
Ranked #17 on
Few-Shot Object Detection
on MS-COCO (30-shot)
Task-Aware Feature Generation for Zero-Shot Compositional Learning
In this work, we propose a task-aware feature generation (TFG) framework for compositional learning, which generates features of novel visual concepts by transferring knowledge from previously seen concepts.
TAFE-Net: Task-Aware Feature Embeddings for Low Shot Learning
We show that TAFE-Net is highly effective in generalizing to new tasks or concepts and evaluate the TAFE-Net on a range of benchmarks in zero-shot and few-shot learning.
Ranked #1 on
Few-Shot Image Classification
on aPY - 0-Shot
Hierarchical Discrete Distribution Decomposition for Match Density Estimation
Explicit representations of the global match distributions of pixel-wise correspondences between pairs of images are desirable for uncertainty estimation and downstream applications.
Ranked #13 on
Optical Flow Estimation
on KITTI 2015 (train)
Few-shot Object Detection via Feature Reweighting
The feature learner extracts meta features that are generalizable to detect novel object classes, using training data from base classes with sufficient samples.
Ranked #21 on
Few-Shot Object Detection
on MS-COCO (30-shot)
Disentangling Propagation and Generation for Video Prediction
A dynamic scene has two types of elements: those that move fluidly and can be predicted from previous frames, and those which are disoccluded (exposed) and cannot be extrapolated.
Joint Monocular 3D Vehicle Detection and Tracking
The framework can not only associate detections of vehicles in motion over time, but also estimate their complete 3D bounding box information from a sequence of 2D images captured on a moving platform.
Ranked #12 on
Multiple Object Tracking
on KITTI Tracking test
Deep Object-Centric Policies for Autonomous Driving
While learning visuomotor skills in an end-to-end manner is appealing, deep neural networks are often uninterpretable and fail in surprising ways.
Characterizing Adversarial Examples Based on Spatial Consistency Information for Semantic Segmentation
In this paper, we aim to characterize adversarial examples based on spatial context information in semantic segmentation.
Deep Mixture of Experts via Shallow Embedding
Larger networks generally have greater representational power at the cost of increased computational complexity.
PairedCycleGAN: Asymmetric Style Transfer for Applying and Removing Makeup
This paper introduces an automatic method for editing a portrait photo so that the subject appears to be wearing makeup in the style of another person in a reference photo.
BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning
Datasets drive vision progress, yet existing driving datasets are impoverished in terms of visual content and supported tasks to study multitask learning for autonomous driving.
Ranked #5 on
Multiple Object Tracking
on BDD100K test
Reinforcement Learning from Imperfect Demonstrations
We propose a unified reinforcement learning algorithm, Normalized Actor-Critic (NAC), that effectively normalizes the Q-function, reducing the Q-values of actions unseen in the demonstration data.
SkipNet: Learning Dynamic Routing in Convolutional Networks
While deeper convolutional networks are needed to achieve maximum accuracy in visual perception tasks, for many inputs shallower networks are sufficient.
Deep Layer Aggregation
We augment standard architectures with deeper aggregation to better fuse information across layers.
Interactive 3D Modeling with a Generative Adversarial Network
This paper proposes the idea of using a generative adversarial network (GAN) to assist a novice user in designing real-world shapes with a simple interface.
TextureGAN: Controlling Deep Image Synthesis with Texture Patches
In this paper, we investigate deep image synthesis guided by sketch, color, and texture.
Ranked #2 on
Image Reconstruction
on Edge-to-Shoes
IDK Cascades: Fast Deep Learning by Learning not to Overthink
Advances in deep learning have led to substantial increases in prediction accuracy but have been accompanied by increases in the cost of rendering predictions.
Dilated Residual Networks
Convolutional networks for image classification progressively reduce resolution until the image is represented by tiny feature maps in which the spatial structure of the scene is no longer discernible.
FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
In this paper, we introduce the first domain adaptive semantic segmentation method, proposing an unsupervised adversarial approach to pixel prediction problems.
Ranked #2 on
Image-to-Image Translation
on SYNTHIA Fall-to-Winter
End-to-end Learning of Driving Models from Large-scale Video Datasets
Robust perception-action models should be learned from training data with diverse visual appearances and realistic behaviors, yet current approaches to deep visuomotor policy learning have been generally limited to in-situ models learned from a single vehicle or a simulation environment.
Scribbler: Controlling Deep Image Synthesis with Sketch and Color
In this paper, we propose a deep adversarial image synthesis architecture that is conditioned on sketched boundaries and sparse color strokes to generate realistic cars, bedrooms, or faces.
Semantic Scene Completion from a Single Depth Image
This paper focuses on semantic scene completion, a task for producing a complete 3D voxel representation of volumetric occupancy and semantic labels for a scene from a single-view depth map observation.
Ranked #2 on
3D Semantic Scene Completion
on KITTI-360
ShapeNet: An Information-Rich 3D Model Repository
We present ShapeNet: a richly-annotated, large-scale repository of shapes represented by 3D CAD models of objects.
Multi-Scale Context Aggregation by Dilated Convolutions
State-of-the-art models for semantic segmentation are based on adaptations of convolutional networks that had originally been designed for image classification.
Ranked #11 on
Semantic Segmentation
on CamVid
LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
While there has been remarkable progress in the performance of visual recognition algorithms, the state-of-the-art models tend to be exceptionally data-hungry.
Semantic Alignment of LiDAR Data at City Scale
This paper describes an automatic algorithm for global alignment of LiDAR data collected with Google Street View cars in urban environments.
3D ShapeNets: A Deep Representation for Volumetric Shapes
Our model, 3D ShapeNets, learns the distribution of complex 3D shapes across different object categories and arbitrary poses from raw CAD data, and discovers hierarchical compositional part representations automatically.
Ranked #35 on
3D Point Cloud Classification
on ModelNet40
(Mean Accuracy metric)
3D Reconstruction from Accidental Motion
We have discovered that 3D reconstruction can be achieved from asingle still photographic capture due to accidental motions of thephotographer, even while attempting to hold the camera still.
