Search Results for author:
Found 45 papers, 38 papers with code
Distilling Vision-Language Models on Millions of Videos
We show the adapted video-language model performs well on a wide range of video-language benchmarks.
Language-conditioned Detection Transformer
We use this detector to pseudo-label images with image-level labels.
Predicting a Protein's Stability under a Million Mutations
We build Mutate Everything on top of ESM2 and AlphaFold, neither of which were trained to predict thermodynamic stability.
Training a Large Video Model on a Single Machine in a Day
Videos are big, complex to pre-process, and slow to train on.
Ranked #1 on
Multi-Instance Retrieval
on EPIC-KITCHENS-100
Long-tail Detection with Effective Class-Margins
Large-scale object detection and instance segmentation face a severe data imbalance.
PartDistillation: Learning Parts From Instance Segmentation
PartDistillation transfers the part information of an instance segmentation model into a part segmentation model through self-supervised self-training on a large dataset.
NMS Strikes Back
Our detector that trains Deformable-DETR with traditional IoU-based label assignment achieved 50. 2 COCO mAP within 12 epochs (1x schedule) with ResNet50 backbone, outperforming all existing traditional or transformer-based detectors in this setting.
Ranked #2 on
Object Detection
on COCO-O
(using extra training data)
Learning Video Representations from Large Language Models
We introduce LaViLa, a new approach to learning video-language representations by leveraging Large Language Models (LLMs).
Ranked #1 on
Action Recognition
on Charades-Ego
Real-time Online Video Detection with Temporal Smoothing Transformers
Streaming video recognition reasons about objects and their actions in every frame of a video.
Cross-view Transformers for real-time Map-view Semantic Segmentation
The architecture consists of a convolutional image encoder for each view and cross-view transformer layers to infer a map-view semantic segmentation.
Ranked #8 on
Bird's-Eye View Semantic Segmentation
on nuScenes
Global Tracking Transformers
The transformer encodes object features from all frames, and uses trajectory queries to group them into trajectories.
Ranked #12 on
Multi-Object Tracking
on SportsMOT
(using extra training data)
Learning from All Vehicles
In this paper, we present a system to train driving policies from experiences collected not just from the ego-vehicle, but all vehicles that it observes.
Ranked #5 on
Autonomous Driving
on CARLA Leaderboard
Detecting Twenty-thousand Classes using Image-level Supervision
For the first time, we train a detector with all the twenty-one-thousand classes of the ImageNet dataset and show that it generalizes to new datasets without finetuning.
Ranked #2 on
Open Vocabulary Object Detection
on OpenImages-v4
Multimodal Virtual Point 3D Detection
For autonomous driving, this means that large objects close to the sensors are easily visible, but far-away or small objects comprise only one measurement or two.
Ranked #63 on
3D Object Detection
on nuScenes
Towards Long-Form Video Understanding
Our world offers a never-ending stream of visual stimuli, yet today's vision systems only accurately recognize patterns within a few seconds.
Ranked #26 on
Action Recognition
on AVA v2.2
Learning to drive from a world on rails
This assumption greatly simplifies the learning problem, factorizing the dynamics into a nonreactive world model and a low-dimensional and compact forward model of the ego-vehicle.
Ranked #12 on
Autonomous Driving
on CARLA Leaderboard
Probabilistic two-stage detection
We develop a probabilistic interpretation of two-stage object detection.
Ranked #20 on
Object Detection
on COCO-O
Simple multi-dataset detection
Experiments show our learned taxonomy outperforms a expert-designed taxonomy in all datasets.
Memory Optimization for Deep Networks
Deep learning is slowly, but steadily, hitting a memory bottleneck.
Domain Adaptation Through Task Distillation
We use these recognition datasets to link up a source and target domain to transfer models between them in a task distillation framework.
Center-based 3D Object Detection and Tracking
Three-dimensional objects are commonly represented as 3D boxes in a point-cloud.
Ranked #1 on
Robust 3D Object Detection
on nuScenes-C
Lossless Image Compression through Super-Resolution
We introduce a simple and efficient lossless image compression algorithm.
Tracking Objects as Points
Nowadays, tracking is dominated by pipelines that perform object detection followed by temporal association, also known as tracking-by-detection.
Ranked #4 on
Multiple Object Tracking
on KITTI Tracking test
Learning by Cheating
We first train an agent that has access to privileged information.
Ranked #16 on
Autonomous Driving
on CARLA Leaderboard
A Multigrid Method for Efficiently Training Video Models
We empirically demonstrate a general and robust grid schedule that yields a significant out-of-the-box training speedup without a loss in accuracy for different models (I3D, non-local, SlowFast), datasets (Kinetics, Something-Something, Charades), and training settings (with and without pre-training, 128 GPUs or 1 GPU).
Ranked #1 on
Video Classification
on Kinetics
Does computer vision matter for action?
Thus the central question of our work: Does computer vision matter for action?
Monocular Plan View Networks for Autonomous Driving
Convolutions on monocular dash cam videos capture spatial invariances in the image plane but do not explicitly reason about distances and depth.
Don't let your Discriminator be fooled
We experimentally show that any GAN objective, including Wasserstein GANs, benefit from adversarial robustness both quantitatively and qualitatively.
Objects as Points
We model an object as a single point --- the center point of its bounding box.
Bottom-up Object Detection by Grouping Extreme and Center Points
With the advent of deep learning, object detection drifted from a bottom-up to a top-down recognition problem.
Ranked #128 on
Object Detection
on COCO minival
Long-Term Feature Banks for Detailed Video Understanding
To understand the world, we humans constantly need to relate the present to the past, and put events in context.
Ranked #4 on
Egocentric Activity Recognition
on EPIC-KITCHENS-55
Joint Monocular 3D Vehicle Detection and Tracking
The framework can not only associate detections of vehicles in motion over time, but also estimate their complete 3D bounding box information from a sequence of 2D images captured on a moving platform.
Ranked #12 on
Multiple Object Tracking
on KITTI Tracking test
Assessing Generalization in Deep Reinforcement Learning
Our aim is to catalyze community-wide progress on generalization in deep RL.
Out-of-Distribution Generalization
reinforcement-learning
+1
Video Compression through Image Interpolation
An ever increasing amount of our digital communication, media consumption, and content creation revolves around videos.
Compressed Video Action Recognition
), we propose to train a deep network directly on the compressed video.
Ranked #46 on
Action Classification
on Charades
(using extra training data)
Sampling Matters in Deep Embedding Learning
In addition, we show that a simple margin based loss is sufficient to outperform all other loss functions.
Ranked #5 on
Image Retrieval
on CARS196
Generative Visual Manipulation on the Natural Image Manifold
Realistic image manipulation is challenging because it requires modifying the image appearance in a user-controlled way, while preserving the realism of the result.
Adversarial Feature Learning
The ability of the Generative Adversarial Networks (GANs) framework to learn generative models mapping from simple latent distributions to arbitrarily complex data distributions has been demonstrated empirically, with compelling results showing that the latent space of such generators captures semantic variation in the data distribution.
Learning Dense Correspondence via 3D-guided Cycle Consistency
We use ground-truth synthetic-to-synthetic correspondences, provided by the rendering engine, to train a ConvNet to predict synthetic-to-real, real-to-real and real-to-synthetic correspondences that are cycle-consistent with the ground-truth.
Constrained Structured Regression with Convolutional Neural Networks
We present a regression framework which models the output distribution of neural networks.
Data-dependent Initializations of Convolutional Neural Networks
Convolutional Neural Networks spread through computer vision like a wildfire, impacting almost all visual tasks imaginable.
Learning Data-driven Reflectance Priors for Intrinsic Image Decomposition
We propose a data-driven approach for intrinsic image decomposition, which is the process of inferring the confounding factors of reflectance and shading in an image.
Learning a Discriminative Model for the Perception of Realism in Composite Images
What makes an image appear realistic?
Constrained Convolutional Neural Networks for Weakly Supervised Segmentation
We propose Constrained CNN (CCNN), a method which uses a novel loss function to optimize for any set of linear constraints on the output space (i. e. predicted label distribution) of a CNN.
Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
In this paper, we consider fully connected CRF models defined on the complete set of pixels in an image.
